DHEA-MECD: An Embodied Intelligence-Powered DRL Algorithm for AUV Tracking in Underwater Environments with High-Dimensional Features

In recent years, autonomous underwater vehicle (AUV) systems have demonstrated significant potential in complex marine exploration. However, effective AUV-based tracking remains challenging in realistic underwater environments characterized by high-dimensional features, including coupled kinematic states, spatial constraints, time-varying environmental disturbances, etc. To address these challenges, this paper proposes a hierarchical embodied-intelligence (EI) architecture for underwater multi-target tracking with AUVs in complex underwater environments. Built upon this architecture, we introduce the Double-Head Encoder-Attention-based Multi-Expert Collaborative Decision (DHEA-MECD), a novel Deep Reinforcement Learning (DRL) algorithm designed to support efficient and robust multi-target tracking. Specifically, in DHEA-MECD, a Double-Head Encoder-Attention-based information extraction framework is designed to semantically decompose raw sensory observations and explicitly model complex dependencies among heterogeneous features, including spatial configurations, kinematic states, structural constraints, and stochastic perturbations. On this basis, a motion-stage-aware multi-expert collaborative decision mechanism with Top-k expert selection strategy is introduced to support stage-adaptive decision-making. Furthermore, we propose the DHEA-MECD-based underwater multitarget tracking algorithm to enable AUV smart, stable, and anti-interference multi-target tracking. Extensive experimental results demonstrate that the proposed approach achieves superior tracking success rates, faster convergence, and improved motion optimality compared with mainstream DRL-based methods, particularly in complex and disturbance-rich marine environments.

💡 Research Summary

The paper tackles the challenging problem of autonomous multi‑target tracking for underwater vehicles operating in realistic marine environments that are characterized by high‑dimensional, heterogeneous observations and time‑varying disturbances such as ocean currents and acoustic noise. To overcome the limitations of conventional deep‑reinforcement‑learning (DRL) approaches—namely perception fragmentation and a single monolithic policy that cannot adapt to different motion stages—the authors propose a hierarchical Embodied Intelligence (EI) architecture together with a novel DRL algorithm called Double‑Head Encoder‑Attention‑based Multi‑Expert Collaborative Decision (DHEA‑MECD).

The EI architecture separates the perception‑to‑action pipeline into three layers: embodied perception, embodied decision, and embodied execution. This decoupling enables the system to treat high‑level tracking logic independently from low‑level hydrodynamic actuation, thereby improving robustness to environmental perturbations.

DHEA‑MECD introduces two parallel encoder heads. One head processes spatial‑geometric features (e.g., relative positions, angles), while the other processes dynamic and environmental variables (e.g., current velocity, noise levels). Both heads feed into a multi‑head self‑attention module that explicitly learns cross‑modal dependencies, mitigating the “perception fragmentation” problem that plagues flat‑vector DRL pipelines.

On top of the enriched representation, a motion‑stage‑aware multi‑expert decision mechanism is built. Several expert networks are pre‑trained for distinct motion regimes such as rapid pursuit, precision station‑keeping, and emergency collision avoidance. During online operation, a Top‑k expert selection strategy evaluates the current state and motion stage, picks the k most relevant experts, and aggregates their Q‑values (or policy outputs) to produce the final action. This modular expert system allows the agent to balance exploration and exploitation across heterogeneous action spaces (both discrete and continuous) without sacrificing real‑time performance.

The authors also provide a detailed physical modeling of the underwater domain. Ocean currents are represented by a finite‑dimensional Gaussian radial‑basis‑function (RBF) field with time‑varying weights, while acoustic noise is decomposed into vehicle‑induced, biological, geological, and turbulence components, each modeled with appropriate stochastic processes (Gaussian, α‑stable, colored Gaussian, AR(1)). These models are integrated into the AUV dynamics, yielding a realistic Markov Decision Process (MDP) for training and evaluation.

Experimental validation is conducted in a high‑fidelity simulator that incorporates the aforementioned current and noise models. DHEA‑MECD is compared against state‑of‑the‑art DRL baselines including Soft Actor‑Critic (SAC), Deep Deterministic Policy Gradient (DDPG), and Proximal Policy Optimization (PPO). Performance metrics cover tracking success rate, average convergence episodes, and trajectory optimality (path length and energy consumption). Results show that DHEA‑MECD achieves 12–18 % higher success rates and roughly 30 % faster convergence in disturbance‑rich scenarios, while also reducing energy usage. Ablation studies indicate that a Top‑k value of 3 with five experts offers the best trade‑off between computational overhead and tracking performance.

The paper acknowledges several limitations. Expert pre‑training relies on simulated data, leaving open the question of transferability to real‑world ocean datasets. The double‑head encoder and multi‑head attention increase computational load; the authors do not provide detailed profiling or hardware specifications to assess feasibility on embedded AUV processors. The Top‑k selection is based solely on Q‑value rankings, which may be vulnerable to sensor failures or high uncertainty. Finally, reproducibility suffers from insufficient disclosure of simulator parameters, random seeds, and hyper‑parameter settings.

In summary, the work makes a substantial contribution by integrating structured attention‑based perception with a modular expert decision framework within an embodied‑intelligence paradigm. It demonstrates that such a combination can effectively handle high‑dimensional underwater observations and dynamically changing motion requirements, outperforming conventional DRL methods. Future research directions include real‑world sea trials, model compression for on‑board deployment, and uncertainty‑aware expert selection (e.g., Bayesian or risk‑sensitive criteria) to further enhance robustness and applicability.