End-to-end Optimization of Belief and Policy Learning in Shared Autonomy Paradigms

Shared autonomy systems require principled methods for inferring user intent and determining appropriate assistance levels. This is a central challenge in human-robot interaction, where systems must be successful while being mindful of user agency. Previous approaches relied on static blending ratios or separated goal inference from assistance arbitration, leading to suboptimal performance in unstructured environments. We introduce BRACE (Bayesian Reinforcement Assistance with Context Encoding), a novel framework that fine-tunes Bayesian intent inference and context-adaptive assistance through an architecture enabling end-to-end gradient flow between intent inference and assistance arbitration. Our pipeline conditions collaborative control policies on environmental context and complete goal probability distributions. We provide analysis showing (1) optimal assistance levels should decrease with goal uncertainty and increase with environmental constraint severity, and (2) integrating belief information into policy learning yields a quadratic expected regret advantage over sequential approaches. We validated our algorithm against SOTA methods (IDA, DQN) using a three-part evaluation progressively isolating distinct challenges of end-effector control: (1) core human-interaction dynamics in a 2D human-in-the-loop cursor task, (2) non-linear dynamics of a robotic arm, and (3) integrated manipulation under goal ambiguity and environmental constraints. We demonstrate improvements over SOTA, achieving 6.3% higher success rates and 41% increased path efficiency, and 36.3% success rate and 87% path efficiency improvement over unassisted control. Our results confirmed that integrated optimization is most beneficial in complex, goal-ambiguous scenarios, and is generalizable across robotic domains requiring goal-directed assistance, advancing the SOTA for adaptive shared autonomy.

💡 Research Summary

The paper tackles a central problem in shared‑autonomy: how to infer a user’s intent while simultaneously deciding how much assistance to provide. Existing approaches typically treat these two sub‑problems separately—either using fixed blending ratios, a two‑stage pipeline (goal inference followed by arbitration), or relying on MAP estimates of the goal distribution. Such separations lead to sub‑optimal performance, especially in unstructured environments where goal uncertainty is high and environmental constraints (e.g., obstacles, precision requirements) vary over time.

To address this, the authors introduce BRACE (Bayesian Reinforcement Assistance with Context Encoding), a novel end‑to‑end framework that jointly learns a Bayesian intent inference module and a context‑adaptive assistance policy. The Bayesian module maintains a full probability distribution b(g) over a predefined set of goals G, updating it online from observed human actions using a recursive Bayesian filter. Unlike prior work that collapses the distribution to a MAP estimate, BRACE explicitly retains the distribution’s entropy and concentration (parameter λ), allowing the system to reason about the degree of uncertainty.

The assistance policy is modeled as a continuous blending factor γ∈