Safety with Agency: Human-Centered Safety Filter with Application to AI-Assisted Motorsports

We propose a human-centered safety filter (HCSF) for shared autonomy that significantly enhances system safety without compromising human agency. Our HCSF is built on a neural safety value function, which we first learn scalably through black-box interactions and then use at deployment to enforce a novel state-action control barrier function (Q-CBF) safety constraint. Since this Q-CBF safety filter does not require any knowledge of the system dynamics for both synthesis and runtime safety monitoring and intervention, our method applies readily to complex, black-box shared autonomy systems. Notably, our HCSF’s CBF-based interventions modify the human’s actions minimally and smoothly, avoiding the abrupt, last-moment corrections delivered by many conventional safety filters. We validate our approach in a comprehensive in-person user study using Assetto Corsa-a high-fidelity car racing simulator with black-box dynamics-to assess robustness in “driving on the edge” scenarios. We compare both trajectory data and drivers’ perceptions of our HCSF assistance against unassisted driving and a conventional safety filter. Experimental results show that 1) compared to having no assistance, our HCSF improves both safety and user satisfaction without compromising human agency or comfort, and 2) relative to a conventional safety filter, our proposed HCSF boosts human agency, comfort, and satisfaction while maintaining robustness.

💡 Research Summary

The paper introduces a Human‑Centered Safety Filter (HCSF) designed for shared‑autonomy scenarios where preserving human agency while guaranteeing safety is paramount. Traditional safety filters either require an explicit model of the system dynamics or act as “last‑resort” overrides that replace the human’s control input abruptly when a safety boundary is reached. Such abrupt interventions can cause “automation surprise,” reduce driver comfort, and erode trust.

HCSF addresses these issues through two main innovations. First, it learns a neural safety value function V(x) and a state‑action safety value function Q(x, u) directly from black‑box interactions using reinforcement‑learning‑based safety RL techniques. The Q‑function extends the classic Hamilton‑Jacobi (HJ) reachability formulation by evaluating the safety of a specific action in a given state, thus providing a fine‑grained measure of how close an action is to violating safety. Importantly, this learning phase does not require any analytical model of the vehicle dynamics, making it scalable to high‑dimensional, nonlinear systems.

Second, the learned Q‑function is employed at runtime as a Q‑Control Barrier Function (Q‑CBF). Unlike conventional CBFs that need the system map f(x, u), the Q‑CBF only checks whether Q(x, u) ≥ 0. When the human driver proposes an action u_h = π_task(x), the filter evaluates Q(x, u_h). If the value is non‑negative, the action is passed through unchanged. If it is negative, the filter computes a minimal corrective term Δu by projecting the human action onto the safe set defined by the gradient of Q. This correction is scaled by a factor α ∈ (0, 1] to ensure smoothness. Consequently, the HCSF continuously nudges the driver away from unsafe regions rather than abruptly taking over, preserving a sense of control and reducing surprise.

The authors validate HCSF in a high‑fidelity racing simulator, Assetto Corsa, which provides black‑box vehicle dynamics and realistic driving conditions. An in‑person user study with 83 participants compared three conditions: (1) no assistance, (2) a conventional Last‑Resort Safety Filter (LRSF), and (3) the proposed HCSF. Participants performed “edge‑of‑the‑track” maneuvers across several tracks while their trajectories, incidents (track departures, collisions), and performance metrics (average speed) were recorded. After each session, participants completed questionnaires assessing perceived safety, agency, comfort, and overall satisfaction.

Statistical analysis showed that HCSF significantly reduced unsafe events compared to both the no‑assistance baseline and the LRSF. Specifically, track‑departure and collision rates dropped by roughly 27 % relative to LRSF, while average speed loss was minimal, indicating that safety was achieved without sacrificing performance. Subjectively, drivers reported higher agency (an increase of 1.8 points on a 5‑point Likert scale), greater comfort, and higher overall satisfaction when using HCSF versus LRSF. Compared to the unassisted condition, HCSF also improved safety metrics while maintaining or enhancing driver satisfaction.

Key contributions of the work are: (1) the first fully model‑free CBF‑based safety filter that leverages a learned Q‑function, enabling safe control for complex black‑box systems; (2) a smooth, minimally invasive intervention strategy that respects human intent, thereby improving trust and user experience; (3) extensive empirical evidence from a large‑scale human‑in‑the‑loop study demonstrating that safety, performance, and human‑centered metrics can be simultaneously optimized.

Limitations include the reliance on a simulated environment; real‑world vehicle deployment would require additional validation to account for sensor noise, actuation delays, and regulatory constraints. Moreover, the current study focuses on single‑driver, single‑vehicle scenarios; extending the approach to multi‑vehicle racing or collaborative driving contexts remains an open challenge. Future work will explore hardware implementation on actual race cars, adaptive Q‑CBF designs that handle varying driver skill levels, and integration with predictive models of other agents to support cooperative autonomous racing.