Operating Imperfect AI: Reliability Drift and Human Congestion

The deployment of machine learning in high-stakes services relies on human-in-the-loop'' architectures to mitigate algorithmic uncertainty. However, existing static policies fail to address a fundamental tension: algorithms suffer from stochastic reliability drift,’’ while human override capacity is scarce and congestible. We formulate the management of such systems as a dynamic queueing control problem. The system state is defined by the tuple (queue backlog, reliability regime), and the control variable is a state-dependent risk threshold. We prove that the optimal escalation policy is driven by the endogenous Shadow Price of Capacity.'' We establish two key structural monotonicity results: (i) Congestion Shedding, where the threshold rises with backlog to sacrifice marginal accuracy for responsiveness; and (ii) Safety Buffering, where the threshold lowers during drift to use the queue as a risk capacitor.’’ Furthermore, we identify a critical ``Capacity Phase Transition’’ in the arrival-drift parameter space, beyond which no policy can maintain safety standards without causing structural system failure (infinite queues). Our results provide rigorous operational rules for managing the interface between imperfect algorithms and congested experts.

💡 Research Summary

The paper addresses the fundamental tension in human‑in‑the‑loop (HITL) services between stochastic reliability drift of machine‑learning models and the congestible, finite capacity of human experts. The authors formulate the management problem as a continuous‑time stochastic control (CTMDP) where the system state is a pair (q, θ): q denotes the current backlog in a human service queue (modeled as an M/M/m system) and θ denotes the reliability regime of the automated classifier, evolving as a finite‑state Markov chain. Each incoming task arrives as a Poisson process with rate λ, carries an observable risk score s∈