Emergent Alignment via Competition

Aligning AI systems with human values remains a fundamental challenge, but does our inability to create perfectly aligned models preclude obtaining the benefits of alignment? We study a strategic setting where a human user interacts with multiple differently misaligned AI agents, none of which are individually well-aligned. Our key insight is that when the users utility lies approximately within the convex hull of the agents utilities, a condition that becomes easier to satisfy as model diversity increases, strategic competition can yield outcomes comparable to interacting with a perfectly aligned model. We model this as a multi-leader Stackelberg game, extending Bayesian persuasion to multi-round conversations between differently informed parties, and prove three results: (1) when perfect alignment would allow the user to learn her Bayes-optimal action, she can also do so in all equilibria under the convex hull condition (2) under weaker assumptions requiring only approximate utility learning, a non-strategic user employing quantal response achieves near-optimal utility in all equilibria and (3) when the user selects the best single AI after an evaluation period, equilibrium guarantees remain near-optimal without further distributional assumptions. We complement the theory with two sets of experiments.

💡 Research Summary

The paper tackles the long‑standing AI alignment problem from a market‑centric perspective. Instead of trying to build a single perfectly aligned model, the authors consider a setting where a human user (Alice) can converse with many AI providers (Bob₁,…,Bob_k), each of which may be misaligned in its own way. The central assumption is that Alice’s true utility function u_A lies approximately inside the convex hull of the providers’ utility functions U_i; formally, there exist non‑negative weights w_i and a translation constant c such that the supremum over actions a and states y of |∑_i w_i U_i(a,y)+c − u_A(a,y)| is bounded by a small ε. This “approximate market alignment” condition becomes easier to satisfy as the number and diversity of models increase.

The interaction is modeled as a multi‑leader Stackelberg (or Bayesian persuasion) game. Each provider commits to a communication rule (a conversational policy) before Alice acts. Alice, knowing all rules, engages in a multi‑round dialogue, updates her posterior belief about the hidden state y using both her private observations x_A and the information disclosed by the providers (x_B), and finally selects an action a that maximizes her expected utility. Providers receive utility based on Alice’s eventual action, so they strategically choose their communication rules. The equilibrium concept is a Nash equilibrium over the providers’ commitments, with Alice best‑responding given the induced posterior.

Three main theoretical results are proved under increasingly weak assumptions:

1. Ideal Bayesian Learning (Section 3). If a perfectly aligned provider could induce Alice to learn the Bayes‑optimal action a* = argmax_a E