Aegis: Towards Governance, Integrity, and Security of AI Voice Agents

With the rapid advancement and adoption of Audio Large Language Models (ALLMs), voice agents are now being deployed in high-stakes domains such as banking, customer service, and IT support. However, their vulnerabilities to adversarial misuse still remain unexplored. While prior work has examined aspects of trustworthiness in ALLMs, such as harmful content generation and hallucination, systematic security evaluations of voice agents are still lacking. To address this gap, we propose Aegis, a red-teaming framework for the governance, integrity, and security of voice agents. Aegis models the realistic deployment pipeline of voice agents and designs structured adversarial scenarios of critical risks, including privacy leakage, privilege escalation, resource abuse, etc. We evaluate the framework through case studies in banking call centers, IT Support, and logistics. Our evaluation shows that while access controls mitigate data-level risks, voice agents remain vulnerable to behavioral attacks that cannot be addressed through access restrictions alone, even under strict access controls. We observe systematic differences across model families, with open-weight models exhibiting higher susceptibility, underscoring the need for layered defenses that combine access control, policy enforcement, and behavioral monitoring to secure next-generation voice agents.

💡 Research Summary

The paper introduces Aegis, a systematic red‑team framework designed to evaluate the governance, integrity, and security of AI voice agents powered by Audio Large Language Models (ALLMs). While prior work on ALLM trustworthiness has focused on harmful content generation, hallucination, and model‑level robustness, it has largely ignored the unique attack surfaces introduced by multimodal, real‑time spoken interactions. Aegis fills this gap by modeling the full deployment pipeline of voice agents and constructing realistic adversarial scenarios drawn from the MITRE ATT&CK matrix.

Three high‑stakes domains are selected as testbeds: (1) banking call centers, (2) IT support desks, and (3) logistics dispatch services. Each domain comprises an authentication phase (PINs, security questions, multi‑factor prompts) and a service phase (account balance queries, password resets, shipment rescheduling, etc.). The framework defines five adversarial scenarios: authentication bypass, privacy leakage, resource abuse, privilege escalation, and data poisoning. These scenarios capture how an attacker might exploit voice spoofing, social engineering cues, or malicious prompt injection to compromise the system.

A key methodological contribution is the comparison of two database‑access paradigms. In the direct‑read mode, the voice agent can query raw records, which makes it vulnerable to immediate data exfiltration once authentication is bypassed. In the intermediary‑query mode, the agent is limited to pre‑defined API calls that return aggregated results, reflecting a common industry practice of restricting raw data exposure. Experiments across the three domains reveal that while strict access controls effectively mitigate authentication‑bypass and privacy‑leakage attacks, they do not prevent behavioral threats such as privilege escalation, instruction poisoning, and resource abuse.

The empirical results also uncover a systematic difference between model families. Open‑weight models (e.g., Llama‑Audio variants) exhibit higher susceptibility to prompt‑injection and data‑poisoning attacks compared to closed‑source, commercially‑hosted models (OpenAI, Gemini). This suggests that the proprietary safety layers employed by vendors provide an additional, albeit incomplete, shield against adversarial manipulation.

Attacker personas (e.g., fraudsters, insider threats) and subtle voice cues (gender, accent, intonation) were found to have limited impact when robust operational policies are in place, indicating that policy clarity and multi‑factor authentication can dampen social‑engineering vectors. However, the study highlights a critical blind spot: behavioral monitoring. Without real‑time detection of anomalous dialogue patterns or repeated high‑risk commands, a compromised agent can continue to execute malicious actions, leading to financial loss, system overload, or data exfiltration.

Consequently, the authors advocate a layered defense strategy:

1. Access control & authentication to block straightforward credential‑based attacks.
2. Policy enforcement at the prompt level, including blacklists of dangerous commands, context‑aware constraints, and rate limiting.
3. Behavioral monitoring and anomaly detection, leveraging speech‑level signals and interaction histories to flag and abort suspicious sessions.

The paper concludes that securing next‑generation AI voice agents requires integrating these layers rather than relying on any single mechanism. It also calls for future research on hardening open‑weight models, developing more sophisticated real‑time behavior analytics, and expanding the adversarial taxonomy to cover emerging multimodal threats.