A Survey on Authentication and Key Agreement Protocols in Heterogeneous Networks

Unlike current closed systems such as 2nd and 3rd generations where the core network is controlled by a sole network operator, multiple network operators will coexist and manage the core network in Next Generation Networks (NGNs). This open architecture and the collaboration between different network operators will support ubiquitous connectivity and thus enhances users’ experience. However, this brings to the fore certain security issues which must be addressed, the most important of which is the initial Authentication and Key Agreement (AKA) to identify and authorize mobile nodes on these various networks. This paper looks at how existing research efforts the HOKEY WG, Mobile Ethernet and 3GPP frameworks respond to this new environment and provide security mechanisms. The analysis shows that most of the research had realized the openness of the core network and tried to deal with it using different methods. These methods will be extensively analysed in order to highlight their strengths and weaknesses.

💡 Research Summary

The paper addresses the emerging security challenge of initial authentication and key agreement (AKA) in next‑generation networks (NGNs), where the core infrastructure is no longer owned by a single operator but is shared among multiple operators. This openness promises ubiquitous connectivity and improved user experience, yet it creates a new attack surface because mobile nodes must be authenticated and authorized across heterogeneous administrative domains.

The authors review three major research and standardization efforts that aim to provide AKA mechanisms for such an environment: the IETF HOKEY Working Group, the Mobile Ethernet framework, and the 3GPP extensions to the traditional AKA protocol. For each approach the paper describes the underlying cryptographic primitives, the protocol flow, and the assumptions about trust and key management.

HOKEY proposes a certificate‑based mutual authentication combined with a Diffie‑Hellman key exchange. The design offers strong cryptographic guarantees and a clear trust‑anchor hierarchy, but it relies on a heavyweight public‑key infrastructure (PKI). Managing certificates, handling revocation, and scaling the PKI across many operators introduce significant operational complexity and cost.

Mobile Ethernet builds on the existing Ethernet ecosystem. It reuses Extensible Authentication Protocol (EAP) variants such as EAP‑TLS or EAP‑TTLS for the authentication phase and then protects the data plane with MACsec. This approach is attractive because it can be deployed with minimal changes to current Ethernet equipment and provides low‑latency hand‑over. However, frequent re‑keying can cause noticeable latency and bandwidth overhead, and MACsec key management remains a non‑trivial problem in a multi‑operator scenario.

The 3GPP solution extends the well‑known AKA protocol to support Inter‑Operator Authentication (IOA). By coupling SIM‑based symmetric keys with public‑key certificates, a mobile device can authenticate simultaneously with several operators while preserving user transparency. This method balances security and usability, but it pushes the computational limits of SIM cards and requires additional standardization to ensure compatibility with legacy 2G/3G/4G infrastructures.

The comparative analysis highlights three axes of trade‑off: security, performance, and scalability. HOKEY maximizes security at the expense of operational overhead; Mobile Ethernet optimizes performance but offers a lower security margin; 3GPP seeks a middle ground but must address PKI integration and SIM constraints. Common shortcomings across all proposals include: (1) the lack of a unified trust‑anchor model for cross‑operator verification, (2) insufficient mechanisms for automated certificate revocation and re‑issuance, (3) limited post‑compromise key recovery procedures, and (4) inadequate consideration of future quantum‑resistant algorithms.

To overcome these gaps, the authors propose several research directions. A blockchain‑based distributed trust model could allow each operator to maintain its own trust anchor while still providing a globally verifiable ledger of authentication events. Lightweight post‑quantum cryptography (e.g., lattice‑based key exchange) could future‑proof the AKA process against quantum adversaries. Integrating AI‑driven anomaly detection with dynamic key renegotiation would enable real‑time response to emerging threats. Finally, a meta‑framework for policy harmonization among operators could automate certificate lifecycle management and streamline cross‑domain key distribution.

In conclusion, the paper argues that effective AKA in heterogeneous NGNs must go beyond pure cryptographic strength. It must also address operational efficiency, scalability across many operators, and robust trust management. By systematically evaluating existing proposals and outlining concrete avenues for improvement, the work provides a roadmap for future standardization and real‑world deployment of secure, interoperable authentication mechanisms in the open core networks of the next generation.