Deciding security properties for cryptographic protocols. Application to key cycles

There is a large amount of work dedicated to the formal verification of security protocols. In this paper, we revisit and extend the NP-complete decision procedure for a bounded number of sessions. We use a, now standard, deducibility constraints formalism for modeling security protocols. Our first contribution is to give a simple set of constraint simplification rules, that allows to reduce any deducibility constraint system to a set of solved forms, representing all solutions (within the bound on sessions). As a consequence, we prove that deciding the existence of key cycles is NP-complete for a bounded number of sessions. The problem of key-cycles has been put forward by recent works relating computational and symbolic models. The so-called soundness of the symbolic model requires indeed that no key cycle (e.g., enc(k,k)) ever occurs in the execution of the protocol. Otherwise, stronger security assumptions (such as KDM-security) are required. We show that our decision procedure can also be applied to prove again the decidability of authentication-like properties and the decidability of a significant fragment of protocols with timestamps.

💡 Research Summary

The paper revisits and extends the well‑known NP‑complete decision procedure for security protocol analysis when the number of sessions is bounded. It adopts the standard deducibility‑constraint formalism, where protocol messages are represented as terms built from function symbols (pairing, symmetric and asymmetric encryption, signatures, hashing) and variables, and the intruder’s capabilities are captured by a set of deduction rules. The authors’ first major contribution is a compact set of constraint‑simplification rules that transform any deducibility‑constraint system into a collection of “solved forms”. These solved forms preserve all solutions of the original system, not merely its satisfiability, and the transformation terminates in polynomial time despite being nondeterministic. A solved form can be seen as a finite graph that symbolically encodes all possible message traces generated by the protocol and the attacker’s actions.

Using this normal form, the paper tackles the problem of key cycles – situations where a key encrypts itself (enc(k,k)) or where a set of keys encrypt each other forming a cycle (e.g., enc(k1,k2), enc(k2,k1)). Such cycles break the usual soundness assumptions of the symbolic model; their presence would require stronger cryptographic guarantees such as key‑dependent‑message (KDM) security. The authors show that detecting the generation of key cycles in a bounded‑session execution is NP‑complete. Their method works for several definitions of key cycles found in the literature, including strict cycles (as in Abadi‑Rogaway), non‑strict cycles (as in Laud), and ordered key relations (as in Backes et al.). The detection reduces to a simple cycle‑finding problem on the graph extracted from the solved form.

The second contribution demonstrates the versatility of the same procedure for other security properties. The authors introduce a small logic capable of expressing authentication‑like properties (e.g., agreement, secrecy, freshness) and prove that any formula in this logic can be decided by checking satisfaction on solved forms, again in nondeterministic polynomial time. They also extend the approach to protocols that use timestamps. By treating timestamps as integer‑valued terms and restricting comparisons to a decidable fragment, they obtain NP‑completeness for the verification of timestamp‑based freshness and replay‑prevention properties, covering a larger class than previously identified (e.g., Bozga et al.).

Throughout the paper the authors emphasize that their algorithm can be implemented by adapting the existing AVISPA tool, which already incorporates the original Rusinowitch‑Turuani decision procedure for secrecy and authentication. Because the simplification rules are few and modular, extending the tool to handle key‑cycle detection, timestamp reasoning, or new cryptographic primitives requires only modest engineering effort.

In summary, the work provides a unified, polynomial‑time‑bounded framework for deciding a broad spectrum of security properties—including the previously unaddressed key‑cycle problem—by reducing them to the analysis of solved forms derived from deducibility constraints. This advances both the theoretical understanding of the complexity of protocol verification and offers a practical pathway for integrating these results into automated verification tools.