An authentication scheme based on the twisted conjugacy problem

The conjugacy search problem in a group $G$ is the problem of recovering an $x \in G$ from given $g \in G$ and $h=x^{-1}gx$. The alleged computational hardness of this problem in some groups was used in several recently suggested public key exchange protocols, including the one due to Anshel, Anshel, and Goldfeld, and the one due to Ko, Lee et al. Sibert, Dehornoy, and Girault used this problem in their authentication scheme, which was inspired by the Fiat-Shamir scheme involving repeating several times a three-pass challenge-response step. In this paper, we offer an authentication scheme whose security is based on the apparent hardness of the twisted conjugacy search problem, which is: given a pair of endomorphisms (i.e., homomorphisms into itself) phi, \psi of a group G and a pair of elements w, t \in G, find an element s \in G such that t = \psi(s^{-1}) w \phi(s) provided at least one such s exists. This problem appears to be very non-trivial even for free groups. We offer here another platform, namely, the semigroup of all 2x2 matrices over truncated one-variable polynomials over F_2, the field of two elements, with transposition used instead of inversion in the equality above.

💡 Research Summary

The paper introduces an authentication protocol whose security rests on the “twisted conjugacy search problem”, a generalization of the classical conjugacy search problem in non‑commutative groups. In the twisted version one is given two endomorphisms ϕ and ψ of a group G together with elements w and t, and the task is to find an element s such that

  t = ψ(s⁻¹) · w · ϕ(s).

If such an s exists, the pair (ϕ, ψ, w, t) can be used as a public key while s is the secret. The authors argue that, unlike the ordinary conjugacy problem (which corresponds to ϕ = ψ = identity), the presence of two independent endomorphisms makes it much harder to apply length‑based or linear‑algebraic attacks, because the attacker cannot easily cancel the contributions of ϕ(s) and ψ(s⁻¹).

Protocol description.
The scheme follows the Fiat‑Shamir three‑pass paradigm, repeated k times to obtain a forgery probability of 2⁻ᵏ. In a single round:

1. Alice (prover) chooses a random r ∈ G and sends the commitment

  u = ψ(r⁎) · t · ϕ(r)

where ⁎ denotes a fixed antihomomorphism of G (i.e., (ab)⁎ = b⁎a⁎).

2. Bob (verifier) sends a random challenge bit c ∈ {0,1}.

3. If c = 0, Alice reveals v = r; Bob checks u = ψ(v⁎) · t · ϕ(v).
   If c = 1, Alice reveals v = s r; Bob checks u = ψ(v⁎) · w · ϕ(v).

Because u is constructed from r and the public data, both checks succeed if and only if the secret s satisfies the defining equation t = ψ(s⁎) · w · ϕ(s). The protocol reveals no information about s beyond the fact that such an s exists.

Choice of platform.
The authors propose to instantiate G as the semigroup of all 2 × 2 matrices over the ring of N‑truncated polynomials F₂