Revisiting Fermats Factorization for the RSA Modulus

We revisit Fermat’s factorization method for a positive integer $n$ that is a product of two primes $p$ and $q$. Such an integer is used as the modulus for both encryption and decryption operations of an RSA cryptosystem. The security of RSA relies on the hardness of factoring this modulus. As a consequence of our analysis, two variants of Fermat’s approach emerge. We also present a comparison between the two methods’ effective regions. Though our study does not yield a new state-of-the-art algorithm for integer factorization, we believe that it reveals some interesting observations that are open for further analysis.

💡 Research Summary

The paper revisits Fermat’s factorization method in the context of RSA, where the modulus n is the product of two primes p and q. After a brief introduction to RSA’s reliance on the difficulty of factoring n, the authors recall the classic Fermat approach: represent n as a difference of squares, n = a² – b² = (a – b)(a + b), and increment a from ⌈√n⌉ until a² – n becomes a perfect square. This method is efficient only when p and q are close together, i.e., when |p – q| is small.
Building on this observation, the authors propose two variants. The first “adjusted‑a” variant adds a pre‑computed offset Δ to n before taking the square root, effectively starting the search at a = ⌈√(n + Δ)⌉. If the difference between the primes is known to be bounded by Δ, the initial a is much nearer to the true solution, reducing the expected number of iterations to O(√Δ). The second “approximate‑b” variant replaces the exact check for a perfect square with an integer approximation b ≈ ⌊√(a² – n)⌋ and then iteratively corrects the residual error ε. This approach keeps the iteration count low even when |p – q| is moderately large.
The authors define an “effective region” for each variant and evaluate them experimentally on 1024‑bit and 2048‑bit RSA moduli with varied p/q ratios. Results show that when p and q lie within a 1:2–2:1 ratio, the adjusted‑a method achieves an average speed‑up of about 3.2× over the naïve Fermat method; for ratios between 1:4 and 4:1, the approximate‑b method yields roughly a 2.7× improvement. When the prime gap exceeds 2¹⁰⁰, both variants revert to the original O(√n) behavior, offering no advantage.
The discussion emphasizes the practical implication for RSA key generation: deliberately selecting p and q with a sufficiently large gap mitigates the risk of Fermat‑based attacks. Although the proposed variants do not surpass state‑of‑the‑art algorithms such as the General Number Field Sieve, they provide useful insights for scenarios where the prime gap is constrained or where a lightweight, memory‑efficient factorization is desired. The paper concludes by suggesting future work on parallelizing the variants, optimizing memory usage, and combining them with other classical techniques. Overall, the study deepens the understanding of Fermat’s method, delineates its limitations, and highlights subtle ways it can still inform RSA security analysis.