Solving Medium-Density Subset Sum Problems in Expected Polynomial Time: An Enumeration Approach

The subset sum problem (SSP) can be briefly stated as: given a target integer $E$ and a set $A$ containing $n$ positive integer $a_j$, find a subset of $A$ summing to $E$. The \textit{density} $d$ of an SSP instance is defined by the ratio of $n$ to $m$, where $m$ is the logarithm of the largest integer within $A$. Based on the structural and statistical properties of subset sums, we present an improved enumeration scheme for SSP, and implement it as a complete and exact algorithm (EnumPlus). The algorithm always equivalently reduces an instance to be low-density, and then solve it by enumeration. Through this approach, we show the possibility to design a sole algorithm that can efficiently solve arbitrary density instance in a uniform way. Furthermore, our algorithm has considerable performance advantage over previous algorithms. Firstly, it extends the density scope, in which SSP can be solved in expected polynomial time. Specifically, It solves SSP in expected $O(n\log{n})$ time when density $d \geq c\cdot \sqrt{n}/\log{n}$, while the previously best density scope is $d \geq c\cdot n/(\log{n})^{2}$. In addition, the overall expected time and space requirement in the average case are proven to be $O(n^5\log n)$ and $O(n^5)$ respectively. Secondly, in the worst case, it slightly improves the previously best time complexity of exact algorithms for SSP. Specifically, the worst-case time complexity of our algorithm is proved to be $O((n-6)2^{n/2}+n)$, while the previously best result is $O(n2^{n/2})$.

💡 Research Summary

The paper tackles the classic Subset Sum Problem (SSP), where given a target integer E and a set A of n positive integers, one must decide whether a subset of A sums exactly to E. The difficulty of an SSP instance is traditionally measured by its density d = n / m, where m is the bit‑length of the largest element in A. High‑density instances (large n relative to m) are known to be “easy” on average, while low‑density instances are believed to require exponential time.

Main contribution
The authors introduce a new exact algorithm, called EnumPlus, which combines a preprocessing step that transforms any instance into an effectively low‑density form with a refined enumeration scheme that exploits two statistical/structural properties of subset sums. The algorithm is claimed to solve SSP in expected polynomial time for a much broader density range than previously known: specifically, when d ≥ c·√n / log n, EnumPlus runs in expected O(n log n) time. The prior best result required d ≥ c·n / (log n)². In addition, the authors prove that the average‑case time and space complexities of EnumPlus are O(n⁵ log n) and O(n⁵), respectively, and that its worst‑case running time improves slightly over the best known exact algorithms, achieving O((n – 6)·2^{n/2} + n) instead of O(n·2^{n/2}).

Key ideas

1. Statistical observation – For random SSP instances, the distribution of all possible subset sums approximates a normal distribution (by the Central Limit Theorem). Consequently, the number of elements needed to get close to the target E is typically logarithmic in n.
2. Structural observation – When subset sums are organized as a binary tree (each level corresponds to deciding whether to include a particular element), the probability of collisions (different subsets yielding the same sum) drops dramatically after a few levels. This means that the search space shrinks faster than the naïve 2^{n} enumeration.

Leveraging these facts, EnumPlus proceeds in three phases:

Preprocessing / Low‑density conversion: The input set is sorted, and a small “balancing” transformation is applied that adjusts the magnitudes of the numbers so that the instance behaves as if it had low density. This step does not change the solution set but makes the subsequent enumeration more efficient.

Dynamic interval pruning: While recursively building a subset, the algorithm maintains the current partial sum S and the total remaining sum R. If E – S > R, the current branch cannot possibly reach the target and is discarded. This simple bound eliminates a huge fraction of the search tree, especially in the low‑density regime.

Enumerative search with memoisation: The remaining elements are explored depth‑first. At each depth the algorithm records all achievable partial sums in a hash‑based table. When the same sum appears via different paths, the table prevents redundant work. Because of the collision‑reduction property, the size of these tables stays polynomial for the density range of interest.

Complexity analysis

• Average case: Assuming uniformly random inputs, the normal‑like distribution of subset sums ensures that the depth needed to get within a constant factor of E is O(log n). At each depth the number of distinct partial sums is bounded by a polynomial in n, leading to an overall expected running time of O(n log n). The space required to store the hash tables across all levels is O(n⁵).
• Density extension: The authors prove that when d ≥ c·√n / log n, the probability that the preprocessing step yields a “good” low‑density instance approaches 1, thus guaranteeing the expected O(n log n) time bound. This dramatically widens the density regime compared with the earlier bound d ≥ c·n / (log n)².
• Worst case: If the pruning never fires, the algorithm degenerates to a full enumeration of a meet‑in‑the‑middle style search, but the extra factor (n – 6) instead of n in the exponent yields a slight improvement over the classic Horowitz‑Sahni bound.

Experimental validation
The authors benchmark EnumPlus against standard exact SSP solvers (dynamic programming, Horowitz‑Sahni, Schroeppel‑Shamir) on random instances and on cryptographically motivated instances. In the newly covered density range, EnumPlus achieves 2–3× speed‑ups on average while using only modestly more memory (≈1.5× the memory of Horowitz‑Sahni). In worst‑case tests the observed running time matches the theoretical O((n – 6)·2^{n/2}) bound and is about 10 % faster than the previous best exact algorithm.

Significance and future work
EnumPlus demonstrates that a single algorithm can uniformly handle SSP instances across all densities, breaking the long‑standing barrier that required different techniques for low‑, medium‑, and high‑density regimes. The combination of statistical insight (normal‑like sum distribution) and structural pruning (interval bounds and memoisation) may be transferable to other NP‑complete problems such as the knapsack problem or 0‑1 integer programming. Future research directions suggested include (i) memory‑efficient compression of the hash tables, (ii) parallel or distributed implementations to exploit modern multicore architectures, and (iii) a deeper cryptographic analysis of how the widened density threshold impacts the security parameters of subset‑sum‑based cryptosystems.

Conclusion
EnumPlus is a novel exact algorithm for the Subset Sum Problem that, by converting any instance to an effectively low‑density form and then enumerating with aggressive pruning and memoisation, solves SSP in expected O(n log n) time for densities as low as c·√n / log n. Its average‑case space usage is O(n⁵), and its worst‑case time improves slightly over the best known exact methods. Theoretical analysis and empirical results both support the claim that EnumPlus offers a practical, uniform solution for SSP across a much broader spectrum of instance densities than previously achievable.