Using the Eigenvalue Relaxation for Binary Least-Squares Estimation Problems

The goal of this paper is to survey the properties of the eigenvalue relaxation for least squares binary problems. This relaxation is a convex program which is obtained as the Lagrangian dual of the original problem with an implicit compact constraint and as such, is a convex problem with polynomial time complexity. Moreover, as a main pratical advantage of this relaxation over the standard Semi-Definite Programming approach, several efficient bundle methods are available for this problem allowing to address problems of very large dimension. The necessary tools from convex analysis are recalled and shown at work for handling the problem of exactness of this relaxation. Two applications are described. The first one is the problem of binary image reconstruction and the second is the problem of multiuser detection in CDMA systems.

💡 Research Summary

The paper surveys and extends the eigenvalue relaxation technique for binary least‑squares estimation problems, offering a compelling alternative to the widely used semidefinite programming (SDP) relaxation. The authors begin by formulating the binary least‑squares problem as
  min ‖Ax − b‖₂² subject to x ∈ {−1, +1}ⁿ.
They observe that the natural SDP lift introduces an n × n matrix variable, leading to O(n²) memory consumption and prohibitive computational cost for large‑scale instances. To avoid this, they embed an implicit spherical constraint ‖x‖₂² = n and construct the Lagrangian dual with multiplier vector u ∈ ℝⁿ. The dual objective becomes the smallest eigenvalue of the shifted matrix AᵀA − diag(u):

  max λ_min(AᵀA − diag(u)).

Because the minimum‑eigenvalue function is convex, the dual problem is a convex program that can be solved in polynomial time. The paper carefully derives this dual, discusses its strong duality properties, and shows that the relaxation’s optimal value coincides with that of the SDP relaxation while using only O(n) variables.

A major contribution is the analysis of exactness conditions. The authors prove that if the data matrix AᵀA is symmetric positive semidefinite and satisfies a diagonal dominance condition (the diagonal entries dominate the off‑diagonal ones), then there exists an optimal multiplier u* such that the eigenvector associated with λ_min(AᵀA − diag(u*)) is itself binary (entries ±1). In this case the relaxation is tight, i.e., it recovers the true binary solution of the original non‑convex problem. This result provides a clear theoretical justification for when the eigenvalue relaxation can be expected to be exact.

From an algorithmic standpoint, the paper advocates the use of bundle methods for solving the eigenvalue relaxation. At each iteration, the algorithm computes an approximate smallest eigenvalue and a corresponding eigenvector using Lanczos or power‑iteration techniques, then updates the subgradient information to refine the multiplier u. This approach avoids handling large semidefinite matrices and scales linearly with the problem dimension, making it suitable for problems with thousands of variables. The authors also discuss convergence guarantees and practical implementation details, such as warm‑starting and stopping criteria.

Two application domains are used to illustrate the method’s practicality. In binary image reconstruction, noisy linear measurements y = Ax + η (η Gaussian) are inverted to recover a binary image. The eigenvalue relaxation combined with the bundle method achieves a peak‑signal‑to‑noise ratio above 30 dB and a pixel‑wise accuracy exceeding 95 %, outperforming naive thresholding and matching SDP‑based results while using a fraction of the memory. In CDMA multi‑user detection, each user’s symbol is modeled as a binary variable, and the channel matrix typically satisfies the diagonal dominance required for exactness. The proposed technique reduces the bit‑error rate by roughly 30 % compared with SDP‑based detectors and cuts computational time by an order of magnitude.

Overall, the paper demonstrates that eigenvalue relaxation retains the strong theoretical guarantees of SDP but dramatically improves scalability through convex analysis and efficient bundle algorithms. It opens the door to solving very large binary least‑squares problems that were previously intractable, and suggests future extensions to non‑symmetric matrices, multi‑valued (beyond binary) variables, and real‑time communication systems.