Dynamic Updating for L1 Minimization

📝 Abstract

The theory of compressive sensing (CS) suggests that under certain conditions, a sparse signal can be recovered from a small number of linear incoherent measurements. An effective class of reconstruction algorithms involve solving a convex optimization program that balances the L1 norm of the solution against a data fidelity term. Tremendous progress has been made in recent years on algorithms for solving these L1 minimization programs. These algorithms, however, are for the most part static: they focus on finding the solution for a fixed set of measurements. In this paper, we will discuss “dynamic algorithms” for solving L1 minimization programs for streaming sets of measurements. We consider cases where the underlying signal changes slightly between measurements, and where new measurements of a fixed signal are sequentially added to the system. We develop algorithms to quickly update the solution of several different types of L1 optimization problems whenever these changes occur, thus avoiding having to solve a new optimization problem from scratch. Our proposed schemes are based on homotopy continuation, which breaks down the solution update in a systematic and efficient way into a small number of linear steps. Each step consists of a low-rank update and a small number of matrix-vector multiplications – very much like recursive least squares. Our investigation also includes dynamic updating schemes for L1 decoding problems, where an arbitrary signal is to be recovered from redundant coded measurements which have been corrupted by sparse errors.

💡 Analysis

The theory of compressive sensing (CS) suggests that under certain conditions, a sparse signal can be recovered from a small number of linear incoherent measurements. An effective class of reconstruction algorithms involve solving a convex optimization program that balances the L1 norm of the solution against a data fidelity term. Tremendous progress has been made in recent years on algorithms for solving these L1 minimization programs. These algorithms, however, are for the most part static: they focus on finding the solution for a fixed set of measurements. In this paper, we will discuss “dynamic algorithms” for solving L1 minimization programs for streaming sets of measurements. We consider cases where the underlying signal changes slightly between measurements, and where new measurements of a fixed signal are sequentially added to the system. We develop algorithms to quickly update the solution of several different types of L1 optimization problems whenever these changes occur, thus avoiding having to solve a new optimization problem from scratch. Our proposed schemes are based on homotopy continuation, which breaks down the solution update in a systematic and efficient way into a small number of linear steps. Each step consists of a low-rank update and a small number of matrix-vector multiplications – very much like recursive least squares. Our investigation also includes dynamic updating schemes for L1 decoding problems, where an arbitrary signal is to be recovered from redundant coded measurements which have been corrupted by sparse errors.

📄 Content

1 minimization programs. These algorithms, however, are for the most part static: they focus on finding the solution for a fixed set of measurements. In this paper, we will present a suite of dynamic algorithms for solving 1 minimization programs for streaming sets of measurements. We consider cases where the underlying signal changes slightly between measurements, and where new measurements of a fixed signal are sequentially added to the system. We develop algorithms to quickly update the solution of several different types of 1 optimization problems whenever these changes occur, thus avoiding having to solve a new optimization problem from scratch. Our proposed schemes are based on homotopy continuation, which breaks down the solution update in a systematic and efficient way into a small number of linear steps. Each step consists of a low-rank update and a small number of matrix-vector multiplicationsvery much like recursive least squares. Our investigation also includes dynamic updating schemes for

Recovering a signal from a set of linear measurements is a fundamental problem in signal processing.

We are given measurements y ∈ R m of the form

where A is an m × n matrix and e is a noise vector. From these, we wish to reconstruct the unknown signal x ∈ R n . The classical solution to this problem is to estimate x from y using least-squares. Given

or when A is ill-conditioned

where τ > 0 is a regularization parameter. Each of these minimizers can be found by solving a system of linear equations. We can interpret the solution to (3) as the estimate which, depending on the value of τ , strikes a balance between the data fidelity (we want the energy in the mismatch between the simulated measurements Ax of our estimate and the true measurements y to be small) and the complexity of the estimate (among all estimates with the same measurements, we want the one with minimal energy).

Recent developments in the theory of compressive sensing (CS) have shown us that under certain conditions, dramatic gains can be had by promoting sparsity instead of minimizing energy. There are two classes of problems:

CS: In this case, the matrix A is underdetermined, and the signal x is sparse. To promote sparsity in the solution, we penalize the 1 norm of the estimate, solving

For certain types of measurement matrices (namely, matrices that obey a type of uncertainty principle [1]), (4) comes with a number of performance guarantees [2]- [7]. In particular, if x is sparse enough and there is no noise, (4) will recover x exactly as τ → 0 even though A is underdetermined; the recovery can also be made stable when the measurements are made in the presence of noise with an appropriate choice of τ . There are also several variations on (4) which use slightly different penalties for the measurement error. We will also be interested in one of these variations, the Dantzig Selector [8] given in (10) below.

Decoding: In this case, the matrix A is overdetermined, and the error e is sparse. To account for this, we solve minimize Ax -y 1 (5) in place of (2). There are again a number of performance guarantees for (5) that relate the number of errors we can correct (number of non-zero entries in e) to the number of measurements we have collected (rows in A) [9], [10]. If the matrix consists of independent Gaussian random variables, then the number of errors we can correct (and hence recover x exactly) scales with the amount of oversampling m -n.

These 1 minimization programs are tractable, but solving them is more involved than least-squares.

In this paper, we will be interested in how solutions to these problems change as 1) the signal we are measuring changes by a small amount, and 2) new measurements of the signal are added. We will present a suite of algorithms that avoid solving these programs from scratch each time we are given a new set of measurements, and instead quickly update the solution. We will constrain our discussion to small and medium scale problems, where the matrices are stored explicitly and linear systems of equations are solved exactly (within machine precision) using direct methods. We begin with a brief review of how updating works in the least-squares scenario.

When the m × n matrix A has full column rank (is overdetermined), the least squares problem (2) has a unique solution x0 found by solving a system of linear equations:

There is a variety of ways to compute x0 , including iterative methods that have the potential to return an approximate solution at relatively low cost, but in general the computational cost involved for an exact solution is O(mn 2 ). Typical direct methods for solving (2) involve Cholesky or QR decompositions [11], [12]. If we have already computed the QR factorization for A (or Cholesky factorization for A T A), then there is not much marginal cost in recovering additional signals measured with the same matrix A. We can simply use the already computed factorization for the new set of m

This content is AI-processed based on ArXiv data.