GAN-based Projector for Faster Recovery with Convergence Guarantees in Linear Inverse Problems

A Generative Adversarial Network (GAN) with generator $G$ trained to model the prior of images has been shown to perform better than sparsity-based regularizers in ill-posed inverse problems. Here, we propose a new method of deploying a GAN-based prior to solve linear inverse problems using projected gradient descent (PGD). Our method learns a network-based projector for use in the PGD algorithm, eliminating expensive computation of the Jacobian of $G$. Experiments show that our approach provides a speed-up of $60\text{-}80\times$ over earlier GAN-based recovery methods along with better accuracy. Our main theoretical result is that if the measurement matrix is moderately conditioned on the manifold range($G$) and the projector is $\delta$-approximate, then the algorithm is guaranteed to reach $O(\delta)$ reconstruction error in $O(log(1/\delta))$ steps in the low noise regime. Additionally, we propose a fast method to design such measurement matrices for a given $G$. Extensive experiments demonstrate the efficacy of this method by requiring $5\text{-}10\times$ fewer measurements than random Gaussian measurement matrices for comparable recovery performance. Because the learning of the GAN and projector is decoupled from the measurement operator, our GAN-based projector and recovery algorithm are applicable without retraining to all linear inverse problems, as confirmed by experiments on compressed sensing, super-resolution, and inpainting.

💡 Research Summary

The paper introduces a novel framework for solving linear inverse problems (e.g., compressed sensing, super‑resolution, inpainting) by leveraging a pre‑trained Generative Adversarial Network (GAN) as an image prior, but without the heavy computational burden of previous GAN‑based methods. Existing approaches such as CSGM and PGD‑GAN require solving a non‑convex optimization in the latent space at every outer iteration, which involves back‑propagating through the generator to compute its Jacobian or running an inner‑loop optimizer (e.g., Adam) for many steps. This makes them slow (often orders of magnitude slower than classical methods) and prone to getting stuck in local minima.

The authors propose to replace the inner loop with a learned projector. After training a GAN generator (G) on a dataset, they keep (G) fixed and train an inverse network (G^{\dagger}\theta) that maps an image back to its latent code. The projector is defined as (P = G \circ G^{\dagger}\theta). Training uses a multi‑task loss: (i) reconstruction loss (|G(G^{\dagger}\theta(G(z)+\nu)) - G(z)|^2) to force the composition to map noisy generator outputs back onto the range of (G), and (ii) latent consistency loss (|G^{\dagger}\theta(G(z)+\nu) - z|^2) to keep the inverse accurate. Noise (\nu) is added to encourage robustness to points outside the manifold.

With this projector, the recovery algorithm becomes a network‑based projected gradient descent (NPGD):

1. Gradient step: (w_t = x_t - \eta A^\top (A x_t - y)).
2. Projection: (x_{t+1} = P(w_t) = G(G^{\dagger}_\theta(w_t))).

No Jacobian of (G) is required; each iteration only needs a forward pass through (G^{\dagger}_\theta) and (G). The step size is set to (\eta = 1/\beta), where (\beta) is the upper restricted eigenvalue of the measurement matrix on the manifold.

Theoretical analysis introduces a Restricted Eigenvalue Constraint (REC) on the measurement matrix (A): for all (x_1, x_2) in the generator’s range, (\alpha |x_1-x_2|^2 \le |A(x_1-x_2)|^2 \le \beta |x_1-x_2|^2). If (\beta/\alpha < 2) and the projector is (\delta)-approximate (i.e., (|x - P(x)|^2 \le \min_{z}|x - G(z)|^2 + \delta)), Theorem 1 shows that after (n) iterations the loss decays geometrically and the reconstruction error satisfies (|x_n - x^\ast| \le C \delta) after (O(\log(1/\delta))) steps. Thus the algorithm converges linearly to an error proportional to the projector’s approximation quality.

A major practical issue is that random Gaussian matrices often violate the REC condition for moderate numbers of measurements. To address this, the authors design data‑aware measurement matrices. They sample a set of normalized secants (s = (G(z_1)-G(z_2))/|G(z_1)-G(z_2)|) from the generator, forming a matrix (D) whose columns are these secants. The optimization (\min_{A, AA^\top = I_m} \max_{s}|As| / \min_{s}|As|) reduces to maximizing the spectral norm of (AD) under orthonormal rows, whose solution is given by the top‑(m) eigenvectors of (DD^\top). This procedure runs in (O(M n^2 + n^3)) time (with (M) a modest number of secants) and yields matrices that empirically satisfy (\beta/\alpha < 2) even with very few measurements.

Extensive experiments on CelebA, MNIST, and LSUN demonstrate:

• Speed: NPGD is 60–80× faster than CSGM and PGD‑GAN, achieving reconstruction in sub‑second time on a GPU.
• Accuracy: PSNR/SSIM improvements of 2–3 dB over prior GAN‑based methods, and stable recovery at extreme compression ratios (e.g., 61×, i.e., 1.6 % measurements).
• Measurement efficiency: Using the designed matrices, the required number of measurements drops by a factor of 5–10 compared to random Gaussian matrices for comparable performance.
• Generality: The same trained (G) and projector work across different linear operators (random sensing, down‑sampling for super‑resolution, binary masks for inpainting) without any retraining.

In summary, the paper delivers a practically efficient and theoretically grounded approach to incorporate GAN priors into linear inverse problems. By learning a fast projector and designing measurement matrices that respect the manifold geometry, it achieves dramatic speedups, better reconstruction quality, and reduced measurement requirements, while remaining applicable to a broad class of inverse problems without problem‑specific retraining.