Spectrum from Defocus: Fast Spectral Imaging with Chromatic Focal Stack

Hyperspectral cameras face harsh trade-offs between spatial, spectral, and temporal resolution in inherently low-photon conditions. Computational imaging systems break through these trade-offs with compressive sensing, but have required complex optics and/or extensive compute. We present Spectrum from Defocus (SfD), a chromatic focal sweep method that achieves state-of-the-art hyperspectral imaging with only two off-the-shelf lenses, a grayscale sensor, and less than one second of reconstruction time. By capturing a chromatically-aberrated focal stack that preserves nearly all incident light, and reconstructing it with a fast physics-based iterative algorithm, SfD delivers sharp, accurate hyperspectral images. The combination of photon efficiency, optical simplicity, and physical interpretability makes SfD a promising solution for fast, compact, interpretable hyperspectral imaging.

💡 Research Summary

The paper introduces Spectrum from Defocus (SfD), a novel hyperspectral imaging (HSI) approach that achieves state‑of‑the‑art image quality with a remarkably simple optical setup and a fast physics‑based reconstruction algorithm. The hardware consists of only two off‑the‑shelf lenses, a movable second lens, and a grayscale sensor. By translating the second lens to a small number of discrete positions (N = 5 in the experiments), each position brings a different wavelength into sharp focus while all other wavelengths become progressively blurred due to chromatic aberration. The resulting set of five defocused grayscale images forms a “chromatic focal stack” that encodes spectral information without any light‑blocking elements, preserving nearly all incident photons—a crucial advantage for low‑light or high‑speed scenarios.

Mathematically, the forward model is expressed as Y = C H X, where X ∈ ℝ^{H·W·C} is the unknown hyperspectral cube, Y ∈ ℝ^{H·W·N} the captured focal stack, H aggregates wavelength‑dependent point‑spread functions (PSFs) as 2‑D convolution matrices, and C crops the convolution result to match sensor dimensions. Because N ≪ C, the inverse problem is severely under‑determined.

To solve it, the authors adopt a plug‑and‑play ADMM framework with two key augmentations. First, they exploit the low‑rank nature of natural visible spectra: a basis B (v × C) derived from the Harvard spectral dataset is used to project X onto a low‑dimensional subspace (v ≈ 8–10). This reduces the unknowns from C channels to v coefficients, dramatically shrinking computational load. Second, they recognize that sub‑matrices of H possess a Block‑Circulant with Circulant Blocks (BCCB) structure, allowing fast inversion via FFT‑based formulas. The ADMM updates consist of (i) a closed‑form solution for the auxiliary variable v, (ii) a Wiener‑like filter for the coefficient map z that involves the fast inverse of (μ₁ ĤᵀĤ + μ₂ I), and (iii) a denoising step using an off‑the‑shelf deep image denoiser φ_θ. The denoiser operates in the image domain after mapping z back with the basis matrix P = Bᵀ ⊗ I_{HW}, ensuring that the regularizer is applied to realistic hyperspectral images and preventing the hallucination often seen in purely data‑driven reconstructions.

Experimental evaluation on 30 scenes from the Harvard dataset under both bright (5 s total exposure) and low‑light (2.9 s total exposure) conditions shows that SfD outperforms several representative systems, including CASSI‑based methods, MST (a transformer‑based approach), KRISM, 2in1‑Cameras, Spectral Defocus‑Cam, and others. Quantitatively, SfD achieves PSNR = 30.81 dB, SSIM = 0.92, and SAM = 7.35°, the highest among all compared methods. Reconstruction time is only 0.64 s on an NVIDIA RTX A6000, far faster than the >15 min required by some CASSI variants. The optical hardware count is reduced to four components (two lenses, a linear actuator, and the sensor), compared with 10–20 elements in many competing designs.

Key insights include: (1) turning chromatic aberration—a traditionally undesirable lens defect—into a purposeful coding mechanism that preserves photon budget; (2) integrating a low‑dimensional spectral prior with a deep denoiser within a plug‑and‑play ADMM to achieve both speed and spectral fidelity; and (3) leveraging the BCCB structure of the convolutional forward model for sub‑second matrix inversions. By marrying optical simplicity with a lightweight, interpretable algorithm, SfD offers a practical pathway toward compact, fast, and accurate hyperspectral cameras suitable for remote sensing, medical diagnostics, food quality inspection, and industrial monitoring.