Cascades of Regression Tree Fields for Image Restoration

Conditional random fields (CRFs) are popular discriminative models for computer vision and have been successfully applied in the domain of image restoration, especially to image denoising. For image deblurring, however, discriminative approaches have been mostly lacking. We posit two reasons for this: First, the blur kernel is often only known at test time, requiring any discriminative approach to cope with considerable variability. Second, given this variability it is quite difficult to construct suitable features for discriminative prediction. To address these challenges we first show a connection between common half-quadratic inference for generative image priors and Gaussian CRFs. Based on this analysis, we then propose a cascade model for image restoration that consists of a Gaussian CRF at each stage. Each stage of our cascade is semi-parametric, i.e. it depends on the instance-specific parameters of the restoration problem, such as the blur kernel. We train our model by loss minimization with synthetically generated training data. Our experiments show that when applied to non-blind image deblurring, the proposed approach is efficient and yields state-of-the-art restoration quality on images corrupted with synthetic and real blur. Moreover, we demonstrate its suitability for image denoising, where we achieve competitive results for grayscale and color images.

💡 Research Summary

This paper investigates the relationship between half‑quadratic regularization—a popular technique for handling sparse image priors—and Gaussian Conditional Random Fields (CRFs). The authors show that, after the auxiliary variables in a half‑quadratic scheme have converged, the posterior over the latent clean image is a Gaussian random field whose precision matrix and linear term depend on the observed image, the blur kernel, and the noise level. Instead of iteratively updating the auxiliary variables, they propose to predict these Gaussian parameters directly from the input using Regression Tree Fields (RTFs).

An RTF is a semi‑parametric model: a set of regression trees maps image‑derived features (filter responses, intensities, and the current estimate) together with instance‑specific parameters (e.g., the known blur kernel) to the entries of the precision matrix Θ and the vector θ. The predicted Gaussian CRF then yields the restored image in a single linear solve.

To handle the strong variability introduced by different blur kernels, the authors embed the RTFs into a cascade of stages. The first stage produces a coarse deblurred image that removes the dominant blur; this intermediate result is fed back as a feature for subsequent stages, allowing progressively finer parameter estimation. Each stage is trained discriminatively by minimizing a task‑specific loss (PSNR) on synthetically generated training pairs. Synthetic blur kernels are sampled from a simple motion model, and a wide range of noise levels is used to ensure robustness.

Experiments on non‑blind deblurring (known kernel) demonstrate state‑of‑the‑art PSNR on both synthetic benchmark datasets and real camera‑shake images, while maintaining a computational cost far lower than traditional iterative half‑quadratic or sampling‑based methods. The same cascade architecture is applied to grayscale and color denoising, achieving competitive results and showing that jointly modeling color channels outperforms independent per‑channel processing.

Key contributions include: (1) a theoretical bridge between half‑quadratic inference and discriminative Gaussian CRFs, (2) a regression‑tree‑based cascade that can handle arbitrary instance‑specific parameters, (3) a loss‑driven training regime that directly optimizes perceptual quality, (4) evidence that models trained on synthetic blur kernels generalize to real-world blur, and (5) a unified framework that attains top performance on both deblurring and denoising tasks with efficient inference. Limitations involve the dependence on tree depth and cascade depth for speed‑accuracy trade‑offs, and potential challenges when extending to highly non‑stationary blur or non‑Gaussian noise models.