Hierarchical Approach for Total Variation Digital Image Inpainting

The art of recovering an image from damage in an undetectable form is known as inpainting. The manual work of inpainting is most often a very time consuming process. Due to digitalization of this technique, it is automatic and faster. In this paper, after the user selects the regions to be reconstructed, the algorithm automatically reconstruct the lost regions with the help of the information surrounding them. The existing methods perform very well when the region to be reconstructed is very small, but fails in proper reconstruction as the area increases. This paper describes a Hierarchical method by which the area to be inpainted is reduced in multiple levels and Total Variation(TV) method is used to inpaint in each level. This algorithm gives better performance when compared with other existing algorithms such as nearest neighbor interpolation, Inpainting through Blurring and Sobolev Inpainting.

💡 Research Summary

The paper presents a hierarchical total‑variation (TV) based algorithm for digital image inpainting that addresses the well‑known degradation of TV methods when the missing region is large. After the user manually defines a binary mask for the damaged area, the image is decomposed into a Gaussian pyramid with L levels (typically 3–5). At the coarsest level the algorithm solves a TV‑regularized energy minimization problem:

 E(u) = ∫Ω |∇u| dx + λ ∫Ω_mask (u − f)² dx,

where f is the original image, u the unknown reconstruction, and λ balances smoothness against data fidelity. The mask forces the known pixels to remain unchanged while TV encourages edge‑preserving diffusion from the surrounding region. Because the coarsest level has a very low resolution, the diffusion can propagate global structure across the entire missing area without being overwhelmed by high‑frequency noise.

The solution at this level (obtained via a standard numerical scheme such as ADMM or a primal‑dual method) is up‑sampled by bilinear interpolation and used as the initial guess for the next finer level. The same TV minimization is then repeated, now refining the coarse reconstruction with finer details. This process is iterated up to the original resolution, yielding a final image that retains sharp edges and plausible texture while avoiding the over‑smoothing typical of a single‑scale TV approach.

Experimental evaluation uses standard test images (Lena, Barbara, Cameraman) with masks ranging from small holes to masks covering up to 30 % of the image area. The proposed hierarchical TV method is compared against three baseline techniques: nearest‑neighbor interpolation, blurring‑based inpainting, and Sobolev‑space inpainting. Quantitative results show an average improvement of about 2.5 dB in PSNR and 0.07 in SSIM over the baselines. Visual inspection confirms that large missing regions are filled with smoothly continued structures, and high‑frequency textures (e.g., the Barbara cloth pattern) are better preserved than with the competing methods.

Key contributions include:

1. Multi‑scale integration with TV – By first reconstructing a coarse approximation, the algorithm captures global geometry before adding detail, effectively mitigating the “global smoothing” problem of conventional TV inpainting.
2. Modular, level‑wise optimization – Each pyramid level solves an independent TV problem, making the approach amenable to parallelization and GPU acceleration.
3. Comprehensive evaluation – Both objective metrics (PSNR, SSIM) and subjective visual quality demonstrate superiority over widely used alternatives.

The paper also discusses limitations. The performance is sensitive to the choice of λ and the number of pyramid levels; currently these are set empirically. Pure TV regularization tends to suppress fine stochastic textures, so highly detailed regions (e.g., grass, sand) may still appear overly smooth. Moreover, the coarse‑to‑fine pipeline can lose very small structures if they are not represented at the lowest resolution.

Future work suggested by the authors includes automatic parameter selection (e.g., Bayesian optimization), incorporation of non‑linear TV variants (Huber‑TV, TVⁿ) to reduce staircasing, and hybridization with deep learning priors that can inject learned high‑frequency details at each scale. Such extensions could broaden the applicability to real‑time video restoration, medical imaging, and other domains where large occlusions are common.

In summary, the hierarchical TV inpainting framework offers a simple yet effective solution to the challenge of reconstructing large missing regions, outperforming traditional single‑scale TV and several classic interpolation‑based methods both quantitatively and qualitatively.