Image Processing using Smooth Ordering of its Patches

We propose an image processing scheme based on reordering of its patches. For a given corrupted image, we extract all patches with overlaps, refer to these as coordinates in high-dimensional space, and order them such that they are chained in the “shortest possible path”, essentially solving the traveling salesman problem. The obtained ordering applied to the corrupted image, implies a permutation of the image pixels to what should be a regular signal. This enables us to obtain good recovery of the clean image by applying relatively simple 1D smoothing operations (such as filtering or interpolation) to the reordered set of pixels. We explore the use of the proposed approach to image denoising and inpainting, and show promising results in both cases.

💡 Research Summary

The paper introduces a novel, conceptually simple framework for image denoising and inpainting that relies on reordering image patches rather than on sophisticated high‑dimensional transforms or sparse coding. The method proceeds as follows: (1) From the corrupted image Z, extract all overlapping √n × √n patches (typically n=64 for an 8×8 patch). Each patch is vectorized and regarded as a point x_i in ℝⁿ. (2) Define a distance measure w(x_i, x_j) (e.g., squared Euclidean distance) and seek a permutation P that orders the patches so that the total distance along the sequence is minimized. This is equivalent to solving a Traveling Salesman Problem (TSP) on the set of points. Because exact TSP is infeasible for the thousands of patches in a typical image, the authors adopt a greedy stochastic heuristic: start from a random patch, then move to its nearest or second‑nearest unvisited neighbor with probabilities proportional to exp(−h·w). The search is limited to a B × B spatial window to keep complexity low. (3) The permutation matrix P rearranges the image pixels according to the order of the patch centers, producing a 1‑D signal y_p = P y. The key assumption is that if the original image is smooth, then the reordered signal will also be smooth; conversely, even when the image is corrupted, the ordering derived from the noisy patches remains sufficiently robust to bring the corrupted pixels into a near‑smooth arrangement. (4) Apply a simple 1‑D smoothing operator H (e.g., low‑pass FIR filter, linear interpolation) to the permuted noisy signal z_p = P z, then invert the permutation: ŷ = P⁻¹ H(P z). (5) To reduce the sensitivity to a single permutation, the authors employ a cycle‑spinning‑like strategy: generate K independent random permutations P_k, process each as above, and average the resulting reconstructions. (6) An additional refinement called “sub‑image averaging” exploits the fact that each patch contains n different pixel positions. By extracting all n sub‑images (each consisting of the same relative pixel within every patch), applying the same permutation P to each sub‑image, smoothing, and finally re‑injecting the results into the image canvas, the method obtains n candidate reconstructions per permutation. Averaging over all n × K candidates dramatically improves PSNR. The authors also discuss the relationship to BM3D: both stack patches, apply a 1‑D operation, and aggregate results, but BM3D groups patches locally while the proposed method creates a single global chain, and BM3D uses a fixed 2‑D transform before the 1‑D step, whereas the present framework leaves the choice of H open. Experimental results: – Denoising: with M = I, the method achieves PSNR comparable to K‑SVD‑based sparse coding, sometimes surpassing it on textured images. – Inpainting: with a binary mask M, the algorithm restores missing pixels more naturally than simple interpolation or a DCT‑based sparse method. The visual quality is smoother and edge preservation is better. The paper concludes that ordering patches into a near‑optimal TSP path provides a powerful way to turn a 2‑D image restoration problem into a 1‑D smoothing problem, and that the combination of multiple random orderings and sub‑image averaging yields state‑of‑the‑art performance while keeping the processing pipeline extremely simple. Future work suggested includes better distance metrics, more advanced TSP approximations, and integration of learned non‑linear 1‑D filters.