Tensor Oriented No-Reference Light Field Image Quality Assessment

Light field image (LFI) quality assessment is becoming more and more important, which helps to better guide the acquisition, processing and application of immersive media. However, due to the inherent high dimensional characteristics of LFI, the LFI quality assessment turns into a multi-dimensional problem that requires consideration of the quality degradation in both spatial and angular dimensions. Therefore, we propose a novel Tensor oriented No-reference Light Field image Quality evaluator (Tensor-NLFQ) based on tensor theory. Specifically, since the LFI is regarded as a low-rank 4D tensor, the principal components of four oriented sub-aperture view stacks are obtained via Tucker decomposition. Then, the Principal Component Spatial Characteristic (PCSC) is designed to measure the spatial-dimensional quality of LFI considering its global naturalness and local frequency properties. Finally, the Tensor Angular Variation Index (TAVI) is proposed to measure angular consistency quality by analyzing the structural similarity distribution between the first principal component and each view in the view stack. Extensive experimental results on four publicly available LFI quality databases demonstrate that the proposed Tensor-NLFQ model outperforms state-of-the-art 2D, 3D, multi-view, and LFI quality assessment algorithms.

💡 Research Summary

Light‑field images (LFIs) capture both spatial (x, y) and angular (s, t) information, forming a 4‑dimensional data structure that poses unique challenges for quality assessment. Traditional 2‑D or 3‑D image quality metrics ignore the angular dimension and therefore cannot adequately evaluate LFIs. In this paper the authors propose Tensor‑NLFQ, a no‑reference (NR) quality evaluator that explicitly exploits the tensor nature of LFIs.

The method begins by converting each sub‑aperture image (SAI) from RGB to the perceptually uniform CIELAB color space, thereby separating luminance (L*) from two chrominance channels (a*, b*). This step allows the metric to capture both brightness‑related distortions and color‑related artifacts, which are known to affect angular consistency. For each color channel, four view stacks are constructed corresponding to the four principal angular orientations: horizontal (0°), vertical (90°), left‑diagonal (45°) and right‑diagonal (135°). Each stack is a 3‑D tensor (two spatial dimensions plus one angular dimension).

To reduce redundancy along the angular mode, Tucker decomposition is applied to every view stack. Tucker decomposition, a higher‑order generalization of PCA/SVD, yields a core tensor and factor matrices for each mode. By retaining only the first factor along the angular mode, the authors obtain the first principal component (PC) of each stack – a compact representation that preserves the most energetic spatial‑angular structure while discarding less informative angular variations.

Two complementary quality descriptors are then extracted from the PCs.

1. Principal Component Spatial Characteristic (PCSC) – PCSC evaluates spatial quality through (i) a global naturalness measure derived from Mean Subtracted Contrast Normalized (MSCN) coefficients for each of the three CIELAB channels, and (ii) a local frequency analysis (e.g., Gabor or DCT) that captures high‑frequency loss, blur, or ringing. By jointly considering luminance and chrominance statistics, PCSC overcomes the limitation of many NR IQA methods that rely solely on luminance.
2. Tensor Angular Variation Index (TAVI) – TAVI quantifies angular consistency. For each view in a stack, the structural similarity index (SSIM) between the view and the corresponding PC is computed, producing a distribution of similarity scores. The mean and variance of this distribution serve as angular consistency statistics. This procedure is repeated for all four orientations, and the resulting statistics are fused (e.g., weighted averaging) to obtain a single TAVI value. Because SSIM is sensitive to color mismatches as well as structural differences, TAVI naturally incorporates chromatic angular inconsistencies.

The final quality prediction combines PCSC and TAVI using a regression model (the authors employ Support Vector Regression, but any suitable non‑linear regressor could be used). The model is trained on subjective mean opinion scores (MOS) from publicly available LFI databases.

Extensive experiments were conducted on four benchmark LFI quality datasets (Win5‑LID, EPFL‑Lytro, HCI‑LF, and an additional set). The authors compared Tensor‑NLFQ against a wide range of state‑of‑the‑art metrics: full‑reference 2‑D methods (SSIM, MS‑SSIM, VIF, etc.), full‑reference 3‑D methods (3DSwIM, MP‑PSNR), multi‑view FR metrics (MW‑PSNR, MP‑PSNR), reduced‑reference LFI metric (LF‑IQM), and the only existing NR LFI metric (BELIF). Performance was measured using Pearson correlation coefficient (PCC), Spearman rank‑order correlation coefficient (SRCC), and root‑mean‑square error (RMSE). Tensor‑NLFQ consistently achieved the highest PCC and SRCC and the lowest RMSE across all databases. Notably, its advantage was most pronounced for distortions that involve color shifts, combined compression‑reconstruction artifacts, and cases where angular consistency varies across directions.

Key contributions of the paper are:

• Modeling LFIs as low‑rank 4‑D tensors and applying Tucker decomposition to obtain direction‑specific principal components, preserving essential spatial‑angular information while reducing dimensionality.
• Introducing PCSC, which jointly leverages global naturalness statistics and local frequency features across luminance and chrominance channels.
• Proposing TAVI, a novel angular‑consistency index that evaluates similarity distributions for each angular orientation, thereby capturing directional inconsistencies missed by prior work.
• Demonstrating, through comprehensive cross‑database experiments, that the proposed NR metric outperforms existing 2‑D, 3‑D, multi‑view, and LFI‑specific quality models.

The authors suggest future work on real‑time implementation (e.g., lightweight tensor approximations), integration of depth maps for a hybrid spatial‑depth quality model, and application of Tensor‑NLFQ in adaptive streaming pipelines for VR/AR where rapid quality feedback is essential.