A Conditional Random Field Model for Context Aware Cloud Detection in Sky Images

A conditional random field (CRF) model for cloud detection in ground based sky images is presented. We show that very high cloud detection accuracy can be achieved by combining a discriminative classifier and a higher order clique potential in a CRF framework. The image is first divided into homogeneous regions using a mean shift clustering algorithm and then a CRF model is defined over these regions. The various parameters involved are estimated using training data and the inference is performed using Iterated Conditional Modes (ICM) algorithm. We demonstrate how taking spatial context into account can boost the accuracy. We present qualitative and quantitative results to prove the superior performance of this framework in comparison with other state of the art methods applied for cloud detection.

💡 Research Summary

The paper presents a novel cloud‑detection framework for ground‑based whole‑sky images that leverages spatial context through a Conditional Random Field (CRF) model. Traditional pixel‑wise thresholding techniques—such as fixed red‑to‑blue (RB) ratios, saturation‑value (SV) ratios, or normalized blue‑red (NBR) ratios—struggle with overlapping feature distributions between sky and cloud pixels, leading to sub‑optimal accuracy, especially under varying illumination and aerosol conditions. To overcome these limitations, the authors first segment each image into homogeneous regions using the mean‑shift clustering algorithm. This region‑based representation reduces computational load, mitigates noise, and provides a natural graph structure for the CRF.

In the CRF formulation, each region (site) is assigned a binary label (0 = sky, 1 = cloud). Two complementary potentials are defined: (1) an association potential ψ that models the probability of a region being cloud based solely on its NBR value. This is implemented as a logistic regression ψ(xᵢ)=exp(α₀+α₁Kᵢ)/(1+exp(α₀+α₁Kᵢ)), where Kᵢ is the NBR ratio for region i and (α₀,α₁) are learned from training data; (2) an interaction potential φ that captures spatial consistency using the NSV (normalized saturation‑value) ratio. For each region i, the average NSV of neighboring sky regions (V_S) and cloud regions (V_C) are computed within a 200‑pixel radius. The potential φ(x, yᵢ, yⱼ) = (V_S−Vᵢ) when yᵢ=0 and (Vᵢ−V_C) when yᵢ=1 penalizes configurations where a region’s NSV deviates from the typical values of its surrounding class. A scalar β controls the strength of this contextual term.

Parameter estimation is performed via piecewise training rather than full maximum‑likelihood, which would be intractable due to the high‑order cliques (up to ~80 regions). The authors split an eight‑image manually labeled dataset: four images train the logistic regression (α parameters) using R’s glm function, and the remaining four images tune β by minimizing pixel‑wise classification error. Inference seeks the most probable labeling y* = argmax_y P(y|x;θ). Because exact inference is NP‑hard, the authors adopt Iterated Conditional Modes (ICM), a greedy local‑search algorithm. ICM iteratively updates each region’s label to maximize its conditional probability given current neighbor labels, starting from the logistic‑regression output. When a region’s neighborhood contains only one class, global average NSV values replace local statistics to avoid degenerate updates.

The experimental evaluation uses two test sets: Set C (22 images) compares the CRF method against Li et al.’s hybrid fixed/adaptive thresholding, while Set D (26 images) compares against a pure fixed‑threshold approach. Results show that the CRF model consistently outperforms the baselines. With adaptive thresholding (ATS) as a reference, the CRF achieves accuracy 0.9346 ± 0.0269, precision 0.9561 ± 0.0560, and recall 0.9022 ± 0.0471. Against fixed thresholding, the CRF reaches accuracy 0.9436 ± 0.0181, precision 0.9597 ± 0.0341, and recall 0.9095 ± 0.0309. The gains are most pronounced in the ambiguous NBR interval