Towards Physically-Based Sky-Modeling For Image Based Lighting

📝 Abstract

Accurate environment maps are a key component for rendering photorealistic outdoor scenes with coherent illumination. They enable captivating visual arts, immersive virtual reality, and a wide range of engineering and scientific applications. Recent works have extended sky-models to be more comprehensive and inclusive of cloud formations but, as we demonstrate, existing methods fall short in faithfully recreating natural skies. Though in recent years the visual quality of DNN-generated High Dynamic Range Imagery (HDRI) has greatly improved, the environment maps generated by DNN sky-models do not re-light scenes with the same tones, shadows, and illumination as physically captured HDR imagery. In this work, we demonstrate progress in HDR literature to be tangential to sky-modelling as current works cannot support both photorealism and the 22 fstops required for the Full Dynamic Range (FDR) of outdoor illumination. We achieve this by proposing AllSky, a flexible all-weather sky-model learned directly from physically captured HDRI which we leverage to study the input modalities, tonemapping, conditioning, and evaluation of sky-models. Per user-controlled positioning of the sun and cloud formations, AllSky expands on current functionality by allowing for intuitive user control over environment maps and achieves state-of-the-art sky-model performance. Through our proposed evaluation, we demonstrate existing DNN sky-models are not interchangeable with physically captured HDRI or parametric sky-models, with current limitations being prohibitive of scalability and accurate illumination in downstream applications.

💡 Analysis

Accurate environment maps are a key component for rendering photorealistic outdoor scenes with coherent illumination. They enable captivating visual arts, immersive virtual reality, and a wide range of engineering and scientific applications. Recent works have extended sky-models to be more comprehensive and inclusive of cloud formations but, as we demonstrate, existing methods fall short in faithfully recreating natural skies. Though in recent years the visual quality of DNN-generated High Dynamic Range Imagery (HDRI) has greatly improved, the environment maps generated by DNN sky-models do not re-light scenes with the same tones, shadows, and illumination as physically captured HDR imagery. In this work, we demonstrate progress in HDR literature to be tangential to sky-modelling as current works cannot support both photorealism and the 22 fstops required for the Full Dynamic Range (FDR) of outdoor illumination. We achieve this by proposing AllSky, a flexible all-weather sky-model learned directly from physically captured HDRI which we leverage to study the input modalities, tonemapping, conditioning, and evaluation of sky-models. Per user-controlled positioning of the sun and cloud formations, AllSky expands on current functionality by allowing for intuitive user control over environment maps and achieves state-of-the-art sky-model performance. Through our proposed evaluation, we demonstrate existing DNN sky-models are not interchangeable with physically captured HDRI or parametric sky-models, with current limitations being prohibitive of scalability and accurate illumination in downstream applications.

📄 Content

Towards Physically-Based Sky-Modeling For Image Based Lighting Ian J. Maquignaz Universit´e Laval Qu´ebec, Qu´ebec Canada ian.maquignaz.1@ulaval.ca Figure 1. IBL renders of AllSky environment maps and AllSky environment maps generated from user-drawn labels. Abstract Accurate environment maps are a key component for ren- dering photorealistic outdoor scenes with coherent illumi- nation. They enable captivating visual arts, immersive vir- tual reality, and a wide range of engineering and scientific applications. Recent works have extended sky-models to be more comprehensive and inclusive of cloud formations but, as we demonstrate, existing methods fall short in faithfully recreating natural skies. Though in recent years the visual quality of DNN-generated High Dynamic Range Imagery (HDRI) has greatly improved, the environment maps gen- erated by DNN sky-models do not re-light scenes with the same tones, shadows, and illumination as physically cap- tured HDR imagery. In this work, we demonstrate progress in HDR literature to be tangential to sky-modelling as cur- rent works cannot support both photorealism and the 22 f- stops required for the Full Dynamic Range (FDR) of out- door illumination. We achieve this by proposing AllSky, a flexible all-weather sky-model learned directly from phys- ically captured HDRI which we leverage to study the in- put modalities, tonemapping, conditioning, and evaluation of sky-models. Per user-controlled positioning of the sun and cloud formations, AllSky expands on current function- ality by allowing for intuitive user control over environment maps and achieves state-of-the-art sky-model performance. Through our proposed evaluation, we demonstrate existing DNN sky-models are not interchangeable with physically captured HDRI or parametric sky-models, with current lim- itations being prohibitive of scalability and accurate illumi- nation in downstream applications.

1. Introduction Illumination is central to human perception of physical spaces and the visual quality of media and film [16, 44, 56]. Early works modelling skydomes combined data from var- ied sources into pre-computed and parametric sky models for engineering and scientific applications, with the first sky models [38, 45] modelling only luminance. With the advent of the digital age, a new paradigm of applications spurred interest in sky models to enable a wide range of emerging digital applications. Nishita et al. [40] proposed the first colour sky model enabling the generation of extraterrestrial views of the Earth for space flight simulators, and Image- Based Lighting (IBL) techniques [11] were proposed to ren- der synthetic objects into real and virtual scenes. Though conventional Low Dynamic Range (LDR) im- agery is suitable for some applications, High Dynamic Range (HDR) [48] imagery is integral to sky-models and the capture of outdoor scenes. HDR images capture a greater range of illumination and, in the particular case of outdoor lighting, can capture the estimated 22 f-stops of ex- posure necessary for the highlights and shadows of an av- erage real-world outdoor scene [29, 48, 55]. To distinguish the proportionality of scene exposure captured by an HDRI, we define the following:
2. Low Dynamic Range (LDR) Imagery: Display- referenced images with compressed dynamic range which can be clipped and displayed in 8-bit colour.
3. High Dynamic Range (HDR) Imagery: Scene- referenced measures of illumination with uncompressed dynamic range and precision greater than LDR 8-bit colour for later display as LDR. This includes imagery captured by conventional cameras in 12-bit RAW.
4. Extended Dynamic Range (EDR) Imagery: HDR im- ages captured using techniques such as LDR-bracketing for greater exposure range than a singular image from a conventional camera.
5. Full Dynamic Range (FDR) Imagery / Physically- Captured Imagery: HDR images that fully-capture the dynamic range of a reference scene without saturation of the exposure range. This distinction between HDRIs is necessary given the 1 arXiv:2512.15632v1 [cs.CV] 15 Dec 2025 challenges in applying HDR literature to sky-modeling [14, 22, 36, 46, 59, 64]. In this work, we measure HDR as the Exposure Value (EV) of HDRI given EV = log2(|I|max − |I|min + 1), where |I| is grayscale intensity. We find that HDR literature generally does not provide sufficient detail to determine supported exposure ranges, focuses on expo- sure ranges ≤6EV when real-world outdoor scenes are ≥13EV , omit daytime outdoor imagery in results and, if included, demonstrates saturated clouds and solar features. Such limitations are problematic, given exposure range is key to the illumination provided by environment-maps and current sky-models struggle to reproduce the photorealism and illumination of real-world skies. As demonstrated by Fig. 2, incrementally clipping the EV of an HDRI while equalizing exposure to the FDR 15EV ground truth results in visually indiscernible alterations to the environme

This content is AI-processed based on ArXiv data.