FlareX: A Physics-Informed Dataset for Lens Flare Removal via 2D Synthesis and 3D Rendering

Lens flare occurs when shooting towards strong light sources, significantly degrading the visual quality of images. Due to the difficulty in capturing flare-corrupted and flare-free image pairs in the real world, existing datasets are typically synthesized in 2D by overlaying artificial flare templates onto background images. However, the lack of flare diversity in templates and the neglect of physical principles in the synthesis process hinder models trained on these datasets from generalizing well to real-world scenarios. To address these challenges, we propose a new physics-informed method for flare data generation, which consists of three stages: parameterized template creation, the laws of illumination-aware 2D synthesis, and physical engine-based 3D rendering, which finally gives us a mixed flare dataset that incorporates both 2D and 3D perspectives, namely FlareX. This dataset offers 9,500 2D templates derived from 95 flare patterns and 3,000 flare image pairs rendered from 60 3D scenes. Furthermore, we design a masking approach to obtain real-world flare-free images from their corrupted counterparts to measure the performance of the model on real-world images. Extensive experiments demonstrate the effectiveness of our method and dataset.

💡 Research Summary

The paper “FlareX: A Physics-Informed Dataset for Lens Flare Removal via 2D Synthesis and 3D Rendering” addresses a significant challenge in computational photography: creating a high-quality dataset for training robust lens flare removal models. Lens flare, caused by scattering and reflection within the camera lens when shooting towards strong light sources, degrades image quality and hinders downstream vision tasks. The core problem is the extreme difficulty of capturing large-scale paired data (flare-corrupted vs. flare-free) in the real world. Existing datasets rely on simplistic 2D synthesis, where artificial flare templates are randomly overlaid onto background images. This approach suffers from limited flare pattern diversity and, crucially, ignores the physical principles governing flare appearance (e.g., intensity variation with light source distance), leading to poor model generalization to real photographs.

To overcome these limitations, the authors propose a novel, three-stage, physics-informed framework for flare data generation, resulting in the comprehensive FlareX dataset.

Stage 1: Parameterized Flare Template Creation. Using the 3D graphics engine Blender and its plugins, the authors manually define 95 distinct flare types, encompassing basic and complex (“XT”) patterns with components like streaks, glares, and ghosts. Key physical parameters (light intensity, number of reflections, lens smudges) are adjustable. Crucially, mutual constraints between components are enforced, allowing the flare pattern to change realistically as the virtual light source moves. This process yields 9,500 high-quality, diverse flare templates, far surpassing the variety in previous datasets like Flare7K.

Stage 2: Illumination-Aware 2D Synthesis (Flare-2D). This stage improves upon the naive 2D synthesis pipeline. Instead of random brightness adjustment, the authors integrate the laws of illumination. A pre-trained monocular depth estimation model first calculates a depth map for the background image. For each affine-transformed flare template, a Spatial Position Estimation (SPE) step computes the average depth and incident angle of its light source region relative to the image center. This information is fed into a custom Brightness Adjustment Module (BAM), which scales the flare’s intensity according to the physical law that illumination decays with the square of the distance and the cosine of the incident angle. This ensures synthesized flares have realistic intensity proportional to their implied spatial position in the scene.

Stage 3: Physical Engine-Based 3D Rendering (Flare-3D). To complement 2D synthesis and inherently incorporate physics, the authors construct 60 diverse 3D scenes in Blender, placing light sources and flares in semantically appropriate locations (e.g., sun behind a window, indoor lamps). By rendering these scenes from various camera viewpoints, they generate 3,000 image pairs where the flare’s shape, size, color, and position are naturally governed by the 3D geometry and ray-tracing physics of the rendering engine, eliminating the “random placement” issue of 2D synthesis.

The final FlareX dataset is a mixture of the 2D-synthesized (Flare-2D) and 3D-rendered (Flare-3D) data, offering a total of 12,500 image pairs with rich diversity across flare types, lighting conditions (day/night/indoor), and physical realism.

Additional Contribution: Masking-Based Real-World Evaluation. Acknowledging the lack of true ground truth for real flare images, the paper proposes a practical evaluation method. For real captured flare-corrupted images, a masking approach is used to identify the flare regions, which are then inpainted using surrounding background information to create a reliable pseudo-ground-truth. This enables meaningful quantitative evaluation on real-world images, addressing a major shortcoming of previous benchmarks.

Through extensive experiments, the paper demonstrates that models trained on the physics-informed FlareX dataset achieve superior generalization performance on real-world flare removal tasks compared to models trained on previous datasets, validating the effectiveness of their data generation framework. FlareX represents a significant step towards bridging the sim-to-real gap in lens flare removal research.