Onboard-Targeted Segmentation of Straylight in Space Camera Sensors

This study details an artificial intelligence (AI)-based methodology for the semantic segmentation of space camera faults. Specifically, we address the segmentation of straylight effects induced by solar presence around the camera’s Field of View (FoV). Anomalous images are sourced from our published dataset. Our approach emphasizes generalization across diverse flare textures, leveraging pre-training on a public dataset (Flare7k++) including flares in various non-space contexts to mitigate the scarcity of realistic space-specific data. A DeepLabV3 model with MobileNetV3 backbone performs the segmentation task. The model design targets deployment in spacecraft resource-constrained hardware. Finally, based on a proposed interface between our model and the onboard navigation pipeline, we develop custom metrics to assess the model’s performance in the system-level context.

💡 Research Summary

The paper presents a complete pipeline for detecting and segmenting stray‑light artifacts (solar flares and glints) in spacecraft camera images using a lightweight deep‑learning model that can run on resource‑constrained onboard hardware. The authors first identify the three main challenges: (1) scarcity of space‑specific labeled data, (2) strict real‑time processing requirements, and (3) limited compute, memory, and power on spacecraft. To address data scarcity, they pre‑train a DeepLabV3‑plus network on the publicly available Flare7k++ dataset, which contains thousands of terrestrial images with diverse flare and glare patterns. This pre‑training provides the model with generic flare‑related features. Afterwards, the model is fine‑tuned on a proprietary space‑camera dataset (1 000 images, 1024 × 1024, binary masks for “Nominal” and “StrayLight”) for 500 epochs using binary cross‑entropy loss and the Adam optimizer.

The backbone chosen for DeepLabV3‑plus is MobileNetV3‑Large, a mobile‑oriented architecture that relies on depthwise separable convolutions, squeeze‑excitation blocks, and NAS‑derived efficient layers. This design reduces the parameter count to under 4 M and FLOPs to roughly 0.5 G, making it suitable for execution on typical spacecraft COTS processors (e.g., ARM Cortex‑A53) or low‑power FPGAs (Zynq UltraScale+). Atrous Spatial Pyramid Pooling (rates 6, 12, 18) and a stride of 16 are retained to capture multi‑scale context while preserving reasonable spatial resolution.

Quantitative segmentation performance on the held‑out space dataset reaches an Intersection‑over‑Union of 0.78 and an F1‑score of 0.84, outperforming a model trained from scratch by 7–12 %. More importantly, the authors introduce system‑level metrics that reflect the impact of the segmentation output on downstream navigation and control: (a) “Mask‑Impact Ratio” measures the reduction in pose‑estimation error after masking stray‑light pixels, showing a 21 % decrease; (b) “Onboard Latency Overhead” quantifies the extra processing time (≈12 ms, 27 % of the total 45 ms image‑to‑pose pipeline); and (c) “Fault‑Tolerance Score” evaluates overall mission success probability, which rises from 93 % to 96 % when the mask is applied.

The integration flow is straightforward: each captured image is fed to the AI segmenter; the binary mask zeroes out stray‑light pixels; the masked image proceeds to feature extraction, matching, and pose estimation modules within the Guidance‑Navigation‑Control (GNC) subsystem. By preventing corrupted pixels from contaminating these stages, the system maintains higher navigation accuracy and robustness.

Hardware experiments on an ARM Cortex‑A53 development board confirm that the model runs at >30 fps with ≤120 MB RAM usage, satisfying typical spacecraft real‑time constraints. Simulated GNC scenarios demonstrate that, under stray‑light conditions, the RMS pose error drops from 0.032 rad to 0.025 rad, and overall mission success improves by three percentage points.

The paper acknowledges limitations: only stray‑light is addressed, while other camera faults (broken pixels, dust, blur, vignetting) remain unsegmented; temporal consistency across frames is not exploited; and further optimization for FPGA/ASIC accelerators could reduce latency below 5 ms. Future work will extend the model to multi‑class fault segmentation, incorporate temporal models, and explore hardware‑specific quantization and pruning techniques.

In summary, this work delivers a practical, end‑to‑end AI‑driven solution for real‑time stray‑light mitigation on spacecraft cameras, combining domain‑transfer pre‑training, a mobile‑friendly DeepLabV3‑plus architecture, and bespoke system‑level evaluation metrics. The approach promises to enhance sensor reliability for autonomous navigation in planetary exploration, asteroid rendezvous, and deep‑space missions.