InspecSafe-V1: A Multimodal Benchmark for Safety Assessment in Industrial Inspection Scenarios

With the rapid development of industrial intelligence and unmanned inspection, reliable perception and safety assessment for AI systems in complex and dynamic industrial sites has become a key bottleneck for deploying predictive maintenance and autonomous inspection. Most public datasets remain limited by simulated data sources, single-modality sensing, or the absence of fine-grained object-level annotations, which prevents robust scene understanding and multimodal safety reasoning for industrial foundation models. To address these limitations, InspecSafe-V1 is released as the first multimodal benchmark dataset for industrial inspection safety assessment that is collected from routine operations of real inspection robots in real-world environments. InspecSafe-V1 covers five representative industrial scenarios, including tunnels, power facilities, sintering equipment, oil and gas petrochemical plants, and coal conveyor trestles. The dataset is constructed from 41 wheeled and rail-mounted inspection robots operating at 2,239 valid inspection sites, yielding 5,013 inspection instances. For each instance, pixel-level segmentation annotations are provided for key objects in visible-spectrum images. In addition, a semantic scene description and a corresponding safety level label are provided according to practical inspection tasks. Seven synchronized sensing modalities are further included, including infrared video, audio, depth point clouds, radar point clouds, gas measurements, temperature, and humidity, to support multimodal anomaly recognition, cross-modal fusion, and comprehensive safety assessment in industrial environments.

💡 Research Summary

InspecSafe‑V1 is introduced as the first multimodal benchmark specifically designed for safety assessment in real‑world industrial inspection scenarios. The dataset is built from routine operations of 41 inspection robots—both wheeled and rail‑mounted—deployed across 2,239 predefined inspection points in five representative industrial environments: tunnels, power facilities, sintering equipment, oil and gas petrochemical plants, and coal conveyor trestles. In total, 5,013 inspection instances have been collected, each accompanied by synchronized recordings from seven sensing modalities: visible‑spectrum RGB video, infrared thermal video, depth images, LiDAR point clouds, millimeter‑wave radar point clouds, audio recordings, and environmental measurements (gas concentrations, temperature, humidity).

For every instance, the authors provide pixel‑level instance segmentation masks covering 234 industrial object categories, a natural‑language scene description, and a safety‑level label (e.g., normal, warning, hazardous). The language annotations are crafted to reflect practical inspection instructions, enabling research on vision‑language‑sensor fusion for early‑warning systems.

The data acquisition pipeline is carefully engineered. At each inspection point the robot pauses for 10–15 seconds, during which continuous RGB, thermal, and audio streams are recorded, while 3‑second bursts of LiDAR and radar point clouds are captured to obtain stable geometric representations. The dwell time also allows the environmental sensors to sample gas, temperature, and humidity at appropriate rates. To capture diurnal variations, the day is split into two 12‑hour windows, and only the first inspection run in each window is recorded, yielding up to two recordings per point per day. Some critical points are revisited multiple times to characterize temporal dynamics.

A detailed hardware specification is provided: the RGB camera uses a 1/2.8‑inch CMOS sensor with a variable field‑of‑view (55.8°–2.3° horizontally), the thermal camera employs an uncooled detector (53.7° × 39.7° FOV), the depth sensor (Orbbec TM265‑E1) measures from 0.05 m to 5 m, and the LiDAR (MID360) reaches 40 m at 10 % reflectivity (up to 70 m at 80 % reflectivity). The rail‑mounted platform carries a reduced sensor suite due to space constraints, focusing on forward‑facing RGB/thermal, audio, and selected environmental sensors, whereas the wheeled platform integrates the full complement of modalities.

The authors compare InspecSafe‑V1 against a broad range of existing multimodal datasets (e.g., KITTI, nuScenes, MVTec AD, VisA, ScanNet). While many of those datasets focus on autonomous driving, indoor 3D reconstruction, or defect detection under controlled conditions, InspecSafe‑V1 uniquely combines: (1) real industrial disturbances such as strong illumination changes, dust, smoke, specular reflections, and sensor noise; (2) a rich set of safety‑relevant modalities beyond vision and geometry (audio, radar, gas, temperature, humidity); and (3) structured semantic scene descriptions together with safety‑level annotations. This combination enables evaluation of models that must reason about object states, interactions, and environmental hazards—capabilities essential for early warning rather than post‑event detection.

Baseline experiments are presented to illustrate the benchmark’s utility. Single‑modality object detection (RGB only) achieves moderate performance, but multimodal fusion (RGB + thermal + depth + LiDAR + radar) improves mean average precision by roughly 12 % across object categories. Anomaly detection task that incorporates audio and gas measurements shows a notable boost in detecting fire‑related events compared to vision‑only baselines. Finally, a vision‑language‑sensor transformer trained on the dataset can predict safety levels from combined inputs, outperforming a vision‑only classifier by 9 % in accuracy.

The paper emphasizes that the dataset is released with a unified data format, precise timestamp synchronization, and coordinate transformation metadata, facilitating seamless integration into existing deep‑learning pipelines. The annotation guidelines, sensor calibration files, and evaluation scripts are publicly available, promoting reproducibility and encouraging community‑wide benchmarking.

In summary, InspecSafe‑V1 fills a critical gap in industrial AI research by providing a large‑scale, high‑fidelity, multimodal benchmark that captures the complexity and risk patterns of real inspection environments. It offers a solid foundation for developing and evaluating next‑generation safety‑aware perception models, multimodal fusion architectures, and standardized evaluation protocols, thereby accelerating the deployment of trustworthy autonomous inspection systems in hazardous industrial settings.