RadarGen: Automotive Radar Point Cloud Generation from Cameras

RadarGen: Automotive Radar Point Cloud Generation from Cameras Tomer Borreda1 Fangqiang Ding1,2 Sanja Fidler3,4,5 Shengyu Huang3 Or Litany1,3 1 Technion 2 MIT 3 NVIDIA 4 University of Toronto 5 Vector Institute https://radargen.github.io/ Figure 1. Controllable radar synthesis from vision. (Top) Given multi-view camera images, RadarGen generates realistic radar point clouds that align with real-world radar statistics and can be consumed by downstream perception models. (Bottom) The generation is semantically consistent: modifying the input scene with an off-the-shelf image editing tool (e.g., replacing a distant car with a closer truck) updates the radar response, removing returns from newly occluded regions and reflecting the new object geometry. Abstract We present RadarGen, a diffusion model for synthesizing re- alistic automotive radar point clouds from multi-view cam- era imagery. RadarGen adapts efficient image-latent diffu- sion to the radar domain by representing radar measure- ments in bird’s-eye-view form that encodes spatial struc- ture together with radar cross section (RCS) and Doppler attributes. A lightweight recovery step reconstructs point clouds from the generated maps. To better align generation with the visual scene, RadarGen incorporates BEV-aligned depth, semantic, and motion cues extracted from pretrained foundation models, which guide the stochastic generation process toward physically plausible radar patterns. Con- ditioning on images makes the approach broadly compat- ible, in principle, with existing visual datasets and simu- lation frameworks, offering a scalable direction for multi- modal generative simulation. Evaluations on large-scale driving data show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data, marking a step to- ward unified generative simulation across sensing modali- ties. 1 arXiv:2512.17897v1 [cs.CV] 19 Dec 2025 1. Introduction Recent advances in neural and generative simulation have made it increasingly practical to synthesize photorealistic data at scale for autonomous driving. By reconstructing real scenes with neural fields or generating entirely new ones us- ing video diffusion models, these systems can produce di- verse and controllable environments that closely mimic real sensor observations. This capability enables large scale res- imulation of traffic, lighting, and weather conditions with- out costly rerecording or manual setup [43, 51, 62]. De- spite this rapid progress, most neural simulators remain lim- ited to the visual domain, focusing on the generation of RGB imagery and video. Recent efforts have begun ex- tending these ideas to LiDAR, demonstrating controllable three-dimensional point cloud generation from camera in- puts [57, 60, 90]. Radar, however, remains an open fron- tier. Although it is already ubiquitous in production vehi- cles, providing low cost, lightweight, and weather resilient perception, it has received far less attention within the gen- erative modeling community. This imbalance limits the fi- delity of current neural simulators, which cannot reproduce radar’s distinctive sensing characteristics, including signal sparsity, radar cross section (RCS), and Doppler. Generating radar data poses unique challenges. Radar measurements exhibit strong stochasticity due to multipath reflections, interference, and material-dependent scattering that vary with scene geometry. Operating at longer wave- lengths, radar interacts with surfaces and internal structures beyond what cameras or LiDAR perceive, making it highly complementary yet difficult to model from vision alone. A further challenge lies in the nature of available radar data. In most large scale driving datasets, radar is provided only after proprietary signal processing that converts raw radio frequency waveforms into sparse point clouds with RCS and Doppler values. This processing chain, which includes range Doppler transforms, beamforming, and detection al- gorithms such as constant false alarm rate (CFAR), is closed and lossy, discarding phase and other fine grained signal information. In practice, storing raw radar signals is ex- tremely memory intensive, so even commercial survey ve- hicles often record only processed point clouds. As a result, point clouds remain the practical representation of radar data for large scale learning. To address these challenges, we propose RadarGen, a generative framework for synthesizing automotive radar point clouds directly from camera imagery. RadarGen learns a distribution over radar observations conditioned on the visual scene, producing diverse and semantically consis- tent measurements rather than a single deterministic predic- tion, reflecting the inherent stochasticity of real radar sig- nals. Conditioning on images allows RadarGen to leverage existing visual data and simulators, providing a scalable and modular way to enrich them with realistic radar s

…(Full text truncated)…

This content is AI-processed based on ArXiv data.