NTU's RF4D Rebuilds Driving Scenes in Snow and Rain Using Only Radar
A new neural field method reconstructs 4D driving scenes from spinning FMCW radar, keeping moving cars intact where camera and LiDAR pipelines fall apart in snow and rain.
Takeaways
- RF4D is the first neural field method to synthesize radar views of dynamic outdoor driving scenes with moving vehicles.
- A scene flow module predicts per-point motion offsets to keep occupancy temporally consistent across frames.
- A radar-specific power renderer gates reflectivity by occupancy, aligning with the classical radar equation.
- SSIM roughly doubles versus Radar Fields on Oxford RobotCar and Boreas, including snow and rain scenes.
- The method beats CFAR and Bayesian filtering on occupancy without needing any external occupancy labels.
- Code and project page are live at zhan0618.github.io/RF4D; paper at arXiv:2505.20967.
Neural radiance fields have become the default tool for turning a handful of sensor sweeps into a navigable 3D scene, but they inherit the weaknesses of their input sensor. Cameras get blinded by glare, LiDAR clouds shatter into noise in heavy snowfall, and every existing radar-based neural field so far has quietly assumed the world doesn't move. A group from Nanyang Technological University is trying to fix all three problems at once with RF4D, a neural field that takes spinning millimeter-wave radar as its only input and reconstructs outdoor driving scenes that include moving vehicles, in weather that would wreck a LiDAR pipeline.
The paper, accepted to CVPR 2026, is the first neural field for radar that explicitly models time. That single change turns out to matter a lot.
Why radar was the missing modality
The authors represent the scene as a continuous function of both spatial and temporal dimensions, with a scene flow module that promotes temporal coherence by learning motion offsets across frames. The motivation is practical: existing NF approaches that rely on RGB or LiDAR inputs often struggle under adverse weather conditions, limiting their robustness in real-world outdoor environments such as autonomous driving, while millimeter-wave radar is inherently resilient to environmental variations. Radar sees through fog and precipitation, but it has ugly properties: low azimuth resolution, sparse returns, internal noise, and multipath reflections that create ghosts of objects that aren't really there.
The prior state of the art, Radar Fields (SIGGRAPH 2024), took the first real crack at a radar NeRF, but it had two structural problems that RF4D targets directly:
- It only modeled static scenes, so moving vehicles simply disappear from the reconstruction, while RF4D successfully renders the moving object.