Skip to main content
Back

Beihang's See-NeRF Reconstructs Sharp HDR Scenes From Blurry Photos

See-NeRF reconstructs sharp HDR 3D scenes from single-exposure blurry images plus event camera data by explicitly modeling sensor physics.

  • See-NeRF reconstructs sharp HDR 3D scenes from single-exposure blurry images plus event camera data
  • Introduces separate physically grounded CRF models for RGB integration and event sensor latency
  • Achieves 26.49 PSNR on real HDR data, beating EvHDR-NeRF by over 7 points
  • Beats multi-exposure HDR-NeRF reference on real data despite using single-exposure blurry input
  • Code and datasets released at github.com/iCVTEAM/See-NeRF
  • Training takes 2.5 hours per scene; 3DGS backbone variant is 50x faster but lower quality

Reconstructing a 3D scene from a handful of casually captured photos is already hard. Doing it when those photos are motion-blurred, the highlights are burned out, and the shadows are pure black is where most Neural Radiance Field methods fall apart. A new paper from Beihang University proposes See-NeRF, a framework that pairs a standard blurry image with data from an event camera and grounds the whole reconstruction in the actual physics of how each sensor works, delivering sharp high dynamic range (HDR) novel views where previous approaches produced smeared, color-shifted results.

Why blurry LDR images break NeRF

Standard NeRF and 3D Gaussian Splatting assume you feed them sharp, well-exposed images. In extreme lighting with both highlight and low-light areas, a typical RGB camera records physical scene radiance through a frame-based integral model and then the image signal processor applies non-linear ISO gain, demosaicing, tone mapping, gamma compression, white-balance, and quantization to yield an 8-bit LDR image. That pipeline throws away the temporal dynamics that cause motion blur and compresses the radiance information you would need to recover overexposed and underexposed regions.

Event cameras are the natural complement here. They asynchronously record brightness changes in the log domain, preserving high temporal resolution and dynamic range information that is typically lost in conventional frame-based RGB cameras. In practical terms, an event stream keeps recording detail in blown-out highlights and pitch-black shadows where an 8-bit image saturates at 255 or clips at 0.