GaNI Lets Your Phone Scan a Coffee Table's Exact Materials in the Dark
A two-stage neural pipeline reconstructs geometry, albedo, and roughness of multi-object scenes from a phone flashlight capture, handling global and near-field lighting.
Takeaways
- GaNI reconstructs geometry, albedo, and roughness from phone flashlight captures of multi-object scenes.
- Two-stage pipeline: NeuS-based geometry, then InvNeRad-based reflectance with global illumination.
- Key trick: single radiance cache conditioned on flashlight position replaces one cache per frame.
- Albedo MSE 0.066 vs IRON's 0.552 on synthetic scenes; re-render MSE 0.014 vs 0.129.
- Beats NeILF++ on reflectance under natural light, matches on geometry, without requiring outdoor lighting.
- Limitations: needs dark room, per-scene optimization, not yet room-scale; paper on arXiv.
Pointing your phone at a coffee table with the flashlight on and recovering the exact materials of every object on it has been an unsolved problem. Existing inverse rendering methods that use a co-located light and camera work beautifully on isolated objects, but collapse when you drop them into scenes with multiple objects bouncing light off each other. A new method called GaNI from researchers at the University of Maryland and UNC Chapel Hill fixes that by explicitly modeling the two phenomena that trip up prior work: global illumination between objects and near-field falloff from a nearby point light.
Why single-object tricks fall apart
Inverse rendering means decomposing images into geometry, material reflectance (typically parameterized by a BRDF, or Bi-directional Reflectance Distribution Function), and lighting. To make the problem well-constrained, researchers capture images with a co-located flashlight and camera, a setup easily reproduced with a mobile phone where light and camera positions can be calibrated using Structure-from-Motion. Methods like IRON and WildLight nail this for a single object on a turntable.
Scenes with multiple objects induce a pronounced near-field effect, where different regions receive different light intensities, and interactions between objects cause strong inter-reflections or global illumination. A phone flashlight is not a distant sun; it is a point source inches away, so the inverse-square falloff matters a lot, and light bouncing off a shelf back onto a nearby cup wrecks any assumption that observed color equals surface color.