Skip to main content
Back

Chalmers' GeM-NR Reshapes 3D Scenes Across Views Without Any Retraining

GeM-NR is a training-free method that propagates a single edited image to all views of a 3D scene, even when the edit drastically reshapes the geometry.

Paper, Full Access
  • GeM-NR is a training-free method that propagates a single edited image to all views of a 3D scene, including edits that change geometry.
  • It handles nonrigid edits (pose changes, object additions, shape deformations) that break most existing multi-view editing methods.
  • The pipeline uses depth estimation, 3D point cloud alignment, and diffusion-based refinement to transfer edits across viewpoints.
  • It is backbone-agnostic: works with FLUX, Qwen, BrushNet, or any image editor without retraining.
  • Outperforms Omni-3DEdit and independent-per-view baselines on both geometric and photometric consistency metrics.
  • No code or weights released yet; the paper is available as a preprint on arXiv.

Editing a 3D scene consistently across multiple viewpoints is one of those problems that sounds simple until you try it. Change the color of a car in one photo, and a standard diffusion model will produce a slightly different shade in the next photo taken from a different angle. Change the shape of an object entirely -- say, turning a sitting dog into a standing one -- and most methods fall apart completely. A new paper from Chalmers University of Technology, GeM-NR, takes direct aim at this harder version of the problem: nonrigid edits that substantially change scene geometry, not just appearance.

Why this is harder than it looks

Most multi-view editing research sidesteps the geometry problem. Text-based image editing has become increasingly powerful with generative models, but while these methods show impressive results on single images, it remains difficult to use them to edit multiple images of the same scene in a consistent manner. The standard workaround is to lean on the geometry of the unedited scene -- use its depth maps and 3D structure to propagate edits across views. That works fine when the edit is rigid (a texture swap, a color change, a weather effect). 3DGS-based scene editing has achieved remarkable success in rigid editing tasks such as style transfer and appearance adjustment, but it is still challenging to edit non-rigid scenes due to more complicated and arbitrarily varying deformations.

The core issue is that nonrigid edits -- adding an object, changing a pose, reshaping a structure -- produce a new geometry that doesn't match the original scene. Current mainstream approaches rely on different 3D constraint techniques, such as epipolar constraints, consistent 3DGS fine-tuning, or depth map guidance, to enforce cross-view consistency. Although these strategies work well for rigid editing tasks like style transfer and appearance modification, there are obvious limitations in non-rigid scene editing.