Skip to main content
Back

HKUST's 3DGS-DET Beats NeRF at Indoor 3D Detection by 14.9 mAP

A new method uses boundary detection and probability-guided sampling to turn 3D Gaussian Splatting into a competitive backbone for indoor object detection.

  • First method to use 3D Gaussian Splatting as the scene representation for indoor 3D object detection.
  • Boundary Guidance paints class-colored object outlines onto training images so 3DGS embeds edge priors.
  • Box-Focused Sampling projects 2D detector boxes into 3D frustums to keep object blobs and drop background.
  • Beats NeRF-Det++ by +6.0 mAP@0.25 on ScanNet and +14.9 mAP@0.25 on ARKitScenes.
  • Reduces noisy background blobs by 25.7% versus farthest point sampling; feed-forward variant trains in 11 hours.
  • Code and models to be released at github.com/yangcaoai/3DGS-DET after acceptance.

Indoor 3D object detection has long leaned on point clouds or, more recently, Neural Radiance Fields. A new paper called 3DGS-DET flips the script by using 3D Gaussian Splatting as the scene representation, and the gains are substantial: +6.0 mAP@0.25 and +7.8 mAP@0.5 on ScanNet, and +14.9 mAP@0.25 on ARKitScenes over the strongest NeRF-based competitor.

The authors, from the Hong Kong University of Science and Technology, claim this is the first attempt to bring 3DGS into indoor detection, and they had to solve two nontrivial problems along the way. The result is an approach that beats both multi-view image detectors and NeRF-Det++, without designing a new detector head.

Why NeRF was leaving performance on the table

Recent work adapted NeRF for indoor detection through view-synthesis representations, treating the neural field as an intermediate 3D scene format that a detector can operate on. The catch is that NeRF's implicit nature limits its representational capacity for detection. Weights inside an MLP encode geometry, so a detector cannot easily grab discrete 3D primitives to reason about.

3D Gaussian Splatting, in contrast, is explicit. A scene is a set of Gaussian blobs, each with a 3D position, scale, rotation, color, and opacity, which is exactly the kind of structured data a point-cloud detector likes. But dropping raw 3DGS into a detector fails for two reasons the paper identifies:

  • Ambiguous spatial distribution: 3DGS is optimized only against 2D pixel losses, so blob positions in 3D can be geometrically loose, with fuzzy boundaries between objects and background.
  • Excessive background blobs: Most pixels in an indoor photo are walls, floors, and clutter, so the reconstructed 3DGS is dominated by noisy background blobs that drown out object signal.

Painting the boundaries into the training signal

The first fix, called Boundary Guidance, is refreshingly simple. Since 3DGS is trained to match 2D images, the authors edit the images themselves to inject 3D structure.