Skip to main content
Back

Ant Group's LingBot-Depth Cuts Robot Depth Errors by 70% on Glass and Mirrors

Ant Group's Robbyant open-sources LingBot-Depth, a masked depth modeling system that cuts indoor depth error by 70% for robotics and 3D vision

  • Robbyant (Ant Group) open-sourced LingBot-Depth, a model that refines noisy/incomplete depth sensor data into metric-accurate 3D maps.
  • Uses Masked Depth Modeling (MDM): randomly masks depth during training, forcing the model to reconstruct geometry from RGB context.
  • Cuts indoor depth relative error by over 70% vs. PromptDA/PriorDA on NYUv2/ETH3D; reduces RMSE by ~47% on sparse SfM tasks.
  • Trained on a curated 3M-sample RGB-D dataset (2M real + 1M simulated), now publicly released.
  • Three pretrained ViT-L models available on Hugging Face; Apache 2.0 license, no hardware changes required.
  • Orbbec will integrate LingBot-Depth into next-gen depth cameras for embodied AI applications.

LingBot-Depth is a new open-source spatial perception model from Robbyant, an embodied AI lab inside Ant Group. The core idea: take the broken, noisy depth maps that real-world RGB-D cameras produce and turn them into clean, metric-accurate 3D measurements -- using the RGB image itself to fill in what the sensor missed.

Robbyant open-sourced LingBot-Depth as a high-precision spatial perception model designed to enhance robots' depth sensing and 3D environmental understanding in complex real-world environments. The code, pretrained weights, and a 3-million-sample training dataset are all publicly available under Apache 2.0.

The sensor problem nobody talks about enough

Depth sensing systems often struggle to capture reliable depth data from transparent or highly reflective surfaces -- glass, mirrors, and polished metal -- due to inherent limitations of optical physics. These surfaces frequently result in missing depth information or severe noise, causing robots to miss objects or miscalculate distances, potentially leading to operational failures or safety risks.

Spatial visual perception is a fundamental requirement in physical-world applications like autonomous driving and robotic manipulation. Capturing pixel-aligned metric depth using RGB-D cameras is the most viable path, yet it usually faces obstacles posed by hardware limitations and challenging imaging conditions. The standard response has been better hardware. LingBot-Depth bets on better software instead.

How it works: masking depth like BERT masks words

The model introduces a masked depth modeling approach that learns robust RGB-D representations through self-supervised learning. The intuition maps directly to masked language modeling: during training, large chunks of the depth map are randomly hidden, and the model is forced to reconstruct them using the RGB image as context. This teaches the model to deeply understand the relationship between visual appearance and 3D geometry.