Skip to main content
Back

Ant Group's LingBot-Vision Beats a 7B Model With Just 1B Parameters

Ant Group's Robbyant open-sourced a 1B-parameter ViT backbone that treats object boundaries as a pretraining signal, matching 7B DINOv3 on depth benchmarks.

  • Robbyant (Ant Group) open-sourced LingBot-Vision, a 1.1B ViT backbone for dense spatial perception under Apache 2.0.
  • New pretraining objective, masked boundary modeling, treats object boundaries as the primary self-supervised signal.
  • Matches or beats 7B DINOv3 on NYU-Depth v2 while using an order of magnitude less training data.
  • Four sizes released on Hugging Face: ViT-S, ViT-B, ViT-L, ViT-g, all distilled from the giant teacher.
  • Powers LingBot-Depth 2.0, which halves RMSE on hard indoor scenes (0.132 to 0.062) and tops 12 of 16 benchmarks.
  • Read the technical report for the boundary field categorical reparameterization details.

Robbyant, the embodied AI arm of Ant Group, has open-sourced LingBot-Vision, a family of self-supervised Vision Transformer backbones designed specifically for dense spatial perception. Most vision foundation models are trained for semantic invariance: they learn to answer what is in an image while discarding the fine-grained spatial structure that robots depend on. LingBot-Vision inverts that priority, treating boundaries as a native pretraining signal rather than a downstream output, and the payoff is a 1B-parameter backbone that matches or surpasses models up to 7x larger on dense spatial tasks, including the 7B DINOv3.

The release ships alongside LingBot-Depth 2.0, a depth completion model that uses LingBot-Vision as its encoder. Together they target one of robotics' oldest failure modes: glass, mirrors, and transparent surfaces that traditional depth cameras cannot see.

Boundaries as a first-class training signal

The core innovation is a pretraining objective called masked boundary modeling. The authors advocate a boundary-centric view of self-supervised pretraining, in which boundaries are native learning signals rather than downstream outputs, and show they can be bootstrapped from raw images without human labels, external edge detectors, or pretrained backbones.

Mechanically, it is a self-distillation setup with a twist. Masked boundary modeling is a self-distillation framework that forces teacher-discovered boundary tokens into the student's masked set and routes masked tokens by geometry, turning the conflict between semantic abstraction and geometric sensitivity into cooperation. Instead of masking patches at random the way MAE or DINOv2 do, the teacher first identifies which tokens sit on object boundaries, then the student is forced to reconstruct exactly those hard, geometry-rich regions.