Skip to main content
Back

Valeo.ai's IGLOSS Teaches LiDAR to Spot Unseen Objects Without Camera Data

Valeo.ai's IGLOSS swaps CLIP text embeddings for text-to-image prototypes, setting new open-vocabulary segmentation records on nuScenes and SemanticKITTI.

  • IGLOSS achieves state-of-the-art open-vocabulary lidar segmentation on nuScenes and SemanticKITTI.
  • Replaces CLIP text embeddings with text-to-image prototype pictures to dodge the image-text modality gap.
  • Uses a 3D backbone (ScaLR+) distilled from DINOv2 rather than from a VLM.
  • Inference matches 3D point features to 2D prototype features via test-time multinomial logistic regression.
  • Also sets SOTA on annotation-free closed-set segmentation on both datasets.
  • Code, pretrained weights, and generated images released at valeoai/IGLOSS.

Valeo.ai researchers propose IGLOSS, a system that uses generated class images for open-vocabulary semantic segmentation of LiDAR point clouds. The NeurIPS 2026 paper reports state-of-the-art results on nuScenes and SemanticKITTI, while the released GitHub repository includes code, weights, and prototype images.

IGLOSS moves class matching into a DINOv2-derived visual feature space shared by generated images and 3D point features. This design removes the vision-language text encoder from the matching stage and lets the deployed model process LiDAR without synchronized camera images or scan sequences.

Diagram comparing IGLOSS with conventional open-vocabulary 3D segmentation pipelines
IGLOSS represents target classes with generated images and matches them to DINOv2-aligned LiDAR features.

Open-vocabulary semantic segmentation assigns a class to every pixel or 3D point while allowing the evaluation vocabulary to include categories absent from model training. A common pipeline distills a vision-language model such as CLIP into a 3D backbone, then compares each point feature with text embeddings for candidate class names.

CLIP aligns images and text in a shared vector space, but the two modalities retain different feature distributions. That modality gap can reduce accuracy when image-trained representations are transferred to sparse 3D sensor data. Prompt wording adds another variable, with phrases such as “a photo of a truck” and “a LiDAR scan of a truck” producing different class vectors.

Class names become image banks