Skip to main content
Back

NVIDIA's LidarVerse Runs Three Driving Simulation Tasks on One Model

NVIDIA's new LiDAR world model handles generation, forecasting, and resimulation in a single network running at 11 FPS on one H100.

  • NVIDIA's LidarVerse unifies LiDAR generation, forecasting, and resimulation in one conditional generative model.
  • Runs at 11 FPS on a single H100 via a four-step causal student distilled from a bidirectional teacher.
  • Teacher initialized from Cosmos-Transfer2.5-2B, a 28-block DiT with 2048 hidden width and 16 heads.
  • Universal tokenizer handles 32-, 64-, and 128-beam LiDAR and generalizes to unseen scan patterns.
  • Rolling temporal K/V cache plus world-aligned spatial cache enable long-horizon consistent rollouts.
  • Code at GitHub, weights on Hugging Face.

NVIDIA’s Spatial Intelligence Lab has introduced LidarVerse, a conditional generative world model for LiDAR simulation. A world model predicts sensor observations as a scene changes. LidarVerse uses one checkpoint to generate new driving scenes, forecast future point clouds, and replay recorded scenes from altered vehicle trajectories. The distilled causal checkpoint runs at 10 FPS on a single H100.

LiDAR simulation has typically split into two approaches. Reconstruction systems preserve geometry from captured scans and incur minutes or hours of optimization for each scene. Generative systems learn across many scenes and produce new samples quickly, though errors can accumulate during long rollouts. LidarVerse combines learned generation with a persistent memory of observed geometry, allowing the model to retain scene structure without per-scene fine-tuning.

LidarVerse handles LiDAR generation, forecasting, and resimulation with one causal checkpoint
LidarVerse changes tasks by varying the scene evidence supplied to the same checkpoint.

One checkpoint, three jobs

Mode Primary inputs Output Typical use
Generation High-definition map and optional text A new LiDAR sequence with synthesized traffic Creating additional simulation scenarios
Forecasting A prefix of recorded LiDAR sweeps Future point clouds Predicting near-term sensor observations
Resimulation Observed sweeps stored in a spatial cache The scene from a different vehicle path or viewpoint Testing counterfactual driving trajectories

LidarVerse selects among these modes through conditioning, the process of supplying evidence that guides generation. Its conditioning interface accepts a LiDAR prefix, a high-definition map, a persistent spatial cache, and optional text. Changing the available evidence changes the task without loading a separate model.