Skip to main content
Back

TaxiGPT's Illegal Turns Traced to a Compressed Internal Map

A mechanistic look inside TaxiGPT reveals a faithful Manhattan map, a goal compass, and superposition interference behind its illegal turns.

  • TaxiGPT's illegal turns come from superposition interference, not from a broken internal Manhattan map.
  • Causal analysis reveals intersection features, position tracking, and a circular goal compass at layer 16.
  • Steering the goal compass redirects the taxi; ablating it cuts goal-reaching without producing illegal moves.
  • Affordance packing groups intersections with identical legal moves, softening the cost of superposition slips.
  • Failures split into fatal slips, silent slips, full corruption, and give-up slips, each fixable by targeted edits.
  • Code, indicators, and an interactive rides viewer are on GitHub and the project site.

A mechanistic reanalysis of TaxiGPT argues that the model learned Manhattan’s street structure despite producing illegal turns about 1% of the time. Its errors appear when overlapping location features interfere with one another, causing the model to act from the wrong intersection.

The 1% error that fueled the debate

Vafa et al. trained TaxiGPT models from scratch on paths through Manhattan. The GPT-2-style transformers predicted each taxi’s next turn from tokens representing its origin, destination, and previous turns, with outputs such as N, NE, and NW.

The strongest model trained on random walks produced legal turns 99% of the time. Researchers then reconstructed the apparent street map from its behavior and found a tangled graph. That result supported the interpretation that strong next-turn accuracy could conceal an incoherent internal world model.

Pierre Beckmann, Matthieu Queloz, and André Freitas challenge that interpretation in a mechanistic reanalysis. Their experiments indicate that generated routes provide an incomplete view of the model’s spatial knowledge because route quality combines several separate capabilities.

Three navigation systems emerge

The researchers inspect TaxiGPT with probes, causal steering, and ablations. A probe is a small decoder trained to recover information from frozen model activations. Steering changes a selected activation and measures the resulting behavior, while ablation removes a feature to test whether the model depends on it.

System Evidence Behavioral role
Street-map representation Probes recover intersections, connections, and legal outgoing directions. Encodes Manhattan’s graph structure.
Position tracker