Skip to main content
Back

MIT's Mirror Teaches AI Agents to Master Minecraft Without Human Help

MIT's Mirror learns to turn video plans into real actions through self-play, becoming the first agent to craft a Minecraft iron pickaxe with no rewards or action labels.

Paper, Full Access
  • MIT's Mirror learns video-to-action mappings via self-play, no rewards or action labels needed.
  • First method to craft a Minecraft iron pickaxe through self-improvement alone, roughly 9x DreamerV3's rate.
  • Sets new state of the art on offline Minecraft diamond acquisition using VPT human data.
  • Closed-loop replanning after every action chunk prevents long-horizon drift from compounding.
  • Built on a 14B-parameter Cosmos3-Nano causal video planner with sliding-window KV caching.
  • Emergent in-context learning: 76% lava-escape rate over 30 lives when planner context is retained.

MIT researchers have introduced Mirror, a self-supervised system that learns to control an embodied agent by attempting to follow generated videos. The system creates a video plan from the agent’s current view and goal, predicts the controller inputs needed to follow that plan, executes those inputs, and trains on the resulting frames and recorded actions.

Mirror does not require reward functions, teleoperated demonstrations, or expert action labels for its online training loop. The executed actions still provide labels, but the agent generates them through its own attempts rather than receiving them from a human demonstrator.

The authors report that Mirror is the first self-improving method to obtain an iron pickaxe in Minecraft survival without rewards or expert action labels. In a separate offline setting that uses expert-labeled data, it sets a new reported state of the art for obtaining diamonds.

Pixels need a controller

Video generators can propose a visual sequence for completing a task when conditioned on a current frame and a text goal. An embodied agent still needs a video-to-action model, also called an inverse dynamics model, to infer which controller inputs could produce the transition between consecutive frames.

Conventional video-to-action training relies on large collections of expert video paired with the exact actions that produced it. Robotics teams often collect those pairs through teleoperation, while game-agent researchers use instrumented gameplay logs. The process is expensive, tied to a specific action space, and difficult to scale across environments.

Plans turn attempts into labels

Mirror replaces much of that expert collection process with a repeating interaction loop:

  1. Generate a short video plan from the current observation and task instruction.
  2. Use the video-to-action model to predict actions for the first segment.
  3. Execute those actions in the environment.
  4. Record the actual frames and issued commands as training pairs.
  5. Update the video-to-action model and repeat from the new state.

Each attempt contributes training data even when the observed trajectory diverges from the generated video. Because the agent knows which commands it issued, it can train the controller on the transition that actually occurred.