Skip to main content
Back

Stanford's Mulligan Forces Robots to Repeat Failures Until They Improve

Stanford's Mulligan targets a robot's weak spots by restarting failed initial states, boosting real-task success by 10 to 34 points.

  • Stanford's Mulligan restarts failed initial states to focus operator time where the policy actually breaks.
  • HiL-IDQL+Mulligan improves final real-task success by 10 to 34 percentage points over HG-DAgger.
  • Evaluated on 2,550 blinded held-out episodes across three precision manipulation tasks.
  • A value function trained on failed rollouts reranks actor outputs, salvaging data imitation discards.
  • Operator plus robot completes 98% of collection episodes, with no round below 90%.
  • Code, data and paper at mulligan.page and GitHub.

Once a robot policy succeeds on 90% of trials, uniform data collection wastes much of the supervisor’s time on behaviors the model has already learned. Stanford researchers developed Mulligan to concentrate training on the remaining failures. After each evaluation round, the system prioritizes the starting configurations that defeated the policy.

In their paper, Lars Ankile, Chelsea Finn, Shuran Song, and collaborators report gains of 10 to 34 percentage points over an HG-DAgger baseline across three real-world manipulation tasks. The evaluation covered 2,550 blinded, held-out episodes involving tight clearances and broad ranges of initial object poses.

Why DAgger Starts to Idle

DAgger, short for Dataset Aggregation, repeatedly runs a policy, collects expert corrections, and retrains on the expanded dataset. Its human-gated variant, HG-DAgger, lets an operator intervene when the robot approaches failure. Both methods can improve a working policy, but their collection efficiency falls as success rates rise.

When most initial placements are easy, the operator spends much of each session watching successful episodes. Those trajectories add limited information while the difficult regions of the workspace remain sparsely sampled.

The researchers found that failures clustered around particular starting conditions. A nut at a troublesome yaw, a tilted marker, or a cable at an awkward angle could repeatedly break the policy while nearby configurations succeeded.