Skip to main content
Back

AI Code Verification: What a Passing Check Proves

One shared error pulled three of OpenAI's math papers. Write the sentence your check proved under the claim you meant.

Repo, Full Access
  • OpenAI posted 722 AI-generated math manuscripts on October 6, then withdrew 3 papers in its first repo update after a shared sign-error step broke the construction used by 2 dependent papers.
  • In a 12-seeded-bug test, Claude Code fixes passed all 9 required tests (a compile check, a 7-test smoke suite, and the bug's own regression test), yet one rename fix still failed on padded titles.
  • The same OpenAI update also touched 13 additional manuscripts solely so they would cite the revised editions of the withdrawn papers.
  • Two decision models matched the reference verdict on all 12 fixes while a filename-rule heuristic matched only 3, though every picker still caught the one leftover bug.
  • Lauren Tan (@poteto) reports shipping 2,500 PRs in a single month, arguing each should carry a verification artifact rather than rely solely on a passing check.

OpenAI posted 722 AI-generated math manuscripts on October 6, and we all saw the three it pulled (Withdrawals) in its first update to the repo because one shared step was wrong, and the two papers built on it fell with it.

Part of OpenAI's catalog ships with a Lean proof (a language for writing proofs that a small program, the kernel, accepts or rejects).

Thomas Hales (a formal-methods mathematician), in a guest post on Terence Tao's blog, says a Lean proof should not be accepted until the kernel has checked it and a person has confirmed that the theorem is the one they meant.

The same split hits in software. A green check (whether a passing test suite or a kernel-accepted proof) proves the sentence it was handed, and that sentence can sit right next to the claim you meant. The fix is to write the check's sentence under your claim, mark whether the two match, and only then merge.

So, in this piece I will talk about:

  • The three-line card you write before you merge, with the fourth line for dependents.
  • The green build from my own test, where all 9 required tests passed and still proved a neighbor of the claim.
  • What goes stale when a shared result changes, and why callers must run again.
  • Where a small labeling model fits, as a note in a log, never as the merge.

What does a passing check prove?

A passing check proves one sentence, the sentence it was handed, with nothing to say about whether that sentence is the claim you meant.

In math, the handed sentence is the formal theorem. A result counts as formalized once its statement and proof are written in Lean, so the kernel can check them.

So Hales (the mathematician on Tao's blog) splits the work in two: the kernel checks the proof, and a person checks that the formal theorem says what the paper claims. He calls that second job "generally massively easier than checking the proof itself."

Asaf Karagila (set theorist) puts that same failure first when he ranks the ways a formal proof goes wrong: the formal statement does not mean what the written statement means.

Two days earlier he had taken a brief look at the preprint of a set theory paper that OpenAI's catalog lists as formalized, found the writing unclear and muddled, and said a journal should reject it without review. He will not be sending the whisky his Problems page had promised for a solution to that problem.

Still, they weigh expert time differently. Hales calls the statement check the easy half, while Karagila, who grants it for this paper, says hundreds of results at once would swamp the people asked to read them. Neither measured how long the check takes, so I don't pick a winner.

Back in code, the handed sentence is the test's name plus what it asserts, or the command a CI job runs, which is all the green tick vouches for.

The three-line card for a green PR

For a pull request, a card is a short note you write before you merge. This card is three lines written under any passing check before you merge it, with a fourth line for when other work depends on the result:

Claim:        the sentence you wanted to be true
Check proved: what the test, proof, or CI job actually asserted
Verdict:      same | nearby | can't tell
Imported by:  results that depend on this one (their last pass is stale if this check changes)