Skip to main content
Back

QFT-Bench Pushes 39 AI Models Into Particle Physics Research Territory

QFT-Bench puts 39 frontier and open-source models through 75 graduate and research-level quantum field theory problems, exposing where reasoning actually breaks.

Paper, Full Access
  • QFT-Bench introduces 75 graduate to research-level quantum field theory problems across eight subareas, each with a closed-form answer.
  • Evaluation combines SymPy symbolic equivalence checking with an LLM judge for notation-flexible grading.
  • 39 proprietary and open-source models tested; strongest frontier models exceed 95 percent accuracy on nontrivial derivations.
  • Weaker models fail on physical concepts; stronger models mostly fail on arithmetic, conventions, and instruction following.
  • Chain-of-thought analysis finds significant reasoning unfaithfulness: correct answers often lack logically supporting derivations.
  • Project page: qft-bench.github.io; paper at arxiv.org/abs/2610.33134.

QFT-Bench evaluates 39 proprietary and open-source models on 75 advanced quantum field theory problems. The strongest proprietary systems score above 95 percent. A separate analysis finds that some generated derivations fail to support their correct final expressions, showing why answer accuracy alone is an incomplete measure for research workflows.

Why field theory punishes small slips

Quantum field theory supplies the mathematical framework for the Standard Model of particle physics. Its problems often require long symbolic derivations, consistent sign and normalization conventions, and movement among path integrals, Feynman diagrams, operator methods, and renormalization group flows, which describe how physical parameters change with energy.

A misplaced sign, omitted measure factor, or incorrect index can corrupt every later step while leaving the final expression superficially plausible. This combination of conceptual depth and brittle algebra makes QFT a demanding test of whether a model can sustain a derivation rather than recall a formula.

Earlier physics benchmarks have generally emphasized exam-style questions. QFT-Bench moves into advanced graduate and early research territory, with manually constructed problems intended to resemble the technical exercises that precede theoretical research.

Seventy-five problems across eight areas

The benchmark covers eight subareas, from classical field theory and quantization to quantum electrodynamics, quantum chromodynamics, and conformal field theory. Conformal field theory contributes 15 problems, or 20 percent of the dataset.