Skip to main content
Back

Google Cloud's ScientistOne Writes AI Research Papers With Zero Hallucinated References

Google's ScientistOne automates the full research pipeline — literature review, experimentation, and paper writing — while being the only system to achieve zero hallucinated references across a 75-paper audit.

Paper, Full Access
  • ScientistOne is Google Cloud AI Research's end-to-end autonomous research system, presented at ICML 2026, built around a verifiability-first design principle called Chain-of-Evidence (CoE).
  • In a 75-paper audit across 5 systems, every baseline had systematic failures: hallucinated references up to 21%, score verification passing in as few as 42% of papers, and method-code alignment as low as 20%.
  • ScientistOne is the only system with zero hallucinated references (0/337), perfect score verification (12/12), and 93% method-code alignment (14/15).
  • It achieved a 40% paper accept rate from automated peer review — triple the best baseline — and matched or exceeded human expert solver performance on all five benchmark tasks.
  • The system runs a three-stage pipeline: literature grounding (reading up to 100 full-text PDFs), parallel solution discovery, and a five-step claim-verified paper writing process.
  • Key limitations include high seed variance, benchmark coverage limited to systems-optimization tasks, and citation verification that checks existence but not semantic support.

Automating scientific research has been a long-standing goal, but most attempts have stumbled on a quiet, structural problem: the systems produce papers that look right but can't be trusted. Citations are fabricated, reported scores don't reproduce, and the code doesn't match the method section. Google Cloud AI Research is presenting ScientistOne at ICML 2026 this week, a new end-to-end autonomous research system built around a single, unusual design principle: every claim must be traceable back to evidence.

The problem nobody was measuring

Autonomous research agents produce competitive solutions and professional-looking manuscripts, yet their outputs can contain verifiability failures undetectable by evaluations that only assess surface presentation rather than evidence grounding: fabricated citations, unreproducible scores, and method descriptions that diverge from the implementation. These failures share a common root: no existing evaluation protocol audits whether claims are supported, and no existing autonomous research system is designed to trace claims back to evidence.

To quantify this, the team ran a systematic audit across 75 papers from five leading autonomous research systems, all evaluated on the same five real-world computer systems tasks. The results were stark: hallucinated reference rates reach 21%, score verification passes in as few as 42% of papers, and method-code alignment ranges from 20% to 80%. Every single baseline system had at least one systematic failure mode.

Chain-of-Evidence: a new standard

The core innovation in ScientistOne is a framework called Chain-of-Evidence (CoE) , think of it as ACID guarantees for research claims. Just as a database transaction must be atomic and consistent, every claim in a CoE-compliant paper must trace through a recorded evidence chain to a grounding source. The framework defines four claim types that need to be verified:

  • Citation claims , the cited paper must actually exist and say what you claim it says
  • Numerical claims , reported scores must trace to a specific execution log entry
  • Methodological claims , the method description must match what the code actually implements
  • Conclusion claims , must derive from verifiable numerical or methodological claims

This is architecture-agnostic , CoE defines what properties a verifiable artifact must have, not how to build the system. The team also released a CoE Integrity Audit, a reusable post-hoc auditing procedure that any system's outputs can be run through.

How ScientistOne is built

ScientistOne is a three-stage pipeline, each stage designed to produce structured artifacts with provenance metadata baked in from the start.

Stage 1: Literature Grounding. A Problem Investigator (PI) agent builds a citation graph starting from 2-4 seed papers, traversing Semantic Scholar up to 2 hops deep to produce a pool of roughly 2,000-5,000 candidate papers. It then filters these down to ~500 relevant ones, has specialist sub-agents read up to 100 full-text PDFs, and synthesizes findings into a structured research brief. Every reference originates from a real API call , this is what produces ScientistOne's zero hallucination rate.

Stage 2: Discovery. An Ideator generates candidate approaches, scores them on novelty and feasibility, and distributes the top proposals across parallel branches of a Parallel Explore-Exploit (PEE) orchestrator. Each branch runs an isolated cycle where a Solver agent iterates, evaluates, and refines. After several iterations, the best-scoring non-violating solution is selected and ablated to isolate the contribution of each component.

Stage 3: Paper Writing and Verification. This is where ScientistOne most clearly diverges from prior systems. The Paper Writer runs a five-stage pipeline:

  1. Conceive , produces a structured narrative where every factual claim carries an inline evidence tag pointing to a specific artifact
  2. Ground , deterministically validates each tag (reported score must match discovery output, baselines must be traceable)
  3. Critic , audits story-level coherence, overclaims, and missing comparisons
  4. Resolve , rewrites the narrative against Ground flags and Critic issues, dropping unsupported claims
  5. Compose , renders verified content into LaTeX, section by section

A final Claim Verifier then checks every claim in the compiled draft against its declared evidence source before the paper is promoted to final output. Only drafts with zero remaining blocking violations pass.

What the numbers show

ScientistOne is the only system to achieve zero hallucinated references (0/337 bibliography entries), perfect score verification (12/12), and the highest method-code alignment (14/15), while matching or exceeding human expert performance on all five tasks.

The peer review scores tell an equally clear story. Using ScholarPeer (an automated peer review system) as a proxy, ScientistOne achieved a 40% accept rate (6/15 papers), tripling the best baseline. On the best-of-3 seed selection, it reached an overall rating of 6.6 out of 10 and had 4 of 5 tasks accepted. The gap wasn't driven by better algorithms , solver scores were competitive across all systems , but by what happened after the solver finished.

On solution quality, ScientistOne exceeded the human expert baseline on every ADRS task and achieved the best scores on Cloudcast and EPLB. For Cloudcast, it combined a Fractional Multi-Commodity Flow LP relaxation with an ensemble of Randomized Shortest Path Heuristics, outperforming both the human expert and all agentic baselines.

Where it still struggles

ScientistOne is not without limitations. The team is candid about several:

  • High seed variance , on the same task, review scores across three seeds ranged from 1 to 8. Rejected runs tended to have qualitative overclaims ("near-optimal") that the Claim Verifier's current coverage doesn't fully catch.
  • Benchmark depth , ADRS tasks reduce research to single-metric optimization. Real systems papers involve problem formulation, multi-dataset analysis, and deployment tradeoffs that the pipeline doesn't attempt.
  • Reference verification depth , the audit checks whether cited papers exist, not whether they actually support the claim being made. Full passage-level verification remains an open problem.
  • Domain coverage , the current evaluation focuses on systems-optimization tasks with deterministic evaluators. Open-ended domains like biology or materials science require domain-specific verification logic not yet built.

What this changes

Sakana AI's AI Scientist pioneered end-to-end automation but operates on fixed ML templates with frequent hallucinations in writing and limited paper quality. ScientistOne's contribution is not to push the autonomy frontier further , it's to make the outputs of autonomous research systems actually trustworthy. The insight is architectural: no existing autonomous research system is designed to trace claims back to evidence. Building evidence chains at claim-production time, rather than reconstructing grounding after the fact, is what separates verifiable from unverifiable outputs.

The broader implication is significant. OpenAI has identified fully automated AI researchers as a core long-term objective. As these systems scale, the ability to distinguish trustworthy AI-generated research from plausible-looking but broken outputs becomes critical infrastructure , not just for research quality, but for the integrity of the scientific record itself. ScientistOne and the CoE Integrity Audit are a concrete step toward making that distinction tractable.

The system is being presented at ICML 2026 in Seoul this week, with live demos and a Q&A at the Google booth (B206). The paper is available on arXiv, and the project website is at scientist-one.github.io. Related work from the same team includes MARS, a modular agent for automated AI research, and PaperBanana, which automates the generation of publication-ready academic illustrations.