Skip to main content
Back

Ex-OpenAI CTO’s lab makes LLMs consistent

On Mira Murati's paper, open-source Magistral, Notion AI Agents, Claude SDK, and more..

September 19, 2025 4 min read

Hey

Welcome to AlphaSignal, the most read source of news by AI engineers and researchers.


Every day, we identify and summarize the top 1% of news, papers, models, and repos, so you’re always up to date.


Here’s today’s roundup:

Summary

Read time: 4 min 25 sec

Top News

▸ Former OpenAI CTO’s lab releases research making LLMs fully repeatable

HubSpot

▸ Sign up for the HubSpot guide to access 200+ AI side hustles

Signals

▸ Luma Labs releases the first reasoning video model for 4K HDR

▸ Mistral AI improves its open-source Magistral Small reasoning model, adds vision

▸ Anthropic adds tool helpers in beta for Claude SDKs

▸ Notion launches AI Agents to manage multi-step workflows efficiently

▸ Microsoft unveils Fairwater, the world’s largest AI training datacenter

Trending Papers

▸ DeepSeek paper details methods behind R1 breakthrough model

▸ Google finds using every LLM layer makes answers more accurate

▸ Microsoft paper shows in-context learning is fragile, not robust

Top Read

▸ Why “agents” now mean tools in a loop with a goal

Top News

Former OpenAI CTO’s lab releases research making LLMs fully repeatable

alpha_signal_image_2

Thinking Machines Lab, founded by former OpenAI CTO Mira Murati, released research that solves one of the biggest headaches in AI: models giving different answers even when you set temperature to 0.


Normally, you expect the same answer every time, but that’s not what happens. In one test, 1,000 completions of “Tell me about Richard Feynman” produced 80 different outputs.


With the new method, every one of the 1,000 completions was identical. The research paper is titled “Defeating Nondeterminism in LLM Inference.”


Why outputs change today
LLMs return different answers because of how computations run inside the model.

  • When batch sizes change on a server, the math order also changes.

  • Key steps like normalization, multiplication, and attention behave differently.

  • That’s why results vary even if you ask the same thing.

How the fix works
The team rewrote those key steps so the math always runs the same way.

  • Normalization now always reduces numbers in the same order.

  • Matrix multiplication avoids shortcuts that change results.

  • Attention uses fixed rules so sequences process consistently.

Performance trade-off
Making models repeatable slows them down.

  • Standard inference produced 1,000 sequences in 26 seconds.

  • Deterministic inference needed 42 seconds, about a 62% slowdown.

  • Speed is still good enough for research and controlled testing.

How you can try it
The release is open source and ready to use.

  • Works with PyTorch through simple overrides.

  • Runs inside vLLM without heavy modifications.

  • Repository: thinking-machines-lab/batch-invariant-ops.

Why it matters for training
Stable outputs fix a major problem in reinforcement learning.

  • With nondeterministic models, training drifts and rewards collapse.

  • With deterministic inference, training and sampling stay aligned.

  • RL experiments showed zero divergence between training and inference.

Read the paper
alpha_signal_image_3

200+ AI Side Hustles to Start Right Now

From prompt engineering to AI apps, there are countless ways to profit from AI now. Our guide reveals 200+ actionable AI business models, from no-code solutions to advanced applications. 


Learn how people are earning $500-$10,000 monthly with tools that didn't exist last year.

Sign up for The Hustle to get the guide and daily insights.

Get the guide

partner with us

Signals

Luma Labs launches Ray3, a reasoning-powered video model delivering cinematic HDR fidelity

11,485 Likes

Mistral AI enhances Magistral Small, its open-source 24B reasoning model, now supports tool use and vision

1,592 Likes

Anthropic releases tool helpers with validation and automation for Claude SDK workflows

2,617 Likes

Notion introduces personalized AI Agents in 3.0 with memory, custom instructions, and style options

1,461 Likes

Microsoft builds Fairwater datacenter with NVIDIA GPUs, achieving 10x faster AI training performance globally

7,249 Likes

Papers

DeepSeek paper details methods behind R1 breakthrough model

3,515 Likes

The Nature cover article details R1’s $294K training cost and pure RL approach, which replaces human traces. The model invents novel reasoning strategies, scaling beyond human-capped supervised training methods.

Google finds using every LLM layer makes answers more accurate

1,069 Likes

Google introduces SLED, a decoding method that uses logits from all transformer layers. It improves factual accuracy across models without retraining, reducing hallucinations and enhancing reliability in generation tasks.

Microsoft paper shows in-context learning is fragile, not robust

538 Likes

This paper finds in-context learning generalizes poorly across tasks and models. Accuracy varies widely depending on exemplar quality, task type, and robustness to shifts, challenging the “few-shot learning” narrative.

Top Read

Why “agents” now mean tools in a loop with a goal

1,521 Likes

alpha_signal_image_4

Simon Willison, after years of skepticism, accepts a simple, widely-shared definition of “agent”: An LLM agent runs tools in a loop to achieve a goal.


Why this matters
If you’re working with agents, you need shared language. For the first time, “agent” has a concrete definition that cuts through the buzzword noise and makes technical discussions more productive.


Highlights

  • Earlier, “agent” was a buzzword with 200+ conflicting definitions.

  • The new consensus: tools + loop + goal, with a stop condition.

  • Memory isn’t required; conversation history provides short-term state.

  • Business uses of “agents as staff replacements” remain unrealistic. AI lacks accountability.

  • OpenAI’s mixed definitions still add confusion, but the looped-tools framing is winning out.

Read now

At Alpha Signal, our mission is to build a sharp, engaged community focused on AI, machine learning, and cutting-edge language models, helping over 200,000 developers stay informed and ahead. We’re passionate about curating the best in AI, from top research and trending technical blogs to expert insights and tailored job opportunities. We keep you connected to the breakthroughs and discussions that matter, so you can stay in the loop without endless searching. We also work closely with partners who value the future of AI, including employers and advertisers who want to reach an audience as passionate about AI as we are.


Our partnerships are based on shared values of ethics, responsibility, and a commitment to building a better world through technology.Privacy is a priority at Alpha Signal. Our Privacy Policy clearly explains how we collect, store, and use your personal and non-personal information. By using our website, you accept these terms, which you can review on our website. This policy applies across all Alpha Signal pages, outlining your rights and how to contact us if you want to adjust the use of your information. We’re based in the United States. By using our site, you agree to be governed by U.S. laws.

Looking to promote your company, product, service, or event to 250,000+ AI developers?

Work with us