Skip to main content
Back

Why AI is revisiting recursive architectures in reasoning

What if AI reasoned without words? New recursive models do exactly that and it's 100x faster.

May 17, 2026

Hey,

We’ve spent the last three years obsessing over LLM parameter counts, but the industry is quietly hitting a wall on complex reasoning. You can only scale autoregressive transformers so far before chain-of-thought token costs and latency become unbearable for production environments.

The fix isn't another trillion parameters. It is a return to one of AI's oldest architectural concepts: recursion. Researchers are now building 5-million-parameter models that outperform massive frontier LLMs on deterministic logic tasks by abandoning text generation entirely.

Today, we break down the revival of recursive architectures and what they mean for the future of AI reasoning.

Author photo
Today's Author
Ben Dickson is a veteran software engineer and former CTO known as the "Engineer's Journalist" for translating technical AI research into practical, production-focused insights. He is a lead contributor to TechCrunch and VentureBeat.
From Datadog State of AI Engineering Report
How are 1,000+ organizations scaling AI in production?

We analyzed real-world LLM telemetry to reveal shifting model adoption, the rise of agentic workflows, and the hidden costs of tech debt.

Benchmark your stack against the industry's largest dataset on AI engineering.
Sunday Deep Dive
The return of recursion: How AI is rethinking complex reasoning

As the AI industry shifts its focus toward handling advanced and complex reasoning tasks, the limits of classic large language models (LLMs) are becoming evident. To address the reasoning bottleneck, researchers are revisiting "recursion," an older architectural concept that circumvents the need to build larger models.

The core of this shift involves transitioning from token-space reasoning to latent-space reasoning. Instead of generating text to "think," recursive architectures loop through continuous internal memory states. This enables them to operate much faster and at a fraction of the cost when solving deterministic problems.

Recursion vs autoregressive models

Classic LLMs are based on autoregressive Transformer models, which means they read the entire input sequence and produce the next token in a single forward pass. The benefit of this architecture is that it can be scaled. Adding more layers and training data continues to improve the performance of the model.

However, it hits a limit when handling problems that require complex reasoning before producing the answer. A model with a fixed number of layers runs out of computation steps for incompressible tasks.

alpha_signal_image_1

To handle these problems, LLMs rely on chain-of-thought (CoT) prompting to simulate reasoning. CoT forces the model to externalize its intermediate "thoughts" token by token. Those thoughts are then fed back to the model to allow it to continue its reasoning until it reaches the answer.

The problem with this approach is that the model is reasoning one token at a time. It can't compress its thoughts into more abstract representations. And this approach becomes slow and memory-intensive as the CoT sequence grows longer.

Recurrent neural networks (RNNs), the predecessor to LLMs, avoided this by continuously looping through an internal memory state. Recursive models keep a fixed memory footprint and, in theory, they can increase their reasoning capacity by increasing the number of processing loops.

However, RNNs were abandoned because compressing information across multiple loops was very difficult. The main challenge of RNNs (and their variants such as LSTM) happened during training, where unrolling memory loops during training caused gradients to vanish or explode.

Hierarchical reasoning models

A recent architecture called the Hierarchical Reasoning Model (HRM) brings recursion back with inspiration from human cognition. HRM uses "latent reasoning," where instead of generating "thinking tokens," the model reasons in its internal, abstract representation of the problem.

HRM solves the problems of classic recurrent models with two coupled, recurrent modules: a high-level (H) module for slow, abstract planning, and a low-level (L) module for fast, detailed computations.

The L-module acts as a fast loop that addresses a subsection of the problem, running multiple steps until it reaches a stable, local solution. It then passes the result to the slower H-module, which updates its overall strategy, and gives the L-module a new, refined sub-problem to work on.

alpha_signal_image_2

This recursive approach enables HRM to simulate the reasoning depth of a much larger network. HRM's efficiency shows itself in the numbers. A 27-million-parameter HRM achieved state-of-the-art results on ARC-AGI puzzles using only 1,000 training examples.

Operating entirely in latent space gives HRM a speed advantage over token-generating models. It delivers up to a 100x speedup in reasoning for deterministic tasks while outperforming zero-shot reasoning on larger LLMs.

Tiny recursive models

Samsung researchers built on HRM to create the Tiny Recursive Model (TRM), testing whether the recursive loop itself was the core driver of performance. They found that HRM's biological arguments and hierarchical nesting could be removed entirely.

TRM replaces the hierarchical structure of HRM with a single, weight-sharing network of two layers. This brings the total model size down to 5 to 7 million parameters.

The researchers optimized TRM by using a full recursive loop. They showed that increasing recursive steps, rather than adding layers, maximized the model's ability to generalize.

alpha_signal_image_3

TRM outperformed both HRM and frontier models on reasoning benchmarks:

  • The 5M-parameter TRM hit 87.4% accuracy on the Sudoku-Extreme dataset. (HRM scored 55.0% on the same benchmark while models like DeepSeek-R1 scored 0.0%.)
  • TRM reached 45% accuracy on ARC-AGI-1 and 85% on difficult maze navigation tasks.

Practical application of recursive computation

Models like TRM and HRM operate as specialized reasoning engines. A TRM trained to solve mazes cannot draft a document or orchestrate a web search. They lack the semantic world knowledge of LLMs and serve as targeted compute modules.

However, they can serve as complements. For language-based and creative tasks, LLMs continue to be superior. But for complex or deterministic tasks, an HRM-like architecture offers superior performance at a lower cost.

Given their efficiency, they are especially suitable for latency-sensitive fields like embodied AI and robotics, or data-scarce domains like scientific exploration.

Latent-space recursion is also finding new ways to solve emerging problems in advanced AI applications. One example is multi-agent systems.

alpha_signal_image_4

Current multi-agent frameworks suffer from token-space limitations. Agents communicate between each other through text tokens, which slows them down and increases inference costs.

A framework called RecursiveMAS applies recursive principles to agent orchestration. Instead of generating text, agents pass their continuous latent representations to each other, collaborating internally and outputting text only at the end.

This architecture enables the agents to recursively reason internally and between them in latent representations without generating any text. It is as if the agents communicate telepathically in their mathematical language and only generate text when they want to provide their final answer.

Bypassing the text generation step changes the economics of agent deployment:

  • It achieves up to a 2.4x end-to-end speedup in inference.
  • It reduces token usage by 75.6% by the third round of recursion.
  • It yields an average accuracy improvement of 8.3% across code generation and medical reasoning benchmarks.

Latent-space recursion is proving to be a scalable approach for building faster, cost-effective autonomous systems.

We are still in the early innings of rediscovering recursion. From HRM to RecursiveMAS, the field has made a lot of progress. As the AI industry moves to remove new bottlenecks, we can expect what is old to become new again.

Sponsor Logo
AI-Powered Observability and Security
Learn More →
At Alpha Signal, our mission is to build a sharp, engaged community focused on AI, machine learning, and cutting-edge language models, helping over 200,000 developers stay informed and ahead. We’re passionate about curating the best in AI, from top research and trending technical blogs to expert insights and tailored job opportunities. We keep you connected to the breakthroughs and discussions that matter, so you can stay in the loop without endless searching. We also work closely with partners who value the future of AI, including employers and advertisers who want to reach an audience as passionate about AI as we are.

Our partnerships are based on shared values of ethics, responsibility, and a commitment to building a better world through technology.Privacy is a priority at Alpha Signal. Our Privacy Policy clearly explains how we collect, store, and use your personal and non-personal information. By using our website, you accept these terms, which you can review on our website. This policy applies across all Alpha Signal pages, outlining your rights and how to contact us if you want to adjust the use of your information. We’re based in the United States. By using our site, you agree to be governed by U.S. laws.
Looking to promote your company, product, service, or event to 250,000+ AI developers?
Work with us