Advances in large language models have passed a certain threshold that it is now possible to build complex multi-agent systems to solve difficult problems. However, not every problem needs a complex solution.
While orchestrating multiple agents feels like the cutting edge of artificial intelligence, it introduces significant hidden costs. As teams rush to deploy multi-agent systems, they are finding themselves burning through API budgets, battling latency, and dealing with the overhead of orchestration.
In fact, in many cases (if not most), optimizing a single-agent system will match or outperform a multi-agent system. Here’s your guide to choosing single- and multi-agent systems.
Why orchestration is not free
A recent study from Stanford University reveals that many multi-agent benchmarks look impressive simply because they are secretly burning more compute. When the researchers controlled these systems for "thinking budget" (i.e., ensuring both systems use the exact same amount of tokens for reasoning), single-agent systems consistently match or beat multi-agent variants on multi-hop reasoning tasks.
This happens because passing information between agents creates lossy summarization and compounds errors rather than fixing them. The Stanford researchers note that every additional agent introduces communication overhead, more intermediate text, and more places for failures to multiply.
A separate study from Google and MIT provides hard numbers to back this up. Their research shows that independent agent swarms can amplify baseline errors by up to 17.2x.
The Google and MIT researchers also found that “tool-heavy tasks suffer disproportionately from multi-agent coordination overhead.” Their experiments show that in integration setups with 16 tools, single agents achieved a coordination efficiency of 0.466, while multi-agent systems dropped to between 0.074 and 0.234, representing a 2x to 6x efficiency penalty.
Engineering the single agent
With the effective context windows of LLMs steadily increasing, single agents are becoming more effective at handling long-context tasks that span across multiple interactions, documents, and tools.
As the Stanford study recommends, “Stay single-agent when the task can be handled within one coherent context window and when the model can reliably use that context.”
Prompt template for long-thinking single agent systems
However, the Stanford Study found that single agents often fail on complex tasks because they return an answer without reasoning long enough on the task. When this happens, you should not jump to multi-agent orchestration. Instead, you can restructure the prompt to force the model to spend its budget on pre-answer analysis.
This technique involves prompting the model to explicitly identify ambiguities, list candidate interpretations, and test alternatives before committing to a final response. This forces the model to think longer in a single context. The Stanford research states that “some benefits attributed to collaboration can be recovered inside a single-agent setup when the thinking channel is used more effectively."
When you actually need multiple agents
Single-agent systems hit a wall when effective context usage drops. If a workload involves massive, noisy, or partially contradictory retrieval-augmented generation (RAG) data that breaks a coherent context window, a multi-agent system is justified to filter and structure the noise.
The Google and MIT study also identified a "capability saturation" effect. Multi-agent systems can prove useful when a single agent struggles with the task. As a rule of thumb, they found that as long as the accuracy of a single agent on a task is below 45%, using multiple agents can boost performance.
However, if a task is highly decomposable into independent sub-tasks, such as a finance agent splitting revenue analysis and market comparison, multi-agent coordination continues to provide substantial value.
For tasks that are heavily dependent on tools (more than 10) but strictly require a multi-agent setup, developers should default to a decentralized architecture, where agents debate their answers among themselves.
The experiments from the Google/MIT study show that despite the higher overhead, decentralized setups offer superior parallel efficiency with a 66.4% success rate compared to 62.1% for centralized multi-agent systems.
On the other hand, in highly regulated industries like healthcare or finance where strict verification is required, centralized multi-agent architectures offer the best error containment. In this setup, an orchestrator agent controls all sub-agents and cross-checks their reasoning and output before emitting the final response.
"The key differentiator is having a dedicated validation bottleneck that intercepts errors before they propagate to the final output,” the Google/MIT study states.
Developers building these centralized systems must explicitly program the orchestrator to intercept specific failures. Experiments show that cross-checking reduces logical contradictions by 36.4%, and synthesis by the orchestrator reduces context omissions by 66.8%.
The architecture decision matrix
The core principle for developing agentic systems is to treat a strong single-agent baseline as the default starting point, not a weak baseline to be replaced. Developers should only scale complexity when the workload characteristics demand it.
To test architectures internally, developers need to implement a strict evaluation discipline. With open source models, it is easier to measure reasoning traces and control token budgets. With closed models, you have to use workarounds and proxies such as costs and response times. Overall, having a thorough logging system helps keep tabs on the cost/accuracy tradeoffs of your agentic architecture.
Here is how you can map your architecture choices to your workload properties:
- Is the integration tool-heavy (>10 tools)? Build a single-agent system. If a multi-agent system is strictly required, use a decentralized topology for better parallel efficiency.
- Is the system failing due to reasoning depth? Stick to a single-agent architecture, but implement pre-answer scaffolding in your prompts to force the model to compute longer.
- Is the system failing due to context degradation? If you are passing massive, noisy, or contradictory inputs that break a coherent context window, move to a multi-agent system for filtering and structuring.
- Are there natural decomposition boundaries? If sub-tasks can be processed entirely independently, move to a multi-agent system. If tasks need to be accomplished sequentially, stay with a single-agent design.
- Does the output require strict regulatory verification? Move to a centralized multi-agent system and program the orchestrator to aggressively check for logical contradictions and context omissions before aggregating the final output.