UW's TraceLab Reveals Why Coding Agents Cost 60% More Than Expected
A University of Washington team traced 4,300 real Claude Code and Codex sessions to expose where serving stacks actually leak time and money.
Takeaways
- UW's TraceLab releases 4,300 real Claude Code and Codex sessions with 350K LLM steps.
- Prefix tokens drive 59.5% of API cost while output tokens are only 11.2%.
- Median step reads ~120K prefix tokens but emits only ~200 output tokens.
- Top 3 tools account for over 80% of calls; 4% of calls consume 85% of tool time.
- Prefix cache hits 95.7% overall but user-initiated steps miss due to human idle time.
- Dataset, pipeline, and analysis code available at github.com/uw-syfi/TraceLab.
Most public LLM traces still model chatbot traffic, single-turn API calls, or short multi-turn conversations. Coding agents look nothing like that. They run long autonomous loops, hammer the same context on every step, and sit idle for minutes at a time while a human reads the output. A new University of Washington paper called TraceLab puts numbers on all of it, using real traffic from Claude Code and Codex.
The team collected roughly 4,300 sessions containing about 350,000 LLM steps and 430,000 tool calls, gathered from 43 developers over about eight months and spanning both Claude Code and Codex across more than 20 model versions. The dataset, collection pipeline, and analysis code are released at github.com/uw-syfi/TraceLab, and the anonymized traces are available for anyone building serving infrastructure.
Why existing traces miss the point
Serving a coding agent is a different problem than serving a chatbot. Contexts grow over sessions, SLOs are tight, and repeated tool calling puts pressure on both the tool-serving infrastructure and the underlying LLM-serving engines. Nobody publishing serving-systems papers has had good data to work from.
Traces obtained from serving systems like Mooncake and Splitwise capture real user interactions but are not focused on coding agents and lack multi-step tool-call behavior. Capability benchmarks such as Terminal-Bench and SWE-bench measure whether an agent can solve a task, but contain relatively few tasks, each narrowly scoped to a single problem. Replaying a SWE-bench task captures one request in isolation and misses the context growth that accumulates over a real session.
What a real coding session actually looks like
Sessions are long, autonomous, and heavy-tailed. On average a session has more than 9 requests, with a p99 of 137, and most steps are tool-initiated, so the loop is mostly autonomous. Solving one request takes on average around 8 steps with 11 tool calls. Each step issues slightly more than one tool call, so parallel tool use exists but is not the norm.