Weights & Biases Rebuilds Weave to Fix AI Agents Breaking in Production
Weave gets rebuilt around sessions, turns, and steps, with automatic signals that turn live production traffic into the next training set.
Takeaways
- W&B shipped a rebuilt Weave with end-to-end observability for production agents, free to try.
- Sessions, turns, and steps are now first-class concepts instead of generic spans.
- OpenTelemetry-native ingestion means no custom instrumentation if your harness emits OTel.
- Signals auto-flag frustrated users, unsafe content, and code failures from live traffic.
- Works with Claude Agents SDK, OpenAI Agents SDK, Google ADK, OpenClaw, and LangChain.
- Part of CoreWeave's broader push to close the training-to-inference loop with Serverless RL.
Weights & Biases just shipped a rebuilt version of W&B Weave aimed squarely at one problem: how do you keep an agent reliable once it leaves your laptop and starts handling millions of real conversations? The new release treats production traffic as the primary source of truth for agent quality, not a place where things break after the fact.
The pitch is that Weave now provides end-to-end observability to monitor production agents, out-of-the-box signals to surface failure modes, a complete improvement loop from inference to training, and a flexible evaluation framework to prevent regressions. It is free to try, and the company is positioning it as the observability layer for a broader push by parent CoreWeave to close the gap between training and inference.
Why stack traces stopped working
The framing behind the release is worth pausing on. The W&B team argues that agent tracing is no longer just code tracing. A stack trace doesn't describe what an agent did. You need first-class semantics: sessions, turns, steps, tools, sub-agents. Generic observability tools don't speak this language.
They go further on evaluation: LLMs are so good that individual traces are no longer the most useful debugging tool. Aggregate metrics and trends across millions of traces are needed to identify misbehaviors. Offline evals are getting harder to design, as the space of possible valid behaviors is very wide. Signals from online traffic is a more powerful diagnosis tool. In other words, the behavior space has gotten too wide for hand-built eval sets to keep up, and the real failures live in production.
What actually shipped
Four pieces stand out in the new Weave:
- Agent-native tracing. Sessions, turns, and steps are now first-class objects that roll up into the agent itself. You can read a session the way the agent actually executed it, then zoom out across thousands of sessions to spot behavior drift.
- OpenTelemetry-native ingestion. Weave's agent tracing is built on top of the OpenTelemetry GenAI Semantic Conventions out of the box, so if your harness emits OTel, Weave reads it with no custom instrumentation.
- Harness ubiquity. It works with the Claude Agents SDK, the OpenAI Agents SDK, Google ADK, OpenClaw, LangChain, and custom harnesses, with automatic session tracing and a new semantic SDK.
- Signals. Weave continuously analyzes your production traffic and surfaces behavioral issues: frustrated users, unsafe content, code failures, etc. Those signals can then be promoted into offline eval datasets.

The Signals piece is the part most teams will care about. Monitoring millions of incoming traces and manually scoring them to extract signals is virtually impossible. Built-in and custom signals automatically capture and classify agent interactions, so you always know how your agents are behaving. Alerts route what matters through Slack notifications and trigger webhook automations, turning every production insight into a fast iteration.
The bigger loop behind it
Weave isn't shipping in isolation. It is the observability layer of CoreWeave's new agentic AI stack, which the company has been pitching as a closed loop between training and inference. Until now, training reliable AI agents meant running lengthy offline evaluations for months before releasing them to real users for inference. Not only was this process too slow, but the agents often failed because the eval datasets couldn't cover all possible real-world scenarios. As AI accelerates the path toward superintelligence, that process is no longer viable. CoreWeave eliminates this bottleneck, enabling enterprises to close the loop between training and inference.
The plumbing on the training side is aggressive. The platform integrates four components: Serverless RL for training language models without infrastructure management, CoreWeave Inference for production deployment, W&B Weave for observability across agent systems, and W&B Skills with MCP server for autonomous improvement capabilities. The company's Serverless RL service scales with training workloads and reduces costs by up to 40% while accelerating training by approximately 1.4x compared to local H100 GPU environments, according to the company.
What it is useful for
If you are running an agent in production behind any of the popular harnesses, the immediate value is dropping in OTel-based tracing and getting session-level views without writing custom spans. The bet worth thinking harder about is whether you buy the broader thesis: that offline evals are a losing battle and the iteration loop should live where the agent actually runs.
Industry watchers are framing this as a real shift in how agent quality gets owned. As one analyst put it, "Most enterprises are stuck in a cycle of building and testing agents before they ever reach real users, and that cycle is becoming too slow and too expensive to sustain," said Nick Patience, Vice President & Practice Lead, AI Platforms, Futurum. "A platform that closes the production-to-development feedback loop, using real-world experience to automatically improve agent performance, addresses a critical bottleneck standing between enterprises and user-ready agentic AI."
The honest caveat: a system that learns from production traffic is only as good as the signals it picks up and the guardrails around what gets promoted into training data. Teams adopting this loop will need to be deliberate about what "frustrated user" or "unsafe content" means for their domain, and how aggressively they let online evals drive offline ones. Weave gives you the loop. The judgment calls are still yours.