OpenAI Hunted a Rockset Bug and Found an 18-Year-Old Race Condition
OpenAI engineers hunted mysterious C++ crashes for a year, uncovering a hardware fault and an 18-year-old race condition hiding in GNU libunwind.
Takeaways
- OpenAI found two unrelated bugs causing mysterious C++ crashes in Rockset, their ChatGPT data infrastructure service.
- Bug #1 was silent hardware corruption on a single Azure physical host, causing register corruption across multiple VMs.
- Bug #2 was an 18-year-old race condition in GNU libunwind, triggered by a one-instruction-wide window during C++ exception unwinding.
- The breakthrough came from automated population-level analysis of a full year of core dumps, not manual inspection of individual crashes.
- OpenAI switched to libgcc's unwinder as an immediate fix and upstreamed a patch to the GNU libunwind repository.
- The libunwind bug was dormant for 18 years; it only became visible due to Rockset's unusually high exception rate, frequent signal delivery, and a recent signal handler stack growth.
Some bugs don't announce themselves. They show up as a crash, vanish, and leave you staring at a corrupted stack trace wondering if you've lost your mind. That's exactly what OpenAI's infrastructure team faced inside Rockset, the C++ data system that powers ChatGPT's conversation search and knowledge-base connectors. What looked like one inexplicable bug turned out to be two completely unrelated problems, coincidentally surfacing at the same time.
The scene of the crime
Rockset is a cloud-native search and analytics engine that OpenAI acquired in 2024. It's written in C++ for raw performance, and it handles real-time indexing so ChatGPT can retrieve relevant information mid-conversation. The downside of C++ is that memory bugs cause hard crashes, and these crashes were deeply weird.
The symptoms were things that shouldn't happen in normal code:
- A function would finish executing and then return to a NULL address, causing the kernel to kill the process.
- The stack pointer register (
%rsp) would be off by exactly 8 bytes, with no obvious explanation. - Both failure modes crashed on function return, not during execution.
Every hypothesis the team (or ChatGPT) could think of had strong evidence against it, so the bug seemed impossible. A stray write landing precisely on a return address is theoretically possible but vanishingly rare. A stack pointer misalignment with no inline assembly, setcontext, or longjmp in the code path is even harder to explain.
Doctor mode vs. epidemiologist mode
The team's first instinct was to go deep on individual crash dumps -- what they call "doctor mode." They spent days reconstructing pre-crash history from register contents and stack frames. They read kernel source, Azure-specific patches, and ran stress tests. Nothing clicked.
The key shift was deciding to gather high-quality population data. They had ChatGPT write a script that downloaded a prefix of each core file, extracted register state, filtered known false positives, and automatically labeled every crash as return-to-null, misaligned-stack, or other. Then they ran it in parallel over every production core dump from the previous year.
The result was immediate. What they had been treating as one weird bug was actually two separate crash populations. The population-level view made the structure obvious in a way that no amount of individual case analysis could.
Bug #1: the lying host
The misaligned-stack crashes all came from one region, had a clear start date, and never happened on nodes that had been running for a long time. Even though multiple Azure VMs were involved, the pattern pointed at a single physical machine with faulty hardware causing problems for whatever VM happened to land on it.
Armed with a clean list of Kubernetes nodes and timestamps, the team traced the misaligned-stack crashes back to a single physical host, which was easy to denylist. They couldn't reproduce the register corruption in a controlled environment even after weeks of stress testing, but once the bad host was removed from service, that entire class of crashes disappeared.
The fix here wasn't just removing the bad machine. The team also improved their fatal signal handler to log full register state so future hardware corruption is detectable from logs alone, without needing a core dump. They also updated their VM lifecycle policy so VMs are reused rather than recycled, making bad-node detection much easier.
Bug #2: the 18-year-old race condition
With the hardware crashes separated out, the remaining return-to-null crashes became tractable. A key assumption the team had made -- that exceptions weren't involved -- turned out to be wrong. The counterexamples they'd used to rule out exception unwinding were all from the hardware cluster. Once they revisited the remaining cores, they found that the crashes were all happening during exception unwinding.
C++ exception handling is more complex than it looks. When a throw happens, the runtime doesn't just jump to a catch block. It calls into a library that walks the call stack, finds the right handler, runs destructors along the way, and then does a register restore that's closer to a longjmp than a normal return. Rockset's binary links against two libraries that contain implementations of C++ exception unwinding: libgcc and GNU libunwind. GNU libunwind's definitions were the ones chosen by the dynamic linker.
The culprit was in the assembly routine _Ux86_64_setcontext, the function that performs the final register restore when jumping to a catch block. Here's the race:
- The routine builds a
ucontext_tstruct on the stack containing the target register state (including the destination instruction pointer). - It updates
%rspto point to the new stack position -- the moment this happens, the struct is no longer part of the "active" stack. - On the very next instruction, it reads the target instruction pointer (
%rip) from that same struct. - If the OS delivers a signal between those two instructions, the kernel is allowed to write a signal frame at
%rsp - 128, which can overwrite the struct. - The instruction pointer read on the next line gets a corrupted value -- in the observed crashes, NULL.
The vulnerable window is literally one instruction wide. A signal must be delivered after %rsp has been changed, but before the next instruction loads %rip. Several simple instructions like this can be run per cycle on a modern superscalar out-of-order CPU, so the race window is roughly a hundred picoseconds.
Why it became visible now
The GNU libunwind bug is more than 18 years old, present in the first x86_64 version that supported C++ exception unwinding. So why did it only start causing crashes recently? Three factors had to compound:
- Exception rate: Rockset uses exceptions as part of its ingest backpressure mechanism. A single overloaded host can throw on the order of 10,000 exceptions per second.
- Signal frequency: OpenAI's custom CPU accounting tool (
coarse_thread_cputime_clock) delivers aSIGUSR2signal every few milliseconds of CPU time -- far more often than typical programs. - Signal handler stack growth: A recent change added a call to
timer_getoverruninside the signal handler, making it consume more stack -- enough to reach and overwrite the stale struct.
Before that last change, the team observed no crashes at all. After the change, the rate remained low until load increased for use cases that stressed the backpressure mechanism. A back-of-envelope calculation confirmed the math: a 10^-8 probability per exception, times 10,000 exceptions per second, gives a mean time between failures of a few hours per host -- consistent with the observed fleet-wide crash rate of more than a dozen per day.
The fix, upstreamed
The immediate mitigation was switching from GNU libunwind to libgcc's unwinder, which also carries a performance benefit: libgcc's implementation has benefited from a lot of work to reduce lock contention, which matters when scaling to large VMs.
OpenAI also upstreamed a self-contained reproducer and a fix to the GNU libunwind repository. The patch is elegant: instead of reading the target instruction pointer from the stack-allocated struct after %rsp has already moved, it stashes both the target %rip and %rdi values at a known offset below the new stack pointer before adjusting %rsp, then reads them from there. The vulnerable read-after-stack-adjust is eliminated entirely.
The real lesson
The technical details here are genuinely impressive -- assembly-level race conditions, Linux ABI contracts, dynamic linker symbol resolution. But the team is clear that none of that cleverness was the actual breakthrough.
The most important step was not the clever assembly reading or deep knowledge of the details. It was building a high-quality data set. In the absence of this data set, the team was mixing two distinct phenomena into one story and trying to reason their way out of the confusion. Once they had accurate and complete population data, the structure of the problem became obvious.
The broader implication for anyone running C++ services at scale: rare, intermittent crashes that resist hypothesis-driven debugging often benefit more from population-level analysis than from deeper inspection of individual cases. Automated core dump analysis, clean labeling pipelines, and correlation across infrastructure dimensions (host, region, kernel version, deploy time) can turn "impossible" bugs into tractable ones. The bug in GNU libunwind has been sitting in production systems worldwide for nearly two decades -- it just needed the right combination of signal frequency, exception rate, and stack usage to become visible.