Anthropic says Claude load-tested and hardened the Claude Docs service for two weeks before launch
In an engineering post, Anthropic's Vivek Nair describes a "chaos loop" used before the September 16 launch of Claude Docs, a doc editor that people and Claude sessions (from chat to Claude Code) can work in together. Anthropic says the launch drew over 4.6 million docs by September 22 and about 2 billion requests with only 60 server errors, no rollback, and about 50 memory-optimized pods.
- The loop: from September 4, a Claude Code session coordinated sub-agents (usually about 11 at a time, 30 at the peak) against a staging cluster that mirrored production at over a million synthetic docs a day. It picked a scenario, ran the test harness, read telemetry through the Datadog MCP, proposed a fix, shipped it to staging and repeated. Claude asked for approval before anything hard to undo, such as schema or storage changes, and had to keep two invariants: docs open almost instantly, and an acknowledged edit is never lost.
- A memory fix: Claude traced one large doc's 3 GB of memory to every keystroke copying the doc into several data structures, switched them to copy-on-write, and brought it to about 180 MB. Anthropic says this cut its launch estimate from more than 500 pods to about 50.
- "Legible" choices: the service is written in Rust, with few and vendored dependencies, so that measured numbers trace back to code the team wrote. Anthropic says it did not compare Rust with Go.
- Launch week: Anthropic says it rolled production 629 times in September and that seven of the ten failure scenarios rehearsed on staging happened in production, with fixes already behind runtime flags.
- Cost: Nair says the work used roughly $125,000 of inference at Claude Fable 5.2's API list price, using an internal research model he describes as roughly comparable to Fable 5.2, and that a team of engineers would have needed about six months. For a small service, he estimates, without having run it, $3,000 to $10,000 for a two-week push.
Sources
Get news like this as a push.
AI Tracker alerts you when something changes for the tools you use.