Anthropic Rebuilt Its Core API in Rust Without Rust Engineers

Key Takeaways

• Anthropic migrated its primary Messages API from a Python monolith to Rust despite having zero Rust developers on the engineering team.
• The company relied on Claude to write the implementation code while human engineers focused on systems architecture, interface planning, and validation.
• The engineering team ran a full shadow fork of the API against live production traffic to verify parity between the Python and Rust outputs.
• The rewrite reduced CPU consumption by 40 to 50 percent across Anthropic's cloud infrastructure.
• The migration dropped time-to-first-token latency by 10 to 20 percent, directly speeding up responses for end users.

The Method

When cloud providers run out of compute capacity, software efficiency stops being an academic debate. Anthropic hit severe CPU limits across its cloud providers while running the Messages API on Python. Python works well for fast prototyping, but its runtime overhead eats server cycles. Rewriting a core API from scratch is usually a long slog that requires hiring systems specialists.

Anthropic took an uncommon path. Katelyn Lesse explained the starting point: “We chose Rust, which was a crazy decision at the time because we had a team of people like on paper it's obvious, but our team was like, 'We don't have a single Rust developer in the room. Who cares?'”▶ 56:38

Instead of hiring a new team, Anthropic treated Claude as the programmer and its engineers as technical directors. Lesse described the division of labor: “Claude is going to write the code. The humans design the system, right? So the humans are very involved in making sure it's a distributed systems architecture.”▶ 56:54

The rollout followed three operational steps:

First, human engineers defined the target architecture. They mapped every boundary, state transition, and error path in the Python service. The engineers did not write boilerplate. They wrote specifications and integration criteria that Claude used to generate the Rust codebase.

Second, the team built a complete parallel fork of the API. Rather than swapping out individual routes, they stood up a duplicate Rust service alongside the existing Python infrastructure.

Third, they instituted strict live-traffic shadowing. The team routed incoming production calls to both services simultaneously. Lesse recalled: “What we ended up doing was we literally built like an entire fork of the entire API in Rust and then started running shadow traffic through it and verifying that the outputs that we were getting from the Python system and the Rust system”▶ 57:25 matched.

If the Rust fork produced differences in serialized headers or error payloads, engineers fed the traces back into Claude to patch the code. Once outputs matched across millions of shadow requests, they cut traffic over.

The payoff was immediate. Lesse noted: “Ballpark. I guess I would say like we've seen somewhere like 40 to 50% something like really crazy. And then latency, I think we saw somewhere between like 10 and 20% improvements on time to first token specifically.”▶ 58:50

Where This Breaks Down

This method requires two ingredients: output determinism and strong systems architects.

If your service relies on undocumented edge cases, an AI model will generate clean Rust code that subtly breaks business logic. Anthropic succeeded because the Messages API had strict request and response contracts that allowed direct comparisons on shadow traffic. Without test harnesses and shadow traffic infrastructure, you will ship silent regressions.

Second, generating code in a language nobody on your team knows creates operational risk. If a service locks up during an outage due to an allocator panic or channel deadlock, you cannot wait for an LLM to explain memory dumps. Someone must understand operating system calls and memory profiles. Anthropic did not replace systems knowledge; they replaced typing syntax. If your team lacks distributed systems experience, letting an LLM generate low-level code will only build faster failures.

What to Do With This

Inspect your cloud bill tomorrow morning and identify your highest-cost Python service. Set up a shadow proxy that duplicates 1% of read traffic to a containerized test endpoint. Feed the original service code and endpoint schema into Claude to generate a Rust equivalent, then compare the responses over 48 hours before writing a single production route.