Anthropic said it had migrated Messages API to Rust，CPU usage fell by approximately 40%to 50%

Article

Anthropic Rewrite the core with Claude API，CPU Usage reduced 40%to 50%Dongcha Beating AI breaking news,Anthropic platform engineering lead Katelyn Lesse revealed in an interview with Latent Space that the company has migrated its Messages API from Python to Rust. After the overhaul,API service CPU usage reportedly fell by approximately 40%to 50%, while first-token response latency improved by approximately 10%to 20%。

Messages API is the primary interface developers use to call Claude. As request volumes continued to grow, the original Python monolithic service consumed increasing amounts of CPU, so the team decided to rewrite it directly with Rust. However, when the project began, the team responsible for the migration did not have even a single Rust engineer.

Claude ultimately handled most of the code writing, while human engineers were responsible for architecture design and validation. The team first built a complete Rust version so that the new and old systems could process the same requests and have their outputs compared one by one, then gradually shifted production traffic. During the process, they also hired several engineers with Rust experience.

Lesse said that migrations of this scale would previously have typically been split into multiple stages and taken years. This time, the team completed the migration of the entire API and went fully live on September.

Original link https://m.theblockbeats.info/flash/374329