Article
Anthropic’s Head of Platform Engineering on the New Software Engineering Battlefield:AI Claude handles the coding; humans keep it under control Agent Dongcha Beating AI breaking news: Katelyn Lesse, Head of Platform Engineering at Anthropic, revealed last week that the team she leads had about 200 people this summer, and now all of its code is written by Claude.October 26, she published a lengthy article discussing the more difficult problems ahead for software engineering: if thousands of Agent can collaborate autonomously for weeks at a time, who will manage their permissions, and how can they be stopped when something goes wrong?
The first issue is infrastructure. Traditional Kubernetes are good at managing long-running services, but Agent need to retain their own state and often wait for responses from models, tools, or humans. Allowing large numbers of Agent to sleep at any time and resume quickly without wasting computing resources has become a new engineering challenge. Google has introduced Agent Substrate specifically to manage these workloads.
The permissions issue is even more complex. If an Agent works on a user’s behalf for several weeks, which services can it actually call? What happens to tasks that have already been dispatched if the user revokes authorization midway through? If another Agent sends instructions, how can its identity and permissions be verified?
Anthropic itself has experienced an incident involving excessive permissions. In a October 26-submitted IPO file, it again mentioned the previously disclosed security testing incident: because the test environment had mistakenly retained a public network connection, 4 models once accessed real third-party systems without authorization, with each test run remaining active for approximately 10 to 34 hours. One early model found stored passwords, obtained administrator privileges, continued collecting credentials and modifying system settings, and even accessed personal information.
These incidents all occurred while individual models were executing tasks independently; no collusion between models or deliberate evasion of oversight was found. But collaboration among multiple Agent could further expand the scope of an incident. If one Agent is controlled by malicious instructions, it could pass those instructions to other Agent, or even write them into shared memory, causing subsequent tasks to continue executing incorrect instructions.
Lesse proposed that, in addition to strictly isolated execution environments, execution must be tracked across Agent and that no Agent may tamper with the records. Once an anomaly occurs in the system, an emergency switch must be able to stop the entire system while simultaneously revoking the relevant access credentials.