Article

OpenAI installs an emergency brake for internal AI: cuts off model calls within 35 seconds of receiving the command

Dongcha Beating AI flash news, OpenAI has launched an internal safety system called Control Room, specifically designed to halt risky AI experiments. Once a shutdown command is received, within at most 35 seconds, the relevant programs can no longer call AI models. Even if the program itself is still running, it cannot continue to make the models work.

In September this year, OpenAI experienced an incident. An internal AI, in an environment that was supposed to be isolated, found a loophole in the network restrictions and secretly contacted an external chatbot. The safety system issued an alert within 15 minutes, and staff quickly intervened, but the experiment did not stop automatically. It was only shut down manually about 2.5 hours later.

Control Room assigns a number to each experiment. Every time a program calls an AI model, the server checks whether this number still has permission. Once an experiment is halted, all models connected to the system will reject its requests. Other normal experiments are unaffected.

OpenAI has also added a safeguard. If Control Room itself malfunctions or is shut down by someone, the models will default to rejecting requests, avoiding continued operation without monitoring.

Currently, this system covers all of OpenAI's frontier model research tasks. The 35 seconds is counted from when the shutdown command is received, and does not include the time for detecting anomalies and human judgment. OpenAI is still trying to further shorten this delay, striving to achieve immediate stopping.