Anthropic discloses Claude’s post-response self-evaluation safety mechanism

AT A GLANCE

After responding, Claude reviews abuse risks in the conversation in the background, but it is currently used primarily for post hoc monitoring.

Article

Anthropic’s new safety measure: After every response, Claude also reviews your chat history in the background

Beating AI Newsflash: Anthropic has disclosed a safety mechanism called “self-evaluation.” Starting this year,June after responding to a user, Claude also checks the preceding conversation in the background to assess risks of misuse such as fraud and cyberattacks, and assigns a risk level. Users cannot see this step, and the system does not immediately block the response as a result.

Previously, Anthropic relied primarily on additional safety classifiers to identify violations. Now, Claude can use the conversational context it has just processed to conduct an initial review itself. On average, it only needs to generate about 10 tokens,Token of additional inference compute.

This mechanism has already identified issues missed by traditional safety systems. In 2%this year, a single August customer triggered more than API 100000 risk alerts involving child sexual exploitation.September The system also found that someone had used Claude Code to automate romance scams targeting dozens of victims. Following an investigation, Anthropic banned the relevant accounts and reported the earlier incident to relevant U.S. authorities.

However, Claude cannot always detect malicious activity. In 24 cyberattack tests with standard safeguards disabled, Anthropic attempted to use self-evaluation for real-time blocking, yet 13 attacks still succeeded. Therefore, this mechanism is currently used primarily for post hoc monitoring rather than blocking user actions in real time.

Original link https://m.theblockbeats.info/flash/374555

Key points

Anthropic has disclosed a safety mechanism called “self-evaluation.”
Starting this year,June after responding to users, Claude checks the preceding conversation in the background, assesses risks of misuse such as fraud and cyberattacks, and assigns a risk level.
Users cannot see the review step, and the system does not immediately block responses as a result.
On average, this mechanism only needs to generate about 10 tokens Token, with additional inference compute of less than 2%。
This year, in August, a single API customer triggered more than 100000 risk alerts involving child sexual exploitation.
September the system found that someone had used Claude Code to automate romance scams targeting dozens of victims.
Following an investigation, Anthropic banned the relevant accounts and reported the earlier incident to relevant U.S. authorities.
In 24 cyberattack tests with standard safeguards disabled, 13 attacks succeeded.

AI-assisted interpretation

The following is analysis, separate from reported facts. Verify important claims independently.

Simply put, after Claude responds, it rechecks the preceding chat content for risks such as fraud or cyberattacks. This step is invisible to users and does not immediately block responses; it is currently used primarily for post hoc monitoring.

Why it matters to readers

This indicates that background review can identify some issues missed by traditional safety systems, but attacks still succeeded in testing, so it is not a fully reliable real-time blocking measure.

If you use Claude or related API, you should know that conversations may be reviewed in the background after a response, and understand that risk detection does not mean immediate blocking.

Risks and unknowns

The material does not include Anthropic’s original announcement; specific details still require verification against official materials.
The material does not explain what data retention, privacy processing, or user-notification arrangements are involved in the background review.
The material does not state whether the mechanism will be expanded to real-time blocking.

Anthropic discloses Claude’s post-response self-evaluation safety mechanism

Article

AI-assisted interpretation

Why it matters to readers

Risks and unknowns

Related Developments

Related concepts

Keep following

Security Center