OpenAI, Google, Microsoft, and Anthropic reportedly adopt unified AI training opt-out rules

Article

OpenAI, Google, and other giants unify AI training opt-out rules, eliminating the need for websites to configure them company by company

OpenAI, Google, Microsoft, and Anthropic have reached an agreement: going forward, websites need only add `robots.txt` and `User-Agent: __usage_ai-training` and `Disallow: /` to simultaneously tell these companies not to use their content to train AI. Previously, websites needed to configure rules separately for crawlers such as GPTBot、ClaudeBot.

These rules apply only to AI training; they do not also prohibit crawlers from crawling websites, nor do they necessarily affect AI citing website content in responses. Proposed by an Anthropic engineer, they are currently a technical draft of the Internet Engineering Task Force (IETF).

The companies do not implement the rules in exactly the same way. Microsoft allows websites to separately prohibit AI training while retaining the possibility that content may be cited in AI search. Google, meanwhile, plans to link this rule to its existing Google-Extended setting, potentially also restricting Gemini from citing website content.

Anthropic has already begun adopting it; Microsoft plans for it to take effect in 2027;February; Google plans to support it in 2027; and OpenAI will launch it in the coming months.

However, these rules cannot withdraw data already used for training, nor can they compel AI companies to comply.

Original link https://m.theblockbeats.info/flash/374854

Key points

The brief states that OpenAI, Google, Microsoft, and Anthropic have reached an agreement.
Websites can add robots.txt to User-Agent: __usage_ai-training and Disallow: / to tell these companies simultaneously not to use website content to train AI。
This rule applies only to AI training; it does not also prohibit crawlers from crawling websites, nor does it necessarily affect AI citing website content in responses.
These rules are currently a technical draft of the Internet Engineering Task Force (IETF).
Anthropic has already begun adopting it; Microsoft plans for it to take effect in 2027;February; Google plans to support it in 2027; OpenAI will launch it in the coming months.
These rules cannot withdraw data already used for training, nor can they compel AI companies to comply.

AI-assisted interpretation

The following is analysis, separate from reported facts. Verify important claims independently.

Websites previously had to separately configure crawler rules for GPTBot、ClaudeBot and others. According to the brief, in the future they may use a robots.txt rule to express an opt-out of AI training, but the companies' implementation methods may differ.

Why it matters to readers

Website operators may reduce the work of separately maintaining multiple sets of opt-out rules; meanwhile, whether website content continues to be cited in AI search may vary depending on each company's settings.

If you operate a website, this concerns whether your website content can be used for AI training and whether it may still appear in AI response citations.

Risks and unknowns

The material comes from a Chinese news brief forwarded on Telegram and does not include direct confirmation from the companies or IETF.
The rules remain a technical draft, and it is not yet possible to confirm whether they will ultimately be adopted or how they will be implemented.
The companies' planned launch and effective dates have not been independently confirmed in the material.
The rules cannot withdraw data already used for training, nor can they compel AI companies to comply.

OpenAI, Google, Microsoft, and Anthropic reportedly adopt unified AI training opt-out rules

Article

AI-assisted interpretation

Why it matters to readers

Risks and unknowns

Related Developments

Related concepts

Keep following

Security Center