🔍 Read the full analysis: What The Four-Company AI Opt-Out Covers, And What It Doesn’t on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
OpenAI, Google, Microsoft and Anthropic have said they will support a shared robots.txt signal asking that a website’s content not be used to train generative AI models. The signal is voluntary, does not block crawling or govern AI answers, and the companies plan to implement it on different schedules; Meta has voiced support without committing to honor it.
OpenAI, Google, Microsoft and Anthropic said on October 30 that they will honor a shared signal in a website’s robots.txt file expressing a preference against using its content to train generative AI models. The proposed common rule could reduce the need for publishers to configure separate training controls for each company, but it is a voluntary request, not a technical block, and the companies have set different implementation dates.
The signal is defined in an Internet-Draft titled Machine-Readable Training Preference Signal, published by Anthropic standards lead Kevin Kelley. A publisher would add a rule for User-Agent: __usage_ai-training followed by Disallow: /. The draft is an individual submission, not an adopted internet standard. The IETF working group discussing AI preferences has not declared consensus on it.
The companies’ commitments vary. Anthropic says it has started implementing support, though it expects work across its services to take time. OpenAI says it aims to implement the signal “in the coming months.” Google says it aims to support the control in 2027; its crawler documentation specifies early 2027. Microsoft says it will honor the training preference from February 1, 2027, as it crawls sites.
Google says that if a publisher uses the shared signal but has no separate Google-Extended rule, it plans to treat the shared signal as a disallow for Google-Extended. Existing company-specific controls, including GPTBot, Google-Extended and ClaudeBot rules, are set to continue applying. Microsoft is also introducing a separate page-level preference for AI Grounding, its use of web content to inform answers in Bing, Copilot and Microsoft 365; that is not the same as opting out of model training.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
A Simpler Signal, Not a Block
For publishers, the proposal could make expressing a training preference less repetitive: one shared rule is intended to communicate the same request to several participating companies. That matters to sites that want to state how their material may be used without maintaining a separate training instruction for every crawler.
But a robots.txt entry is a signal to cooperating crawlers, not an access-control system. It does not technically prevent a company from retrieving publicly available pages, and the announcements do not establish a new enforcement mechanism. The practical effect depends on whether each company implements and follows the preference. The staggered rollout also means that, for a period, publishers may need to retain existing company-specific instructions.
The distinction between training and AI answers is material. The shared rule is for training generative models; it does not, by itself, opt a site out of content being used to ground an answer from current web pages. Microsoft announced a separate control for that use. Publishers may need to make separate choices about training, AI answers and search visibility rather than treating them as one setting.
Products Worth Considering
Dell 34-Inch S3425DW Curved WQHD USB-C Monitor
Dell 24 Plus Monitor S2425HSM – 23.8 FHD (1920×1080) 144Hz 1ms, 2 x 3W Speakers, HDMI, Height/Tilt/Pivot/Swivel Adjustability, AMD FreeSync
Samsung Essential S3 27-Inch Curved FHD Monitor
As an affiliate, we earn on qualifying purchases.
Why Publishers Need Separate Rules
Before this proposal, site operators commonly used different crawler names and directives for different services. OpenAI has controls associated with GPTBot, Google with Google-Extended and Anthropic with ClaudeBot. Microsoft’s existing page tags, NoArchive and NoCache, have also been used in its controls. The shared signal is intended to add a common way to express a training preference; it does not make those older instructions immediately redundant.
The draft was published through the IETF’s AI Preferences working group, which discusses ways to express machine-readable preferences. Its publication does not mean the IETF has approved it as a standard. At a working-group meeting on October 21, a chair said, “We haven’t declared consensus on anything yet.” That status matters because the proposal’s wording and adoption may still change.
Companies have also described different treatment of adjacent services. Google says the training signal will not affect Google Search, and Microsoft says its preferences do not change a page’s eligibility for Bing search results. Microsoft’s separate AI Grounding preference is evidence that a training opt-out alone does not settle every question about how web content may be used.
““has already started implementing support for this draft””
— Anthropic
Adoption and Enforcement Remain Open
The draft has no formal standing in the IETF standards process, and its text could change as the working group discusses it. The companies’ statements describe intentions and implementation plans, not an independent technical guarantee that every service will act on the signal in every case.
Meta has voiced support but has not said whether it will honor the rule. The source material also reports no commitment from Apple, Amazon, xAI or Perplexity. It remains unclear which other AI services will adopt the signal, how compliance will be monitored, and what recourse a publisher would have if a company did not follow its preference.
The shared signal does not remove material already included in trained models, prevent crawling, set payment terms or govern use of live web content in AI answers. Microsoft’s separate Grounding control addresses one distinct use, but the announced plans do not establish a universal opt-out for AI-generated answers across providers.
Company Rollouts Through Early 2027
Publishers can add the proposed rule if they want to express a preference against training use, but should check each provider’s documentation and keep existing company-specific crawler controls until the relevant company’s support is in place. The rollout timetable given so far runs from Anthropic’s early implementation and OpenAI’s planned work in the coming months to Google’s target of early 2027 and Microsoft’s February 1, 2027 start.
Microsoft says NoArchive and NoCache will stop counting as a training or grounding preference after March 1, 2027. Site operators relying on those tags will need to review their settings before that date and decide separately whether they want to express preferences about training and AI Grounding. Further changes may follow as the IETF working group considers the draft and more companies state whether they will support it.
Key Questions
What does the shared AI opt-out signal do?
It is a robots.txt rule expressing a publisher’s preference that participating companies not use the site’s content to train generative AI models. It is not a technical barrier that blocks access to pages.
Which companies have said they will honor it?
OpenAI, Google, Microsoft and Anthropic have announced support, with different rollout schedules. Meta has voiced support for publishers expressing preferences but has not committed to honoring the signal.
Does the rule opt a website out of AI answers?
No. The shared signal covers training, not the use of live web content to inform AI answers. Microsoft has announced a separate AI Grounding preference for that type of use in its services.
Does it remove a website from models that were already trained?
No. The announced preference does not remove content from models already trained. The companies’ announcements concern how participating services will treat the signal going forward.
Should publishers remove their existing crawler rules?
Not yet based on the announcements. Existing rules are expected to keep applying, and company support will arrive at different times. Publishers should check provider documentation before changing their current settings.
Source: ThorstenMeyerAI.com
