K2 Ascent
Field notes · News

New AI Training Opt-Out: What It Means for Your Website

9 Nov 20265 min readBy the K2Ascent team

A single new line in your website’s robots.txt file will soon let you tell Google, Microsoft, OpenAI and Anthropic whether your content can be used to train their AI models. It is a preference about AI training only: it does not remove you from Google or Bing search results. Most businesses don’t need to rush, but you should know what your site currently says before the major providers start honouring it in 2027.

What was announced?

On 30 October 2026, Anthropic’s Kevin Kelley published a proposal at the IETF (the Internet Engineering Task Force, the group that develops many of the internet’s technical standards) called a “Machine-Readable Training Preference Signal”. Google, Microsoft, OpenAI and Anthropic announced support for it the same day.

The idea is simple. Until now, if you didn’t want your content used to train AI, you had to write a separate rule for each company’s crawler (the automated program that visits and reads web pages). The new signal gives you one shared rule that every participating provider agrees to read.

It isn’t an official standard yet. The draft is an individual submission with no formal IETF standing, and the companies have committed to supporting it voluntarily.

What is robots.txt, and what does the new rule look like?

Robots.txt is a small, plain-text file that sits at the root of your website (for example, yourbusiness.com.au/robots.txt). It tells automated visitors which parts of your site they may access. Most website platforms create one for you.

The new rule uses a special name, __usage_ai-training, in place of a crawler’s name. The draft’s own example looks like this:

User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/

In plain English: “Please don’t use my site for AI training, except for my press kit.” A Disallow line expresses a preference against training on those pages; an Allow line expresses a preference in favour.

Does opting out of AI training hurt my Google ranking?

No, according to Google. Its crawler documentation says Google-Extended, its existing AI training control, does not affect whether a site is included in Google Search and is not used as a ranking signal. Google plans to support the new shared signal in early 2027, and if you use it without mentioning Google-Extended separately, Google will treat it as if you had blocked Google-Extended.

Microsoft says its new preferences won’t change a page’s eligibility for normal Bing search results.

It’s also important to know what the rule doesn’t do. The draft says it is a usage preference, not an instruction about crawling, so it doesn’t stop bots visiting your pages. If you have no __usage_ai-training section, you haven’t expressed any preference at all; existing rules aimed at “all crawlers” don’t count.

What about appearing in AI answers?

This is where business owners need to be careful, because AI tools use your content in two different ways.

Training means your content helps teach the AI model in general, long before anyone asks a question. Grounding means the AI looks up live web pages at the moment someone asks a question, then summarises them and often links to the source. Grounding is how sites get cited in AI answers.

The new shared rule covers training only. Microsoft has added separate, page-level grounding settings: Allow (the default), DisallowExtended (only Microsoft’s own products, such as Copilot, may use the page) and Disallow. Microsoft says blocking grounding is expected to reduce your brand’s visibility in AI experiences.

For most businesses, being cited in AI answers is valuable marketing. Opting out of training is a reasonable choice for some; blocking grounding usually isn’t, unless you have a specific reason.

When does this take effect?

The providers are moving on different timelines:

Microsoft is also retiring an older approach. If your site uses NoArchive or NoCache tags (instructions that previously told Bing not to store or reuse your pages), Microsoft will stop treating them as AI preferences after 1 March 2027.

What this means for your business

You don’t need to change anything this week, but a 10-minute check now will save confusion later.

  1. Look at your current robots.txt. Type your domain followed by /robots.txt into a browser. If you see lines naming GPTBot, Google-Extended or similar, someone has already set AI preferences; note what they say.
  2. Check it in Google Search Console. In Search Console, open Settings, then the robots.txt report. It shows the file Google last fetched and flags any errors. A broken file can cause problems well beyond AI.
  3. Decide your position on training. Ask yourself: is my content the product (original research, paid guides, photography), or is it marketing that I want seen as widely as possible? Content creators may prefer to opt out; most service businesses gain little by doing so.
  4. Don’t block grounding by accident. If AI visibility matters to you, keep pages available for grounding and search. Avoid copying blanket “block all AI” templates from the internet.
  5. Ask about NoArchive or NoCache. If your developer added these tags, put a reminder in your calendar before 1 March 2027 to review your Bing settings.

Frequently asked questions

What is the new AI training opt-out?
It’s a shared robots.txt rule, using the name __usage_ai-training, that tells participating AI providers whether you want your website content used to train their AI models.
Will opting out remove my website from Google?
No. Google says its AI training control does not affect inclusion in Google Search or rankings, and Microsoft says the preferences don’t change eligibility for Bing search results.
Will opting out stop my business appearing in AI answers?
Not by itself. The shared rule covers training. Using live pages to answer questions, known as grounding, is controlled separately, and Microsoft warns that blocking grounding is likely to reduce AI visibility.
When do the big providers start honouring it?
Microsoft from 1 February 2027 and Google from early 2027. OpenAI is reported to be implementing it in the coming months, and Anthropic says it will take some time.
Is this an official internet standard?
Not yet. It’s an individual IETF draft with no formal standing, which the four companies have chosen to support voluntarily.

Not sure what your robots.txt says?

The K2Ascent team can review your robots.txt and help you balance AI training choices with AI visibility. Get in touch for a quick SEO and AEO check.

Get in touch

Sources

← All articles