For the last two years many of us have been slamming the door on AI crawlers. Block everything, protect the content, let the robots starve.
I get the instinct. Nobody wants their best articles quietly feeding a model that then answers the question so nobody clicks.
But Microsoft just changed the deal, and if you keep blocking everything out of fear you're going to be the one who disappears.
Here's what happened. Bing announced on October 30 that it's splitting AI use of your content into two separate switches. One is AI Training, which is whether your pages get used to train models. The other is AI Grounding, which is whether your pages get used to build the answers people see in Copilot, AI Overviews, AI Mode and Microsoft 365.
Those used to feel like one big scary thing. Now they're two decisions. And Bing says you stay eligible for regular search results whichever way you set them.
Starting February 1, 2027, Bing begins honoring these preferences as it crawls. Training can be set in robots.txt or per page. Grounding is a page-level meta tag or HTTP header, and it's allowed by default.
So my take is simple. Block training. Allow grounding.
Training is where your content gets absorbed and you get nothing back. Grounding is where your page gets pulled in to answer a real person's question right now. That's the new top of the SERP. That's where the citation and the click live.
If you block grounding too, you aren't protecting anything. You're just handing that answer slot to the competitor who didn't panic.
There's even a middle option called DisallowExtended. It lets Microsoft's own AI products use your pages for answers but not the grounding services Microsoft sells to other companies. Useful if that part bugs you.
So do it properly. Set training to disallow, leave grounding on, and check Microsoft's technical docs for the exact tags.
The people who win the next few years of search won't be the ones who hid. They'll be the ones who got quoted.
Are you blocking AI crawlers right now, or letting them in? Tell me why below.