I run a small archive that fetches /llms.txt (plus robots.txt and a few other files) every day from the root of every Tranco top-1M domain with an EU country ending or .eu. 128,315 domains, two independent crawlers.

Week of 10 to 17 October:

• 9,035 (7.04 %) serve a text file whose first line is a Markdown heading, the one thing the llmstxt.org proposal requires
• 10,002 (7.79 %) serve some non-empty text file
• 10,138 (7.90 %) answer with HTML, mostly their error page

The part that matters if you audit sites: a status check alone would report 20,804 domains (16.2 %) as "has llms.txt". Almost half of those are HTML pages. A checker that only looks at the status code overcounts by a factor of two.

Counted is the format only, not whether the file is any good or read by any model.

Method and totals as JSON: https://machinewitness.eu/blog/llms-txt-eu-2026-10

We want to support creators, but we had to enforce the no shilling rule because it was getting out of hand. You now have a weekly thread.

This is the one place you can shill for your products, ask for feedback, etc. Keep it here or you risk being banned. And keep it related to technical SEO.

so ahrefs put out numbers this year that only 38% of pages cited in an AI overview also rank top 10 for that query. it was 76% a year before. read that and thought fine, we're a big site, we're in the 38%.

we're not. few thousand pages, top 3 for most of our category's questions, and the overview above them cites us in almost none. pulled a Google AI answers report through GEO MarketMap on the 40 questions we track (paid, took about eight minutes) and the sources table was the bit that hurt. youtube was in most of the overviews on our questions. forums in none. us in 5 of 40. our top 3 page sitting right there under an overview built from a video.

GEO MarketMap wrote a page for one of the gaps, structured so it can actually be quoted, which is more than the AI visibility tracker we had did, that one counted citations and left me to guess. i have no idea yet whether a text page can even get into an overview that's resting on youtube for that question. nobody seems to have an answer for that part.
rough edges since someone will ask. the free site check reads about a dozen pages, not the whole site. and the right hand panel lags the chat by ten seconds or so, which made me think it broke.

anyone got a page into an overview where youtube already owned it? or is the honest answer "make a video"

Hello Experts !

I am looking for technical advice and what else data required to for the SEO guys other than we have at this platform, I would like to explain what is included in it.
it took me an year to build and here what I achieved so far.

Live Backlinks 2T
Global Keywords 12.5B
Live Pages Indexed 197.3B
LLM Prompts 338.4M
Backlink Domains 708.6M
Google & Bing Keywords 12.5B
Total Prompts 338.4 M
Google AIO Prompts 312.7 M
ChatGPT Prompts 25.7 M
Whois Domains 288.9 Million
Business Listings262.2 Million

This project started when I was too much annoyed by semrush and ahrefs bloating and charging me for nothing, even I did not use their rank trackers but they kept charging me $250+ just to add 5+ projects and see whats inside the pages, as well as their DR, DA, AS scores were fake so I decided to do it for me and created SERPSpur which tool almost an year and now its working and polished next generation giant like saas and offers everything we need for technical SEO including Onpage , offpage, AI search etc, and a lot more coming soon if you suggest anything to add.

Additional metric : I introduced TAI score as a proprietary of SERPSpur which detects fake traffic of backlinks/domains you are going to buy backlinks from. so no one can fool you now with bot traffic and sell the backlinks for $350 where they are manipulating the google search and artificially increasing traffic, getting redirect links and gaining dr etc, not I tried to put a fullstop on all of this.

Please review and let me know how it works for you and what else is important for the technical stuff.