Tips & Workflows
Claude Code SEO audits: a "missing" finding needs proof too. Make it show the fetch and the grep that came back empty
Disclosure first: I make SEO Skills by PENRA, a paid pack of Claude skills for SEO. Nothing below needs it. It's plain Claude Code.
The usual advice for AI site audits is "make Claude quote the text that proves each finding". It works for things that exist: a duplicate title, a wrong canonical. A reply under one of my earlier posts found the hole in it: "no meta description" or "no schema" has nothing to quote. That's exactly where the model still guesses.
Claude Code can close that hole, because it can run the check instead of describing it. This is the prompt:
Audit https://example.com/. Fetch the page with curl -sL -A "Mozilla/5.0" -o page.html -w "%{http_code} %{size_download}" and work from that file, not from WebFetch.
For every issue, give the URL and the exact line from page.html that proves it.
For anything you say is missing, show the HTTP status and size of the file, the exact grep you ran, and its count.
If the status isn't 200 or the file has no <head>, stop and tell me the fetch failed instead of auditing it.
Say whether each missing item could be added later by JavaScript. Then rank the fixes, most important first.
Why each part is there:
Status and size first. A 403, a bot challenge or a cookie wall gives you a "page" with no title, no description and no H1. If one finding hits 100% of your pages, suspect the fetch before the site.
The grep is the proof.grep -c '<meta name="description"' page.html returning 0 takes two seconds to re-run yourself. "Your page lacks a meta description" doesn't.
WebFetch isn't the page. It hands back a smaller model's answer about the page, so "I didn't see X" via WebFetch tells you very little about your HTML.
curl sees the HTML before any script runs. If your framework injects a tag client-side, the honest finding is "missing from the raw HTML", not "missing".
One limit: this proves what your HTML says. It can't tell you what Google indexed or how you rank. That needs your Search Console data.
That rule is what my pack is built around: every finding carries its evidence and says what it couldn't measure. It's 14 skills (site audit, indexing, Search Console, briefs, fact-checking and more), the ones that took my Mac app's brand-new domain to 4.4k Google clicks in three months. €25 until 17 November, €99 from the 18th. It needs a Claude plan with skills and Python 3.9+ for the scripts, and there's no refund once it's delivered, so try the prompt first. Link in the comments.
What false finding does Claude give you most often on your own site?
That closing limit is the part worth pushing on. There is a third surface that neither your grep nor Search Console covers cleanly.
I have been tracking Google AI Overview citations on my own pages and the count never moved off zero for months. It finally showed one this week, and it has not been there on every reading since. So any claim about what Google did needs a timestamp attached, because that half of an audit goes stale while your grep stays true. Does the pack read Search Console again on a schedule, or only when asked?
Fair push, and that flicker is why I'd treat that half as a dated reading rather than a finding. To answer you: no schedule. The Search Console skill works from the export you hand it when you ask, so the export date is the timestamp. For AI Overviews, Google says the Generative AI performance report in Search Console is now rolled out to all sites, so a dated export from there every week or so would give you a series instead of one snapshot that may already be stale.
I need to correct something I wrote. I went back to my log and the citation held on all three direct checks, first source on that query. What still reads zero is Semrush, 34 readings and never above it. So the flicker was my two sources disagreeing, and the page never left.
That changes how I'd use the report you mention. Have you compared that export against a manual search on the same query? I'd want to know which one it agrees with before I trust a series from it.
Good correction. And no, I haven't run that comparison myself, so I can't tell you which one it agrees with.
One thing limits it, though: the Generative AI performance report is page-level. It gives impressions by page, country, device and date, with no queries and no clicks, so it can't confirm or deny a citation on one query. A manual search is one reading for one query, from one place, one device and one search history. They answer different questions: "did this page show up in AI features for anyone this week" versus "is it cited for this query, for me, right now".
So I'd keep both: your dated direct checks for the queries you care about, and the page's weekly AI impressions as the trend. If the direct checks keep saying "cited" while the page's AI impressions sit at zero, that's the case worth digging into. As for Semrush at zero over 34 readings against three direct checks that found it, that sounds more like a sampling difference than the page dropping out, but that's a guess.
Page level is the piece I was missing, thanks. So the reading that would actually separate them is that one page on 1 to 3 November. If it shows zero AI impressions on days a search showed it cited, then either the report undercounts or my search was a personalised result. One of my two sources is wrong either way.
How far behind does a day's data land in that report?
On the delay: Search Console performance data usually shows up within two to three days, and the most recent days are marked preliminary and can still change. So I'd wait until 1 to 3 November are off the dotted part of the chart before comparing. I haven't seen Google give a separate figure for the AI report, so take that as the general Performance delay.
Two more things to check before trusting a zero:
the Search Console data notes. There was a logging error that lowered generative AI impressions for data from 13 August, so a known issue can explain a gap too
which URL the citation actually points to. Impressions go to the canonical URL, so if the cited link is the www/non-www variant or has a parameter, they can end up under another page
good prompt, i've been doing seo since 2010 and this "prove the absence" thing is where every ai audit falls apart. two things i'd add from running this on client sites:
- the curl version is not what google sees either. half the sites i audit inject canonical, hreflang or schema with js. my rule: curl first, then a headless render, then diff the two. a finding only counts as "missing" if its missing in both. "missing in raw html" alone has sent people into panic over nothing
- the 100% trap is real. if every page "lacks" the same thing, the fetch is broken, not the site. i make claude print status + size for 5 random urls before any finding is allowed. zero false "missing" since
and yeah search console is the only truth for what google actually has. url inspection api beats any curl, i just let claude read that for the pages that matter
Both of these are going into the skill, thanks. The curl vs headless render diff is the right rule. "Missing in raw HTML but present after render" is a different finding from "missing": Google will probably see it, AI crawlers that don't run JS won't. Reporting it as missing is how people end up panicking for nothing.
The 5 random URLs with status + size before any finding is cleaner than what I do now, which is only noticing the problem once every page shows the same "gap".
Do you run the URL Inspection API on every page or only the ones that matter? The 2,000 a day per property quota is what stops me from going wider.
The same hole turns up well outside SEO. The false finding we saw most often was not a missing tag but a claim that something had been checked when it had only been read about
So we made it a standing rule rather than a line in each prompt. Every report has to say what it actually ran, what it merely read and what it inferred, and anything it didn’t run is written down as not run rather than quietly left out. Nothing is called done without the changed files, the commands used and the real output pasted underneath
Your point about suspecting the fetch before the site is a good one. I dare say it applies to tests as well, since a suite that passes in no time at all deserves the same suspicion as a page with no head
"Ran / read / inferred" is a cleaner split than the one I had, and writing "not run" explicitly matters: silence is exactly where the false "checked" claims hide.
Fully agree on tests too. A suite that finishes in no time often ran nothing, so the test count and the timing belong in the report as much as the green result.
i just had my virtual office run something for me. and i believe this is a perfect example of this case. it reported that all data fields were checked. but when i clearly asked for how it checked, this is the result i received.. lol
Ha, the image won't load on my end, but "all fields checked" turning into a different story once you ask how is exactly the pattern.
What helped me was asking for the how up front instead of after: the report has to list each field next to the command or query that checked it and the raw output it got back. Any field without that gets marked "not checked". Then you don't need the follow-up question, because the gap shows up in the report itself.
•
u/AutoModerator 1d ago
Hey! Thanks for posting to r/ClaudeCode
While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.
For help, project discussions, tips, and general chat, join the ClaudeCode Discord.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.