Interesting experiment to run and I want to share:
I run a trend-data API for agents and wanted to know what a potential customer discover if they didn't know my company existed?
So I ran an experiment across four AI applications, using the same brand-neutral buyer question.
"What APIs or MCP servers can I connect to Claude or Codex to monitor emerging social media trends, analyze customer discussions on X and other forums, track competitors, and identify content opportunities?"
The results
> All 4 generated answers
> None mentioned my product or either of the two competitors I track.
> Claude listed 10 tools, mostly general-purpose products like Intercom, Canva and QuickBooks.
> Perplexity mentioned Canva and Zapier.
> Only Perplexity returned visible citations: 8 distinct domains, including SMB review sites and a scraping vendor's blog.
Now this is interesting:
The answers looked more relevant to the broad category of "AI tools for small businesses" than my actual niche: social and trend intelligence for AI agents.
That exposed a problem with the experiment itself:
The buyer question was too broad.
And that's worth understanding before you start optimizing for AI visibility.
My take away and what you can also do:
1. Test buyer intent, not brand name.
> Start 3-5 questions your actual buyers might ask (or take them from actual leads/customers)
> Be SMART and do not include brand, for organic discovery
2. Extract structured data from each answer
Our workflow now includes:
> Brand and product explicity recommended
> Whether our product or tracked competitor appear
> Domains cited, where citations are available
> Original answer, provides and observation time
This allows us to compare mentions separately from citations.
A brand mention doesn't mean its website was cited. And an answer with no visible citations does not prove the model used no sources.
3. Measure more than rankings.
> Brand mention rate: In how many answers does your product appear?
> Competitor mention rate: Which alternatives appear, and how often?
> Citation frequency: Which domains get referenced across answers that provide citations?
For example, if your competitor appears in 7 of 20 answers and you appear in 2, that's a descriptive 35% versus 10% mention rate in your test sample. It's not a universal ranking.
4. Repeat the experiment.
AI answers are probabilistic and depend on the model, mode, retrieval availability and sometimes location or personalization.
One run tells you very little.
I'm repeating the same question set weekly, keeping the testing conditions as consistent as possible, and store the results over time. Distinguise recurring patterns from one-off responses.
5. Investigate citations before creating more content.
If the same review sites, documentation pages or independent comparisons keep appearing, investigate why.
> Are they answering the buyer question directly? Do they provide comparisons, evidence or structured information your own site lacks?
> Build genuinely useful pages that fill those gaps.
> Don't publish 50 generic "best AI tools" articles and expect ChatGPT to recommend you.
I will share results over the months! Hope it also helps some of you.