Ahrefs, Profound, Semrush — none ship llms.txt

Published May 24, 2026

The three loudest voices in AI search visibility right now are Ahrefs, Profound, and Semrush. Between them they publish dozens of posts a week explaining how brands should optimize for AI search. Profound just raised a $96M Series C at a $1B valuation on the thesis. Semrush ships an "AI Visibility Toolkit" as a $99/month add-on. Ahrefs renamed half their product surface "Agent A" and put it on every page.

We checked whether any of them actually do the basic AI-hygiene work they sell.

They don't.

The audit

On Saturday May 23, 2026, we ran a side-by-side GEO posture check on all three platforms. Five mechanical checks per site — the same five-point checklist any Brand Radar customer would run on their own domain after onboarding:

`llms.txt` published at site root — Ahrefs: ❌ 404 · Profound: ❌ 404 · Semrush: ❌ 404
Explicit AI-bot allow in robots.txt (GPTBot, ClaudeBot, PerplexityBot, etc.) — Ahrefs: ❌ generic wildcard only · Profound: ❌ generic wildcard only · Semrush: ❌ generic wildcard only
Claude tracking in their AI-visibility product — Ahrefs: ❌ omitted · Profound: ✅ · Semrush: ❌ omitted
JSON-LD structured data on homepage — Ahrefs: ⚠️ thin (Organization only) · Profound: ❌ none detected · Semrush: ⚠️ inconclusive
Public price for AI-visibility module — Ahrefs: bundled · Profound: "Get a Demo" · Semrush: $99/mo add-on

All three score the same on the most basic GEO hygiene question — does your own site demonstrate the protocols you tell customers to adopt. The answer is no.

Why this matters

llms.txt is the standard that lets a website tell AI crawlers what content to use and which sections to skip. It's not a magic SEO trick — it's a clear policy document that OpenAI, Anthropic, and Perplexity all respect. Google has declined the standard, but the other three major AI surfaces use it actively.

For a company selling AI search visibility tools, shipping llms.txt is the bare minimum credibility test. It's the equivalent of an SEO agency that doesn't have a sitemap. The standard takes 30 minutes to author and ship. None of the three have done it.

The robots.txt picture is similar. Each of the three platforms relies on the generic User-agent: * wildcard rule. None list explicit per-bot allow directives for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, or the half-dozen other modern AI crawlers worth signaling welcome to. Wildcard rules cover the technical case, but explicit allows are a measurable trust signal — particularly for Brave's crawler, which is the underlying index Claude grounds in. Brave's crawler heuristic gives preference to sites that name it explicitly.

Citare ships both. We made a deliberate choice to do the GEO hygiene we sell. Our llms.txt is live. Our robots.txt explicitly welcomes 16 modern AI and search bots. We treat this not as marketing but as the floor of credibility.

Claude is missing from two of three

The second finding from the audit was about platform coverage. Of the five major AI search platforms — ChatGPT, Google AI Overview, Gemini, Claude, Perplexity — Claude is the one most often omitted from competitor product surfaces.

Ahrefs Brand Radar tracks ChatGPT, AIO, AI Mode, Perplexity, Copilot, Gemini, and Grok. No Claude.
Semrush AI Visibility Toolkit tracks ChatGPT, AIO, AI Mode, Gemini, and Perplexity. No Claude.
Profound tracks Claude alongside ChatGPT, AIO, Gemini, Perplexity, plus five more long-tail surfaces.

Claude grounds its web search in the Brave index, which is the smallest of the four major AI search indices. It's also the index where surface rates run 30-40 percentage points lower than ChatGPT for the same brand in the same week. From our recently published Notion audit: Notion's surface rate was 67% on Claude vs 100% on ChatGPT — a 33-point gap that no single-platform monitoring would catch.

Brands that follow Ahrefs or Semrush's AI Visibility coverage are getting an explicitly four-out-of-five-platforms read. That's a meaningful gap on the platform with the lowest surface rates and therefore the largest content-investment leverage. The right read of AI search visibility requires all five platforms, every week. Anything less is measuring less than the full picture.

Citare's Brand Radar covers all five from the base tier. The methodology guide walks through the five-stage pipeline and why parallel five-platform dispatch is non-negotiable.

What this means for buyers

If you're shopping AI visibility tools right now, three concrete questions to ask:

Show me your llms.txt. If it 404s, the tool isn't ready to sell you the practice they don't follow.
Do you track Claude? If no, you're paying for a four-platform read in a five-platform reality.
What's the actual monthly cost for AI visibility? Profound is sales-gated (assume enterprise pricing). Semrush is $99/month on top of a $140 base. Ahrefs bundles it but charges $129/month at the entry tier. Citare ships the same surface in our $35 Pulse tier with a free-forever option for the first project.

A more honest standard for the category

The AI visibility category is young. Standards are still being set. The companies that lead the category in 2027-28 will be the ones whose own GEO hygiene is the strongest, because every buyer audit will check it.

We think the floor should be:

Publish llms.txt.
Explicit AI-bot allows for at minimum GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, Bytespider, Amazonbot, and BraveBot.
JSON-LD schema on every key marketing page (Organization, SoftwareApplication, FAQPage where applicable).
Track all five major AI search platforms in any product that calls itself AI visibility.
Public pricing in at least USD, ideally with regional pricing (INR for India is a 4× wedge currently uncontested).

None of the three leading platforms meet this floor today. Citare does. The audit is reproducible — five mechanical checks, takes 15 minutes. Run it on any AI visibility tool you're evaluating.

Frequently asked

Did you give the three platforms a chance to respond before publishing? No. The findings are mechanical and verifiable — anyone can run the same five checks in 15 minutes. There's no interpretation gap that requires their side of the story. We've already cited the URLs and checks; readers can verify independently.

Will this audit get updated as the platforms ship fixes? Yes. We re-run this audit quarterly. If Ahrefs ships llms.txt, that's good for the industry and we update the post immediately to reflect it.

Is Citare biased in this audit? Yes — we are a direct competitor to all three. We make no claim of neutrality. We also publish the raw source data and the five-check methodology so the audit is independently reproducible. Bias plus reproducibility is a more honest stance than claimed neutrality.

What about smaller AI visibility tools like Otterly or Athena HQ? We focused this audit on the three largest by ARR and category attention. We'll cover Otterly, Athena HQ, and the broader category in a follow-up post.

Where can I run a Citare Brand Radar dispatch against my own brand? The free tier covers one project with a 50-cell weekly dispatch across all five platforms. Onboarding takes ~10 minutes. → Start free.

We audited Ahrefs, Profound, and Semrush — none ship llms.txt