llms.txt — What It Is and Whether It Actually Works
llms.txt is an emerging AI standard. Honest assessment of which platforms read it, what to put in yours, and whether it's worth publishing in 2026.
> 2026-05-15 update — Google has officially declined the llms.txt standard. > > On 2026-05-15, Google's AI Optimization Guide stated unambiguously: "You don't need to create new machine readable files, AI text files, markup, or Markdown." For Google AI Overview and AI Mode specifically, llms.txt provides zero ranking lift. > > This does not kill the standard. OpenAI's GPTBot, Anthropic's ClaudeBot, and Perplexity all still respect llms.txt-style site hints. The honest takeaway: skip llms.txt if Google AI is your only target; ship it anyway if you're optimizing across multiple AI search engines (the four-index reality). Cost is one static file; benefit applies to three of the four major engines. > > Full analysis of Google's announcement and what it changes for AI search measurement: Google says GEO is still SEO — and why that makes AIO + Gemini tracking more important, not less.
llms.txt is the most-debated emerging standard in AI search optimization in 2026.
Some practitioners treat it as essential foundational infrastructure. Others dismiss it as performative — a low-cost gesture that produces no measurable AI citation lift. The honest assessment sits in the middle: llms.txt is worth publishing, but the leverage is forward-compatibility, not immediate visibility. Different platforms respect it to different degrees, and treating it as a citation hack misreads the standard.
This guide walks through what llms.txt actually is, what to put in it, which AI platforms currently consume it, and whether it's worth your time to deploy in 2026. The conceptual context for the broader AI metadata stack is in How to Measure AI Search Visibility and AI Crawler Access Guide.
What llms.txt Is
llms.txt is a plain-text file at the root of your website (/llms.txt) that provides AI systems with a curated summary of your site, its key topics, and the pages you most want AI models to attend to.
The format is structured markdown with a defined section pattern:
- An H1 with the site or organization name
- A brief description (1-2 sentences) of what the site is
- Sectioned lists of relevant URLs grouped by purpose
The standard was proposed by Jeremy Howard (Answer.ai) in mid-2024 and gained meaningful adoption through 2025. It's inspired by — but conceptually distinct from — robots.txt and sitemap.xml.
The intent: give AI systems a "what should I know about this site" file analogous to how robots.txt gives crawlers a "what can I access" file. It's metadata for AI ingestion specifically, not a crawl-control mechanism.
What llms.txt Is NOT
Three common misconceptions that shape unrealistic expectations:
Not a replacement for robots.txt
robots.txt controls crawl access — what a crawler is allowed to fetch. llms.txt provides metadata — what the site is and what content matters. Different purposes. You need both.
Not a replacement for sitemap.xml
sitemap.xml is for search-engine crawlers and lists all crawlable URLs with their lastmod timestamps. llms.txt is curated rather than comprehensive — you list the priority pages you want AI to focus on, not every URL on the site.
Not yet universally consumed by major AI platforms
Adoption is real but uneven. Anthropic and Perplexity explicitly read llms.txt. Google and OpenAI are partial — there's evidence of some consumption but not consistent or guaranteed citation behavior tied to it. Treating llms.txt as a guaranteed citation channel misreads where the standard is in 2026.
Adoption Status — Which Platforms Actually Read It
As of mid-2026, the honest landscape:
Reads it
- Anthropic (Claude) — explicit support; Claude documentation references llms.txt as input to its grounding
- Perplexity — incorporates llms.txt content into source ranking
- Several smaller AI products — emerging adoption across newer LLM-powered tools
Partial / inconsistent
- Google (AI Overview + Gemini) — DECLINED. Google's 2026-05-15 AI Optimization Guide explicitly stated llms.txt and similar AI-specific files are not needed; standard SEO foundations (helpful content, technical SEO, structured data) drive AIO ranking instead
- OpenAI (ChatGPT) — limited evidence of consumption; OpenAI has not officially endorsed the standard
Not yet consuming
- Various legacy and smaller AI-search products
The trajectory is toward broader adoption. The current state is patchy. Publishing llms.txt today is a forward-compatibility bet, not an immediate-citation play.
Where llms.txt Actually Helps in 2026
Three concrete benefits available now:
1. Anthropic and Perplexity citations. For brands whose audiences over-index on Claude or Perplexity (B2B SaaS, technical brands, research-led companies), llms.txt produces direct citation lift on these platforms specifically.
2. Forward-compatibility. AI platforms moving toward respecting it. Brands publishing llms.txt today are positioned for citation lift as adoption broadens — at zero ongoing cost.
3. Internal alignment. The discipline of writing llms.txt forces you to articulate concisely what your site is and what content matters. The artifact is useful internally even if AI consumption is partial.
The Format — What to Put in Your llms.txt
A complete llms.txt has three to five sections. The convention emerged organically through 2025 and is now reasonably stable.
Sample llms.txt for a SaaS company
# Citare
AI search visibility platform measuring brand presence across ChatGPT,
Gemini, Perplexity, and Google AI Overview. We help B2B brands track
their surface rate, identify citation gaps, and benchmark against named
competitors.
## Docs
- [What is GEO?](https://citare.ai/guides/what-is-generative-engine-optimization)
- [GEO vs SEO](https://citare.ai/guides/geo-vs-seo)
- [The Four AI Search Platforms](https://citare.ai/guides/four-ai-search-platforms)
- [How to Measure AI Search Visibility](https://citare.ai/guides/measure-ai-search-visibility)
- [Google AI Overview Optimization](https://citare.ai/guides/google-ai-overview-optimization)
## Research
- [GEO in India: 300 AI Queries Reveal Brand Visibility](https://citare.ai/blog/geo-india-300-queries)
## Optional
- [Blog](https://citare.ai/blog)
- [About](https://citare.ai/about)What each section does
- H1 + description — the brand entity and what it is
- Docs / Guides — primary reference content you want AI to attend to
- Research / Examples / Use cases — distinctive content (data, case studies)
- Optional — content you want available but de-prioritized
The convention is concise. A good llms.txt is a list of pages plus brief context, not a marketing essay. AI systems extract the structure and the URLs; padding doesn't help.
Where to host it
https://yoursite.com/llms.txt. Plain text. UTF-8 encoded. Accessible without authentication.
The Honest Assessment — Should You Publish One?
Yes. Three reasons:
- Cost is near-zero. A single .txt file. 30 minutes of work. No engineering required for most sites.
- Upside is forward-compatibility. AI platform adoption is trending toward broader respect for the standard. Publishing now means you're already positioned when adoption broadens.
- Treat as part of foundational AI infrastructure. Alongside robots.txt, sitemap.xml, and structured data deployment. It's a should-have in your AI metadata stack.
But — three reasons not to over-prioritize it:
- Don't expect immediate citation lift. The leverage is long-term. Publishing llms.txt today and measuring citation rate next week will not show movement.
- Don't substitute for higher-leverage actions. If you have to choose between publishing llms.txt and adding FAQ schema to priority pages, choose FAQ schema. The latter has measurable near-term effect.
- Don't treat it as a citation hack. Putting your competitor's brand name in your llms.txt does not get you cited for their queries. The standard is metadata, not adversarial keyword stuffing.
Common llms.txt Mistakes
Four patterns we see repeatedly:
1. Treating it as a citation hack. Stuffing competitor names, generic high-volume keywords, or inflated descriptions. AI systems weight credibility; manipulative llms.txt content earns less attention, not more.
2. Putting marketing copy instead of factual structured info. "Citare is the world's most trusted AI search visibility platform" is marketing copy. "Citare measures brand presence across ChatGPT, Gemini, Perplexity, and Google AI Overview" is factual structured info. The latter is what llms.txt should contain.
3. Forgetting to update it. Like sitemap.xml and structured data, llms.txt goes stale. Update it when key pages change, when you publish significant new content, or quarterly minimum.
4. Trying to make it longer than it needs to be. Concise wins. A well-curated llms.txt with 10-20 priority URLs outperforms a 200-URL exhaustive list. AI systems prefer quality signals over quantity.
Frequently Asked Questions
Do AI platforms actually read llms.txt?
Some yes, some no, some not yet. Anthropic and Perplexity explicitly read it. Google officially declined the standard in its 2026-05-15 AI Optimization Guide ("you don't need to create new machine readable files, AI text files, markup, or Markdown"). OpenAI's GPTBot still respects llms.txt-style hints. Several smaller AI products also read it. The honest answer for 2026: partial adoption, with Google explicitly opting out and the other three majors continuing to honor it.
Should I add llms.txt even if my site is small?
Yes. The cost is near-zero. The upside is forward-compatibility. There's no scale threshold below which llms.txt doesn't make sense. Even a 10-page brochure site benefits from a curated metadata file as adoption grows.
Is there a tool to generate llms.txt automatically?
Several open-source generators exist (search GitHub for "llms.txt generator"). For sites with structured content (Sanity, WordPress, Webflow), you can generate llms.txt programmatically from your CMS. For small sites, manual creation in 30 minutes works fine.
Will llms.txt eventually replace robots.txt?
No. They serve different purposes. robots.txt controls crawl access (what bots can fetch). llms.txt provides metadata (what the site is and what matters). Both will coexist — like robots.txt and sitemap.xml have coexisted for two decades.
How does llms.txt relate to structured data (JSON-LD)?
Different layer. JSON-LD is structured data embedded in HTML for AI parsers to extract specific facts (Organization name, Article author, Product price). llms.txt is a site-level summary file telling AI what the site is and what content matters. JSON-LD is page-level facts; llms.txt is site-level metadata. Use both — they complement each other.
Should llms.txt be in robots.txt?
No specific convention requires it, but it's reasonable to add Sitemap: and any related metadata file references in robots.txt for crawler discovery. AI bots will look at robots.txt anyway during their crawl.
Where do I check the llms.txt specification?
The original proposal and current specification are at llmstxt.org. The standard is open and lightweight; the spec fits on one page.
Run Your AI Visibility Audit
Citare measures your AI surface rate across ChatGPT, Gemini, Perplexity, and Google AI Overview — and surfaces foundational gaps (robots.txt issues, missing structured data, llms.txt absent) alongside content-side recommendations.
Run your free AI visibility audit → [citare.ai/audit]