llms.txt
A proposed standard file for guiding AI models to your most important content — promising in theory, largely unread in production today.
llms.txt is a proposed standard: a Markdown file placed at the root of your site (e.g. /llms.txt) that lists and links to the content you most want AI models to read and understand, in a clean, easily-parsed format. The idea, introduced in 2024, is to do for LLMs what a sitemap does for crawlers — point them at the canonical, high-value pages. It is distinct from robots.txt (which grants or denies crawler access): llms.txt tries to curate and prioritise content for models. The honest caveat is that, as of 2026, adoption by the major AI providers is unconfirmed and largely unverified in production — there is little public evidence that ChatGPT, Gemini or Grok read it or that it changes whether you get cited. Treat it as a low-cost, optional experiment, not a lever that drives AI visibility.
What is llms.txt for?
The proposal aims to hand models a concise, structured map of your site: a short Markdown document with links to your most important pages and brief descriptions, so an LLM doesn’t have to infer your priorities from a sprawling site. In principle that could improve how accurately a model represents you.
How is llms.txt different from robots.txt?
- robots.txt is about access — which crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) may fetch which paths. Keeping these allowed is what actually affects whether models can read you.
- llms.txt is about curation — suggesting which content matters most. It does not grant access; it tries to guide attention.
Does llms.txt actually work?
Our measured position: it is over-hyped. The standard is sensible and cheap to implement, but there is no reliable public confirmation that the major models consume it or that it moves citations. If you want to influence whether AI cites you, the evidence points elsewhere — extractable content, a coherent entity footprint, and third-party corroboration — which we cover in how do LLMs choose which sources to cite? Add llms.txt if you like; just don’t expect it to be the thing that changes your AI share of voice.
What should you do instead?
Start from evidence, not hope. Run the free Domain Check to see which queries the models already cite you on and where the gaps are — your ghost routes. Fixing those with extractable, well-corroborated content will move the needle far more than any single file at your site root.
Frequently asked questions
Should I add an llms.txt file?
It is low-cost to ship, so there is little downside. But do not expect it to change your citations on its own — as of 2026 the major AI crawlers do not honour it in any documented way.
Is llms.txt the same as robots.txt?
No. robots.txt grants or denies crawler access; llms.txt tries to curate and prioritise content for models. Unlike robots.txt, adoption of llms.txt by major providers is limited and its real-world effect is unproven.