
Do you need llms.txt? For most business websites, it is not urgent. llms.txt is an optional Markdown file at the root of your site that lists your key pages for language models. It is a proposed convention, not a standard like robots.txt, and having one does not guarantee that ChatGPT, Gemini or any other assistant will mention you.
It is also cheap to add and does no harm. Below we explain what the file is and what is publicly documented about who reads it. Then we cover when it is worth your time, how to write one, and how to check whether any bot ever requests it.
What is llms.txt, and how is it different from robots.txt and sitemap.xml?
llms.txt is "a proposal to standardise on using an /llms.txt file to provide information to help agents use a website". Jeremy Howard of Answer.AI first published it on 3 September 2024. The current version is v2, and the proposal page was last modified on 10 August 2026.
The file is written in Markdown, a simple text format where # marks a heading and [name](url) marks a link. According to the spec, only one element is required: an H1 with the name of the site or project. After it you can add a blockquote with a short summary, some optional prose, and then H2 sections with lists of links. Each link can have a short note after a colon.
The proposal itself separates llms.txt from robots.txt. robots.txt tells automated tools what access is acceptable. llms.txt is meant to be read on demand, when an agent needs information while it helps a user. The author expected it to matter mainly for inference (answering questions) rather than for training.
The proposal also explains why a sitemap cannot do the same job. Sitemaps often lack versions of pages that a language model can read easily, they do not include useful external links, and they usually list far more pages than fit into a model's context window (the amount of text a model can work with at once).
| robots.txt | sitemap.xml | llms.txt | |
|---|---|---|---|
| Purpose | Says which crawlers may access which paths | Lists URLs you want search engines to discover | Gives a short, curated map of key pages for AI agents |
| Who reads it | Search and AI crawlers that follow the protocol | Search engines | No major AI assistant documents reading it. Coding agents and some tools use it |
| Controls access? | Yes, for crawlers that respect it | No | No |
| Format | Plain-text directives | XML | Markdown |
| Status | IETF standard (RFC 9309) | Established sitemaps protocol | Community proposal (v2, August 2026) |
What changed in llms.txt v2
Many articles still describe the first version. The v2 changes matter if you add the file now:
- Subpath files. The file can live at
/llms.txtor at any subpath, such as/docs/llms.txt. It covers the URLs under its path, and agents should use the most specific file. - Markdown page versions. The proposal recommends clean Markdown copies of pages, either with
.mdappended (page.html.md) or with the extension replaced (page.md). Both forms are allowed. - Discovery through link relations. A page can point to its Markdown version with
rel="alternate" type="text/markdown", and to the llms.txt that covers it withrel="describedby". You can use an HTML<link>element or an HTTPLink:header. - The "Optional" section is just a convention now. You can still use it for secondary links, but it no longer has a special technical meaning.
Which AI systems actually read llms.txt?
The short answer from primary sources is that no major AI assistant documents using other sites' llms.txt files to build its answers. Here is what each side has said publicly.
Google: the Search team says it does not use it
On 17 June 2025, Google's John Mueller wrote: "FWIW no AI system currently uses llms.txt." He added that consumer chatbots fetch pages for training and grounding, but not the llms.txt file, and that server logs make this obvious.
His 2026 statements say the same. On Reddit he called the question "purely a hypothetical", because no AI sites require or use the file. He also argued that llms.txt cannot help a model choose between sites, because the site that wants to be chosen writes it. When asked whether llms.txt files on Google's own developer properties are an endorsement, he answered "no".
Google's 2026 guidance on optimizing for AI features in Search goes further. According to summaries by Optimizely and Contentful, it says you do not need llms.txt or other AI-specific text or Markdown files to appear in AI Overviews or AI Mode. Keeping one neither helps nor hurts your rankings.
OpenAI, Anthropic and Perplexity: documentation covers robots.txt, not llms.txt
OpenAI's crawler documentation describes OAI-SearchBot, which surfaces websites in ChatGPT search, and GPTBot, which collects training data. You control each one separately in robots.txt. The documentation does not describe any role for llms.txt. We have not found documentation from Anthropic or Perplexity saying that their assistants read other sites' llms.txt files either.
What confuses people is that these companies publish llms.txt files themselves. llmstxt.org lists developers.openai.com/llms.txt, docs.anthropic.com/llms.txt and ai.google.dev/gemini-api/docs/llms.txt. These files help developers and coding tools navigate the vendors' own documentation. Publishing a file is not the same as reading yours.
What studies and server logs show
- SE Ranking analyzed about 300,000 domains and found llms.txt on 10.13% of them. Neither a statistical correlation nor a machine-learning model showed any effect on how often AI cited those sites. Removing the llms.txt variable actually made the model more accurate.
- Log studies compiled by Optimizely found very few requests for the file. Limy recorded 408 llms.txt requests across more than 500 million AI bot visits over 90 days. Otterly recorded 84 out of more than 62,100 AI bot visits over 90 days, about 0.1%.
- Adoption estimates depend on the sample. SE Ranking Visible reports 13.3% of 37,894 AI-cited domains (Trakkr, March 2026) and a valid file on 5.61% of the top 10,000 sites (HTTP Archive, June 2026).
Where llms.txt does get used
According to the proposal, the files are used most heavily in software documentation, where coding agents follow them to API references and tutorials. The same page notes that Chrome's Lighthouse checks for an llms.txt as part of its agentic browsing audits. That is a tooling check, not evidence that AI answers use the file.
Do I need llms.txt? When it's worth adding and what to fix first
Treat llms.txt as low-cost housekeeping, not as a visibility lever. Whether it is worth 10 minutes of your time depends on your site.
When llms.txt is worth adding
- Docs, SaaS and API products. This is where the file is actually used, by coding agents and documentation tools. Developer-facing products benefit most.
- Sites that rely heavily on JavaScript. If your prices or service descriptions only appear after scripts run, a Markdown map that links to clean versions of the pages can be a useful fallback for agents that read raw text. It does not replace fixing the pages themselves.
- Service businesses with facts that are hard to extract. Clinics, law firms and local services often scatter prices, hours and locations across many pages. Collecting them in one file also forces you to check that the facts are correct.
- Your platform already creates one. According to llmstxt.org, Yoast SEO, AIOSEO, Wix, Mintlify and GitBook can generate the file automatically. Wix creates one for every site. If you have one already, keep it, but open it and check that it lists the right pages.
When llms.txt is low priority: fix these first
- robots.txt blocks AI crawlers. If GPTBot, OAI-SearchBot, ClaudeBot or PerplexityBot are disallowed, sometimes through a CDN setting or rules copied from another site, an llms.txt changes nothing. Our map of robots.txt directives and other AI-crawler signals shows which rules matter.
- Bots get errors or challenges. 4xx and 5xx responses, CAPTCHA pages and firewall challenges served to bots stop them before they read anything.
- Key facts are missing from your pages. If your prices, hours, service area or contacts are not written in the HTML of your pages, put them there first. As Mueller noted, discovery still runs through normal pages and internal links.
How to create an llms.txt file, step by step
- Pick 5–20 pages that answer real customer questions: services, prices, hours, locations, contacts, policies and, for software, documentation.
- Write an H1 with your business or product name. It is the only required element.
- Add a one- or two-sentence blockquote summary: what you do, for whom and where.
- Optionally, add a short paragraph with key facts. Write plain statements, not marketing copy.
- Group the links under H2 sections such as "Services and prices", "Contacts and hours" and "Policies". After each link, add a short note after a colon.
- Point the links to clean pages:
.mdversions if you have them, or lightweight HTML pages that return status 200. - Save the file as UTF-8 plain text and upload it to
/llms.txt. For documentation that lives in a subfolder, use a subpath such as/docs/llms.txt. - Open the URL and check the response: status 200, a text content type (
text/plainortext/markdown), and no redirect to the home page or bot challenge. - Optionally, add discovery hints from v2:
<link rel="describedby" href="/llms.txt?lang=en">in your page head, or the headerLink: </llms.txt>; rel="describedby". On pages with Markdown copies, addrel="alternate" type="text/markdown". - Test the file. Give it to ChatGPT or Claude and ask "What is this business? What does it charge? When is it open?" The proposal suggests a similar test.
An llms.txt template for a service business
For example, imagine a local plumbing company called Brightline Plumbing. The business, prices and URLs below are fictional and only show the structure:
# Brightline Plumbing
> Licensed plumbing repairs and installations in Springfield and nearby towns. Open Monday to Saturday, 7 a.m. to 7 p.m. Emergency service 24/7.
Brightline Plumbing serves homes and small businesses. Prices below are starting prices; the final quote follows an on-site visit.
## Services and prices
- [Emergency repairs](https://www.example.com/services/emergency.md): from $120 service call fee, 24/7
- [Water heater installation](https://www.example.com/services/water-heaters.md): from $950, removal of the old unit included
- [Drain cleaning](https://www.example.com/services/drains.md): from $150
## Contacts and hours
- [Contact and booking](https://www.example.com/contact.md): phone, email, online booking
- [Service area](https://www.example.com/service-area.md): towns and ZIP codes covered
## Policies
- [Warranty](https://www.example.com/warranty.md): 12 months on parts and labor
- [Payment and cancellation](https://www.example.com/payment.md): accepted payment methods, cancellation terms
## Optional
- [Maintenance tips](https://www.example.com/blog.md): seasonal advice for homeowners
Do you need llms-full.txt too?
llms-full.txt is a separate convention: your full documentation joined into one Markdown file. It is not part of the v2 proposal text, although documentation sites often use it. Anthropic, for example, serves both docs.anthropic.com/llms.txt and /llms-full.txt.
We recommend both files for documentation sites. For most marketing and service sites, the short map is enough.
How to check whether AI bots actually request your llms.txt
Adding the file is the easy part. Checking whether anything reads it tells you whether to spend more time on it.
Option 1: server or CDN logs. Search your access logs for requests to the file, then filter by AI user agents:
grep "llms.txt" access.log | grep -Ei "gptbot|oai-searchbot|chatgpt-user|claudebot|claude-user|perplexitybot|perplexity-user"
Then verify the IP addresses. Anyone can fake a user agent. OpenAI publishes its crawler IP ranges at openai.com/searchbot.json and openai.com/gptbot.json, as described in its crawler documentation. While you are in the logs, also check which of your regular pages these bots request and what status codes they receive.
Option 2: Pulse. Chetver Pulse shows which bots from ChatGPT, Claude, Gemini and others visit your site, what they read and where they run into problems. Each problem comes with advice on what to change. On the Pro plan, Pulse shows which AI bots read which pages and where they run into errors. You can see whether /llms.txt appears among the URLs they request and whether your key pages return errors.

If nobody requests the file, nothing is broken. That matches the log studies above. Leave the file in place, keep it up to date, and put your effort into the pages bots do read: make sure they return 200, load without a challenge, and state your facts in plain text.
Where Chetver's AI Page fits
The AI Page is one page with your business facts for ChatGPT, Claude and other AI assistants: prices, opening hours, contacts and services. Chetver builds it from your website, and you publish it on aipage.yoursite.com. It comes with an llms.txt file, so you do not have to write one by hand.
The same caveat applies to the AI Page as to llms.txt: no file or page guarantees mentions in AI answers. Their value is that your facts exist in one clean, correct, readable place. For a wider view of llms.txt, robots.txt directives and the other signals you can send to AI crawlers, see our research map of AI-crawler signals in 2026.
A 10-minute llms.txt checklist
- Open
yoursite.com/llms.txt. Your platform may have created one already. - Check robots.txt first: GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot should not be blocked by accident.
- Make sure your key pages state prices, hours, locations and contacts in plain text.
- Write or fix the H1 and the one-line blockquote summary.
- List 5–20 key pages under H2 sections, each with a short factual note.
- Click every link and confirm it returns 200.
- Confirm the file is served as plain text with status 200, with no redirect or bot challenge.
- Optionally, add
rel="describedby"pointing to/llms.txt. - Paste the file into ChatGPT or Claude and ask what your business does and what it charges.
- Put a monthly reminder in your calendar to update the file when prices, hours or services change, and to check your logs for requests.
Your next step: before you edit llms.txt, spend five minutes on the second item in this list. A crawler that cannot reach your site will not read any file you publish. Once access and facts are in order, llms.txt is a quick, harmless finishing touch. To see how AI assistants describe your business today, and what to do so they name you, run the free AI check.
Frequently asked questions
Does ChatGPT read llms.txt?
There is no public documentation saying it does. OpenAI's crawler documentation covers OAI-SearchBot and GPTBot and how to control them in robots.txt, with no role for llms.txt. OpenAI publishes an llms.txt for its own developer docs, but that does not mean ChatGPT reads yours.
Does Google use llms.txt for AI Overviews or AI Mode?
No. Google's John Mueller has said repeatedly that Google does not use llms.txt. According to summaries of Google's 2026 guidance on AI features in Search, you do not need llms.txt to appear in AI Overviews or AI Mode, and having one neither helps nor hurts rankings.
Is llms.txt an official standard like robots.txt?
No. llms.txt is a community proposal by Jeremy Howard of Answer.AI, first published in September 2024, with version 2 released in August 2026. robots.txt is an IETF standard (RFC 9309).
Can llms.txt block AI crawlers or AI training?
No. llms.txt only lists pages and does not control access. To allow or block crawlers such as GPTBot, OAI-SearchBot, ClaudeBot or PerplexityBot, use robots.txt.
Can llms.txt hurt my SEO?
According to summaries of Google's 2026 guidance, keeping an llms.txt neither helps nor hurts rankings. The practical risk is outdated information, such as old prices or broken links, so update the file whenever your facts change.
What is the difference between llms.txt and llms-full.txt?
llms.txt is a short Markdown map of your key pages. llms-full.txt is a separate convention that joins your full documentation into one file, and it is not part of the llms.txt v2 proposal text. Documentation sites often publish both; most business sites only need the short map.