What is llms.txt, and should your site have one?

Ask for welcomeai.dev/llms.txt and you get 24 KB of Markdown: a paragraph saying what we do, then 109 links, each with a sentence explaining what is on the other end. No navigation, no cookie banner, no markup to unpick.

That is the whole idea. A proposed convention, not a standard anybody enforces: one Markdown file at the root of your domain that tells an assistant what your site contains and where to find it, in the format models read most reliably.

The interesting question is not what it is. It is whether yours needs one, and the honest answer depends on what kind of site you run.

The shape of the file

It is Markdown with a fixed skeleton: an H1 with the site's name, a blockquote summarising it, optional prose, then H2 sections holding lists of links where each link carries a description.

# Acme

> One paragraph on what Acme is, for somebody who has never heard of it.

## Docs

- [Getting started](https://acme.example/docs/start): What to do first, and what it needs
- [API reference](https://acme.example/docs/api): Every endpoint, with the fields it returns

## Optional

- [Changelog](https://acme.example/changelog): Every release, newest first

That is it. Any text editor makes one, and no library is involved.

Our own file, part by part

Ours is generated on every deploy, so it cannot drift from the site. Step through the parts to see what each one is doing.

There is a second file too, and the pair is worth understanding as a pair.

llms-full.txt is every documentation guide inlined into one file, about 39,000 words of it. The short file is a map. The long file is the territory, for an assistant that would rather read once than fetch forty pages.

It is not robots.txt, and it is not a sitemap

Four files at your root, four different jobs. Mixing them up is the most common mistake in this whole subject.

Two consequences. An llms.txt grants nothing and blocks nothing, so it is not how you control AI crawlers. And it does not replace a sitemap: a sitemap is exhaustive by design, while this file is a short curated list, and its value comes from what you leave out.

Does anything actually read it?

Here is the part most articles skip. No major assistant provider has publicly committed to fetching llms.txt as part of its production web search, and analyses of server logs through 2026 keep finding the same thing: most published files get almost no requests from AI crawlers, and much of the traffic they do get is SEO tools checking whether the file exists.

Where it demonstrably works is narrower and more useful: tools that fetch a file when a person points them at it. Coding assistants pulling current documentation into a session are the clearest case, which is why documentation-heavy software companies are most of the adoption so far.

So should your site have one?

Ours exists because we are a documentation-heavy software product whose readers use exactly the tools that read these files, and because generating it costs us nothing per deploy. Those are the two conditions worth copying.

• Worth it if you publish documentation or an API reference, if developers and assistants read you, and if you can generate the file automatically.
• Probably not yet if it would be a hand-maintained copy of your menu, if your site is small enough that an assistant reading the pages loses nothing, or if you are hoping for a ranking effect. There is no evidence of one.

Either way, the work that pays regardless is the same work: pages with clear headings, one subject each, and a first paragraph that answers the question in the title. That is what helps an assistant quote you correctly, with or without a file at your root.

Questions people ask

If you want one, generate it, put a real sentence on every link, and keep the list short enough that you would defend every entry. And if your site is not that kind of site, skip it and go improve the pages themselves.