llms.txt
llms.txt is a proposed convention: a Markdown file at the root of a site listing its most useful pages, so a language model given the site has a map instead of a crawl. It is a community proposal, not a standard, and Google has said no Search system reads it. This site publishes one as a supplemental map.
What the file is#
A Markdown document at /llms.txt: a heading with the site's name, a blockquote summarising it, then sections of links, each with a short description of what the page contains. It is meant to be read rather than parsed — the format's whole argument is that a curated list of the right pages, in prose, is more useful to a model than a sitemap of every URL with no indication of which matter.
A companion convention, llms-full.txt, concatenates the content itself rather than linking to it, for a consumer that would rather have one long document than follow twenty links.
The problem it addresses is real. A model handed a website has to spend its context discovering structure that the site's owner already knows: which pages are canonical, which are navigation, which answer the question actually being asked.
What it is not#
It is not an IETF or W3C standard, has no registry, no conformance tests and no obligation on anyone to read it. Adoption among consumers is the open question: Google has said that no Google Search system reads llms.txt, and no major search or answer engine has committed to treating it as an input.
- It is not access control. Listing a page grants nothing and omitting one hides nothing.
- It is not a crawl directive. That is robots.txt, which is a real standard with real compliance.
- It is not a sitemap. A sitemap is for discovery of everything; this is a curated subset with commentary.
- It is not a substitute for HTML that a machine can read. A site whose pages are unreadable without executing scripts is not fixed by a text file listing them.
Does Connect use llms.txt?#
Yes — this site publishes one at /llms.txt, and an /llms-full.txt beside it, with the reasoning stated plainly rather than dressed up. Both are generated by the documentation build from the same registry as the pages, so the map cannot drift from the corpus it describes.
Three honest reasons, and no fourth:
- The file costs one generator step and no maintenance, because it is generated rather than written.
- An agent or a person who is *handed* the file gets an accurate map — the topic hubs, the machine interfaces, the capability status manifest — which is genuinely faster than crawling a thousand pages.
- If a consumer ever does start reading these files, the map is already correct. That is a cheap option, not a prediction.
What the file is explicitly not doing here is standing in for anything that works. The HTML is server-rendered and readable without scripts, robots.txt states crawler policy per agent, sitemaps list every public page, and the documentation is also published as Markdown alternates and as machine manifests. llms.txt is a supplement to those, and it is listed as an MCP resource for the same reason — it is a convenient map, not a load-bearing surface.
If you are deciding whether to publish one#
Generate it rather than writing it.
Result A hand-written map decays quietly. A generated one fails a build when it stops matching.
Fix your HTML first if it needs fixing.
Result A machine-readable site helps every consumer. A text file helps the ones that read it, which today may be none.
Describe each link in one honest sentence.
Result The descriptions are the only part that does something a sitemap cannot. A bare list of URLs is a worse sitemap.
Say what it is when you announce it.
Result Nobody is misled into thinking a search engine is now reading your site differently.
Questions#
Will publishing llms.txt improve how AI systems answer questions about my site?
There is no evidence that it does, and Google has said no Search system reads it. What demonstrably helps is HTML a machine can read without executing scripts, clear structure, and a crawler policy that allows the agents you want. Publish llms.txt because it is cheap and correct, not because it is expected to change an outcome.
How is it different from robots.txt?
Entirely. RFC 9309 is a published standard that well-behaved crawlers obey, and it expresses what may be fetched. llms.txt is a proposal that expresses what is worth reading, and nothing is obliged to read it.
What is in this site's file?
The product summary, the version, a link to the machine-readable capability status, and a curated set of pages by topic — each with a sentence saying what it contains. See llms.txt on this site for the generated file and how it is built.