An AI search engine citing a page
A citation needs four things to be true, in order: the page was fetched, it was retained, it was selected for a particular question, and the answer credited it with a link. Only the first is directly observable from this side. The rest are inferred, and the honest position is that citation share is not something a publisher can measure.
The four conditions#
- Fetched. A crawler or a user-initiated client requested the page and received the real content. This is the only step that leaves direct evidence on this side.
- Retained. The content was kept in whatever index or store the product uses. Nothing on this side can see that; a fetch is not a promise of retention.
- Selected. For one particular question, this page beat everything else the product considered. Invisible, and question-by-question rather than site-wide.
- Credited. The answer named the source and linked it. Whether a product cites at all, and how, is the product's design decision, not the publisher's.
Most confusion about AI visibility comes from collapsing these four into one. A page that is fetched constantly and never cited has failed at step two, three or four, and those are different problems with different fixes — or, in the case of step four, with no fix available to a publisher at all.
Crawl to referral, stage by stage#
- Trigger — a crawler schedules a fetch, or a person asks an AI product a question that makes its client fetch this page.
- External event — the request arrives and is served the full page, its structured data and its canonical link.
- Authentication and workspace resolution — none. Everything in this flow is public.
- Ingest — the fetching product parses the page; the Markdown alternate exists for clients that prefer plain text over HTML.
- Canonical record — on this side, nothing. On the other side, an index entry that cannot be observed from here.
- Classification — the product decides what the page is about. The page's own answer paragraph, headings and capability status are the strongest signals it has.
- Knowledge and rules — the corpus is internally consistent on purpose: one page owns each primary question, and two pages never answer the same one differently.
- Selection — a person asks something; the product picks sources. Not observable, not schedulable, and not the same decision twice.
- Action — the product writes an answer and may attach a citation.
- Result — a citation, an uncredited use, or nothing. All three look identical from this side unless a link is followed.
- Referral — a reader clicks through. Some products send a referrer and some do not, so referral counts are a floor rather than a measurement.
- Audit — what can be recorded honestly is recorded: verified fetches per agent, and referrals that identify themselves. Nothing else is inferred into a number.
Measurable, inferable, unknowable#
| Question | Status | How |
|---|---|---|
| Did agent X get the page? | Measurable | Fetch as that agent and compare what came back; verify the fetches in the logs rather than counting user-agent strings |
| Is the page complete and parseable? | Measurable | The built checks: one H1, a canonical link, valid structured data that agrees with the visible text |
| Do readers arrive from an AI product? | Partly measurable | Referrals that identify themselves. Products that send no referrer are invisible, so this undercounts by an unknown amount |
| Is the page in a product's index? | Not observable | There is no publisher-facing view of another company's index |
| How often is it cited? | Not observable | Answers are generated per question and not published as a corpus anyone can count |
| Is it cited more than a competitor? | Not observable | Any figure of this shape is an estimate presented as a measurement |
What is actually within a publisher's control#
- Reachability: server-rendered HTML, no interstitial, no script requirement, and a policy that names purposes rather than companies.
- Answerability: one page per question, with the answer in the first paragraph rather than after four paragraphs of context.
- Checkability: sources listed on the page, a verified-on date, and a status word that distinguishes running from designed.
- Consistency: no two pages answering the same question differently, which is enforced rather than hoped for — duplicate primary queries fail the build.
- A plain-text twin of every page, so a client that would rather not parse HTML has an equivalent that cannot disagree with it.
None of that guarantees a citation. It removes the reasons a page cannot be used, which is the only part of the chain a publisher owns.
Where llms.txt fits, honestly#
This site publishes an llms.txt discovery map. It is a community proposal rather than a standard from any standards body, and Google has said no Search system reads it. It is published here as a supplement — a short map of the corpus for a client that finds it useful — and never as a substitute for HTML, robots rules or sitemaps, which are what actually govern access and discovery.
Questions#
How do I find out whether an AI product cites this documentation?
Ask it, in the products that matter to you, with the questions your readers actually ask, and look at whether a link comes back. That is an observation, not a metric, and repeating it over time is more informative than any single number a tool reports.
Does structured data make a citation more likely?
It makes a page easier to interpret correctly, which removes one reason to pass over it. Structured data that disagrees with the visible text does the opposite, which is why agreement is checked rather than just validity.
Should content be written for models instead of people?
No — the two goals overlap almost completely. Answering the question in the first paragraph, naming things precisely, stating limits and failure modes, and not contradicting the next page along all serve a reader first and a retrieval system as a consequence.