XML sitemaps

What a sitemap is actually for#

The value is concentrated in a few situations, and outside them a sitemap changes very little. Knowing which situation you are in decides how much effort the file deserves.

• Deep or weakly linked pages. A page four clicks from anywhere, or reachable only through a filter, may be found slowly. The sitemap shortens that.
• Large corpora. A thousand pages published at once are discovered far faster from a list than from crawling the link graph outward.
• Change signalling. A truthful lastmod tells a crawler which pages are worth re-fetching, which matters more as a site grows.
• New sites. With few inbound links there is little for a crawler to follow, and the sitemap is close to the only entry point.

What a sitemap does not do: it does not cause indexing, it does not override a noindex, it does not consolidate duplicates and it does not confer priority over anyone else's pages. The <priority> and <changefreq> elements in the original protocol are widely ignored by the major search engines, and writing them carefully is effort spent on nothing.

lastmod, and the reason it is usually wrong#

lastmod is the one element still read closely, and it is trusted only while it is accurate. A file that stamps every URL with today's date each time it is generated is claiming the entire site changed today. After a few cycles of that the element is discounted for the whole domain, and you have destroyed the one signal you had.

The date must be a valid W3C datetime — a full date, or a date and time with a timezone offset. A date in the future is treated as invalid, and a timezone-naïve timestamp on a site that publishes across regions will be interpreted in a way you did not choose.

Sizes, index files and splitting#

A single sitemap file holds at most 50,000 URLs and at most 50 MiB uncompressed. Past either limit the file is invalid, not truncated. A sitemap index file is a list of sitemaps and is subject to the same 50,000 and 50 MiB limits on its own entries; indexes may not nest.

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://connectbyjbrh.com/sitemap-docs.xml</loc>
    <lastmod>2026-09-10</lastmod>
  </sitemap>
</sitemapindex>

Split by section rather than by arithmetic wherever you can. A sitemap per area of the site means the lastmod on the index entry is itself informative, and it makes a coverage problem attributable — if one section is under-crawled you can see which.

Does Connect use a sitemap, and how is it built?#

Used, and generated by the documentation build from the same page records the pages themselves are rendered from. Like robots.txt, it is served by the application rather than by a proxy, so a deployment carries its own sitemap and no environment can serve another environment's list.

Because the list and the pages come from one source, the build can hold a rule that a hand-maintained file never sustains: a public page missing from the sitemaps is a build failure, and a sitemap entry that is not a public page is also a build failure. The check runs on every build rather than on an audit schedule, which is why the two can never drift apart by more than one commit.

The application at /app is not listed, because it is not a public documentation page and it carries a noindex meta tag. Listing a noindex URL in a sitemap is a contradiction — an invitation to crawl something you have asked not to be indexed — and the build refuses it for that reason rather than as a style rule.

Questions#

Related#

• robots.txt
• IndexNow
• Canonical URLs
• The machine-readable documentation
• Treating documentation as a build artefact