IndexNow

The exchange, start to finish#

1. Generate a key: 8 to 128 characters, hexadecimal, unique to the host.

   Result The key is not a secret in the cryptographic sense. It is a proof of write access to the host, which is what the protocol needs.

2. Serve it as plain text at https://host/<key>.txt, with the key itself as the file's entire content.

   Result Any submission naming that key can be verified by fetching the file. A key file that 404s makes every submission fail verification.

3. POST the changed URLs, either one at a time as a GET-style request or as a JSON body listing many.

   Result The engine acknowledges receipt. Acknowledgement is not a promise to crawl, still less to index.

POST /indexnow HTTP/1.1
Host: api.indexnow.org
Content-Type: application/json; charset=utf-8

{
  "host": "connectbyjbrh.com",
  "key": "<8-128 hex characters>",
  "keyLocation": "https://connectbyjbrh.com/<key>.txt",
  "urlList": [
    "https://connectbyjbrh.com/docs/technology/indexnow/"
  ]
}

A bulk submission carries at most 10,000 URLs. Every URL in the list must belong to the host named in the host field; a mixed list is rejected in full rather than partially accepted.

Reading the response#

Does Connect use IndexNow?#

Used. The capability registry records IndexNow as available, implemented in tools/docs_indexnow.py as part of the documentation tooling rather than inside the running application. Submission is a publishing step, not a request-time behaviour, and keeping it in the build tools is what makes it possible to submit only what actually changed in a release.

That distinction is the whole design. Because the corpus is built from page records, a build can compare what it just produced against what was published before and derive the changed set. A page whose content is identical is not submitted, which keeps the submissions truthful and keeps them under the bulk limit without any batching logic.

Note also what IndexNow does *not* replace. The sitemap remains the complete list and the entry point for every crawler that does not participate; IndexNow is a notification about a subset. Publishing one without the other leaves either the discovery or the freshness half of the problem unsolved.

Practical cautions#

Key rotation

Rotating the key means serving the new key file before the first submission that names it. The old file may be removed once no in-flight submission references it.

Several keys on one host

Permitted, and useful where two systems publish to the same host. Each names its own keyLocation.

Subdomains

A key file proves control of one host. docs.example.net and example.net need separate key files.

Staging hosts

A preview environment that submits its own URLs is asking search engines to index a preview. Gate submission on the production host, not on a build flag somebody can forget.

Deletions

Submitting a URL that now returns 404 or 410 is legitimate and useful — it tells the engine to re-check a page you removed.

Questions#

Related#

• XML sitemaps
• robots.txt
• The machine-readable documentation
• HTTP caching and revalidation
• Measuring whether AI systems can actually read your site