Check 8 of 12 · Technical health

Sitemap: is the XML sitemap reachable, and does it parse?

Where the sitemap check looks for a sitemap, what it validates, the protocol's size limits, and how to declare one in robots.txt.

What it checks

We look for a Sitemap: line in robots.txt first; if there is none we try /sitemap.xml and nothing else. We fetch it, decompress it if it is gzip, and parse it as a urlset or a sitemap index, counting the entries. We refuse files that declare a DOCTYPE or an ENTITY, and we stop reading at 10 MB.

Why it matters for search engines

A sitemap is how you tell search engines which pages exist and when they changed, without waiting for them to be discovered by links. One that is missing, broken or declared at a dead address means new and changed pages are found late or not at all.

Why it matters for AI assistants

Crawlers that respect robots.txt read the Sitemap: line too. It is the cheapest complete list of your pages you can give any crawler.

How to fix it

Publish /sitemap.xml and declare it in robots.txt with a Sitemap: line. Keep each file within the protocol's limits (50,000 URLs and 50 MB uncompressed) and use a sitemap index when you have more; we parse the first 10 MB of any one file, so smaller files are read in full by everyone.

The line to add to robots.txt
Sitemap: https://example.com/sitemap.xml

What quietwatch reports

ok with the entry count and where the sitemap was found (robots.txt or the conventional location). warn when there is no sitemap, when it is too large to read in full, or when robots.txt declares one that does not answer. fail when the file is corrupt or unsafe to parse. unknown when it could not be fetched.

Sources

All twelve checks · Previous: Indexability · Next: llms.txt

See what your site reports for sitemap

Check your site