X-Robots-Tag checker: is a header hiding this page from Google?
Paste a page address. We fetch it the way a crawler does and read the response headers, on the final page and on every redirect hop, for an X-Robots-Tag that says noindex. View-source never shows a header; this does, and quotes the line.
Requests are sent as QuietwatchBot, never as another bot. Free: 5 checks an hour per address, or one bulk check of up to 10.
Check up to 10 addresses at once
What is the X-Robots-Tag header, and why does a checker for it exist?
X-Robots-Tag is an HTTP response header that carries the same instructions as the robots meta tag: noindex, nofollow, noarchive, nosnippet, unavailable_after and the rest. Google documents it beside the meta tag in its robots meta tag and X-Robots-Tag specifications, and treats the two as equals. The header exists because some files have no HTML to put a tag in: PDFs, images, videos. The trouble starts when it lands on HTML pages by accident, because nothing in the page, the CMS or an SEO plugin shows it. View-source cannot see a header. That is the whole reason for this checker: it reads the response headers, on the final page and on every redirect hop before it, and quotes the line.
| Where an X-Robots-Tag comes from | Typical case | Who can fix it |
|---|---|---|
| Web server config (nginx, Apache .htaccess, IIS) | A rule written for a staging site, copied to live | Whoever runs the server |
| Managed hosting platform | Staging and preview copies get the header; it stays after a push to production | The host, often a support ticket |
| CDN or edge rule (Cloudflare, Fastly, Vercel, Netlify) | A transform rule or a preview-deployment default | Whoever owns the CDN account |
| Application middleware | A framework setting meant for non-production environments | The developer |
| A redirecting hostname | The old domain or the http version still sends the header on its 301 | Whoever runs the old host |
How does the X-Robots-Tag checker read it?
We request the address the way a crawler does, as QuietwatchBot, after reading robots.txt. We follow the redirects and keep the headers of every hop, then read the final response. Three answers come back about the header itself: whether a redirect hop sent X-Robots-Tag with a noindex, whether the final response sends one, and the full value of each so you can see noarchive or nosnippet too. The page's meta robots tag, a googlebot meta tag, the robots.txt verdict and the canonical are shown alongside, because the fix depends on which one is responsible and they are easy to confuse. The result is a link that works for 30 days, which is the thing to send to the host or the developer.
How do I see the header myself?
From a terminal, curl -I https://example.com/page prints the response headers of the final page; add -L to follow redirects, and read each block in turn, because curl prints one set of headers per hop. In Chrome, open DevTools, the Network tab, tick "Preserve log", reload, click the page's own request and read Response Headers. The "Preserve log" box matters: without it, the redirect hops vanish from the list, and a header sent only by a hop is exactly the kind nobody finds. This checker does the same thing and keeps the hops.
Why does a header on a redirect hop still count?
Google reads the headers of every response in a redirect chain, not only the last one. A site that moved from http to https, or from an old domain to a new one, often leaves the old hostname on the previous host. If that host sends X-Robots-Tag: noindex on everything, including the 301, the final page can be perfectly clean and still drop out of Google. Our post on the invisible X-Robots-Tag noindex walks through finding and removing it; the checker shows the hop that sent it, which is the line to take to whoever runs the old host.
How do I remove it?
- Apache: search the site's .htaccess and the vhost config for
Header set X-Robots-Tagand remove or scope the rule. - nginx: search the server and location blocks for
add_header X-Robots-Tag; a rule in a staging server block sometimes applies to live through an include. - Managed WordPress hosts: the header usually comes from the platform's staging protection; ask support to clear it on the production environment and check again.
- Vercel and Netlify: preview deployments carry the header on purpose; production should not. Check
vercel.jsonornetlify.tomlheaders and any middleware that sets it by environment. - Cloudflare: look under Rules for a response header transform rule that adds X-Robots-Tag.
After the change, run the checker again, then use Search Console's URL Inspection: "Test live URL", then "Request indexing". Google reports what it saw at its last crawl, so the report lags the fix until the page is recrawled.
Why it matters
A header noindex is the most expensive kind because it is the one nobody sees. The CMS says the page is indexable, the SEO plugin is green, view-source is clean, and the page still disappears from Google as it is recrawled. Catching the header the day it appears is the difference between an email and a month of lost traffic, which is what quietwatch's daily indexability check is for.
How to use it
- Paste the address of the page you care about and press Check this site.
- Read the verdict at the top of the result, then the evidence under each answer: the exact line, tag, header or date.
- Follow the fix under any failing answer, run the check again, then keep the result link; it works for 30 days.
Questions
What is the X-Robots-Tag header?
An HTTP response header that carries robots directives such as noindex, nofollow, noarchive and nosnippet. Google treats it exactly like the robots meta tag. It exists for files with no HTML, such as PDFs, and causes trouble when it lands on pages by accident.
Why can't I see it in view-source or my SEO plugin?
Because it is not in the page. It is added when the response leaves the server, by the web server config, the hosting platform, a CDN rule or application middleware. Only the response headers show it.
How do I check the X-Robots-Tag header myself?
Run curl -I -L on the address and read every block of headers, one per redirect hop. In Chrome: DevTools, Network tab, tick Preserve log, reload, click the page's request and read Response Headers.
Does an X-Robots-Tag on a redirect count?
Yes. Google reads the headers of every hop in a redirect chain. An old hostname that still sends the header on its 301 keeps the final page out of Google even though the final page is clean.
Can the header say something other than noindex?
Yes: nofollow, noarchive, nosnippet, max-snippet, unavailable_after and more. The checker quotes the full value so you can see them, and only a noindex (or none) fails the page.
How do I remove it?
Search the server config for the rule (Header set in Apache, add_header in nginx), ask a managed host to clear its staging protection on production, check CDN transform rules, and check preview-only settings in Vercel, Netlify or middleware. Then recheck and request indexing.
How is this different from the noindex checker?
Same engine, different first question. The noindex checker asks which of four places says noindex. This page leads with the header, on the final response and on each hop, for the case where the CMS is clean and the page still vanished.