Short answer: Error 1020 means a Cloudflare firewall rule matched the request and told Cloudflare to block it. To find which rule blocked Googlebot, open the site's Security Events log in the Cloudflare dashboard, filter by the Googlebot user agent or by the Ray ID from the error page, and read the Service and Rule columns on the matching row. That row names the rule. Edit or reorder it so verified Googlebot is allowed, then re-test the page in Search Console.
What does error 1020 actually mean?
Error 1020 is the "Access denied" page. According to Cloudflare's error 1020 documentation, it appears when a request violates a firewall rule set on the site (see Sources). It is not a server error and your WordPress install never saw the request. Cloudflare stopped it at the edge, so your origin access logs will show nothing for these hits. That is why it is easy to miss: the site looks fine to you, but Googlebot may get a block page.
If a crawler keeps getting blocked, new or changed pages may not be fetched. Google's Googlebot overview, linked in Sources, is the place to check how Google describes crawling behaviour.
How do you find the exact rule that blocked it?
- Get a Ray ID. Every Cloudflare block page shows one at the bottom. If you do not have one, run the test below to generate one.
- Open the log. In the Cloudflare dashboard, pick the domain, then Security, then Events (the menu wording shifts between dashboard versions).
- Filter. Add a filter for Ray ID equals your value, or for User Agent contains
Googlebot, and set the time range to the last 24 hours. - Read the row. Expand it. The Service column tells you which product acted (custom rules, rate limiting, managed rules, IP access rules, Bot Fight Mode and so on). The Rule column or rule ID points to the specific rule.
- Open that rule and read its expression. Look for country blocks, ASN blocks, threat score conditions, or a user agent match that catches crawlers.
Cloudflare describes this log on its Security Events documentation page, linked in Sources.
Can you reproduce it from the command line?
Yes, with one caveat. This sends a request that claims to be Googlebot:
curl -sI -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" https://example.com/Look at the status line and the cf-ray header. A 403 with a Cloudflare Ray ID is a strong sign a rule matched. The caveat: your laptop is not a Google IP, so Cloudflare may treat this as a fake Googlebot, and a block here does not prove the real crawler is blocked. Use it to get a Ray ID and a first lead, then confirm with real Googlebot hits in Security Events, or with the URL Inspection live test in Search Console, which fetches from Google's own network.
If you want a quick read without the terminal, the Cloudflare Googlebot check does a similar test from outside and tells you what came back.
How do you tell a real Googlebot from a fake one?
Do not trust the user agent string, because anyone can send it. Google's "Verifying Googlebot" page describes a reverse DNS lookup on the requesting IP, followed by a forward lookup to confirm the name maps back to the same IP (see Sources). Its worked example is a command you can run yourself:
host 66.249.66.1As we read that page, the answer should be a hostname under googlebot.com (the page shows crawl-66-249-66-1.googlebot.com). This is Google's public example from its own documentation, not data from any site we monitor. Cloudflare's verified bots documentation says it keeps a list of known good bots, and rules that use the verified bot field can skip real Googlebot while still blocking impostors.
Which rules usually catch Googlebot by mistake?
| Where it shows in Security Events | Typical cause | Fix |
|---|---|---|
| Custom rule | Country block, ASN block, or a broad user agent or path match | Add an exception for verified bots, or tighten the expression |
| IP access rule | A Google IP range blocked by hand or by an old import | Remove the rule for that range |
| Rate limiting | Googlebot crawling fast on a large site | Exclude verified bots or raise the threshold |
| Bot Fight Mode or Super Bot Fight Mode | Bot score too strict for a plugin or crawler pattern | Review the verified bot setting and the rule actions |
Is it a firewall block or a noindex problem?
They look alike in Search Console but have different fixes. A firewall block stops the fetch itself. A noindex rule, set in a robots meta tag or in an X-Robots-Tag HTTP header, lets Googlebot fetch the page but asks Google to keep it out of the index. A robots.txt disallow stops crawling but is not the same as noindex. Google's page on blocking indexing, linked in Sources, covers the difference. If a page shows a 403 with a cf-ray header, start with Cloudflare. If it returns 200 with a noindex header, the firewall is not your problem. Also check that your sitemap URLs return 200 to a Googlebot user agent, since a sitemap full of blocked URLs gives Search Console nothing useful to read.
How do you fix it on a WordPress client site?
- Fix the rule you found. Prefer an expression like "block when not a verified bot" over a hand-maintained IP list.
- Check rule order. A block above your allow or skip rule wins.
- Purge nothing yet. This is a firewall issue, not a cache issue.
- In Search Console, run URL Inspection on the home page and one deep page, then use the live test. Confirm "Page fetch" succeeds.
- Watch Crawl stats in Search Console over the next days for the fetch error count to drop.
Also check security plugins on the WordPress side. A plugin that blocks by country can cause the same symptom with a different error page.
What do you tell the client?
Keep it short and factual: "Our Cloudflare firewall was blocking Google's crawler on some requests. We found the rule, changed it so verified Googlebot is allowed, and confirmed with Google's own test. We are watching crawl reports for a week to make sure fetches recover." Do not promise a ranking change or a date. Google decides when to recrawl.
A pattern worth checking on any client site is a country or ASN rule added months earlier to stop spam, which can catch crawlers as well. Nobody notices because the site loads fine for humans.
How does quietwatch catch this every day?
A one-off test finds the problem today, but rules change and so do bot settings. quietwatch fetches each client site as a crawler every day and alerts you when it gets a block instead of the page, and you can see what quietwatch checks every day. The free Cloudflare Googlebot check above is the place to start.
Sources
- Cloudflare docs: Error 1020 Access denied
- Cloudflare docs: Security Events
- Cloudflare docs: Verified bots
- Google Search Central: Verifying Googlebot (source of the
host 66.249.66.1example) - Google Search Central: Googlebot overview
- Google Search Central: Overview of Google crawlers and fetchers
- Google Search Central: Block Search indexing with noindex