CloudflareBrowserRenderingCrawler
Avg Crawl Delay (when set) 4s
Mentions
4,661,370
0.32% of hostnames
Disallow All
4,660,746
99% of mentions
Allow
426
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
13
0% of mentions
Disallow Some
24
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
154
0% of mentions
No Directives
7
0% of mentions
Average Crawl Delay
4
(when set)