Google-InspectionTool
Avg Crawl Delay (when set) 47s
Mentions
66,449
0.01% of hostnames
Disallow All
10,836
16% of mentions
Allow
11,734
17% of mentions
Disallow All
+ Allow Some
+ Allow Some
315
0% of mentions
Disallow Some
1,937
2% of mentions
Disallow Some
+ Allow Some
+ Allow Some
39,282
59% of mentions
No Directives
2,343
3% of mentions
Average Crawl Delay
47
(when set)