Avg Crawl Delay (when set) 73s
Mentions
26,683
0.0% of hostnames
Disallow All
4,442
16% of mentions
Allow
14,468
54% of mentions
Disallow All
+ Allow Some
+ Allow Some
211
0% of mentions
Disallow Some
1,585
5% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,161
11% of mentions
No Directives
2,813
10% of mentions
Average Crawl Delay
73
(when set)