Avg Crawl Delay (when set) 68s
Mentions
37,311
0.0% of hostnames
Disallow All
4,727
12% of mentions
Allow
20,563
55% of mentions
Disallow All
+ Allow Some
+ Allow Some
250
0% of mentions
Disallow Some
1,526
4% of mentions
Disallow Some
+ Allow Some
+ Allow Some
7,247
19% of mentions
No Directives
2,994
8% of mentions
Average Crawl Delay
68
(when set)