Amazonbot
Avg Crawl Delay (when set) 28s
Mentions
7,277,098
0.61% of hostnames
Disallow All
3,897,342
53% of mentions
Allow
12,423
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
27,846
0% of mentions
Disallow Some
47,888
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,175,549
43% of mentions
No Directives
115,953
1% of mentions
Average Crawl Delay
28
(when set)