Amazonbot
Avg Crawl Delay (when set) 48s
Mentions
12,938,776
0.9% of hostnames
Disallow All
9,214,125
71% of mentions
Allow
70,965
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
34,016
0% of mentions
Disallow Some
93,550
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,436,056
26% of mentions
No Directives
89,873
0% of mentions
Average Crawl Delay
48
(when set)