Amazonbot
Avg Crawl Delay (when set) 46s
Mentions
11,595,146
0.83% of hostnames
Disallow All
7,461,855
64% of mentions
Allow
35,982
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
31,396
0% of mentions
Disallow Some
53,920
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,888,881
33% of mentions
No Directives
122,999
1% of mentions
Average Crawl Delay
46
(when set)