Amazonbot
Avg Crawl Delay (when set) 48s
Mentions
12,576,779
0.88% of hostnames
Disallow All
8,859,670
70% of mentions
Allow
60,880
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
33,706
0% of mentions
Disallow Some
91,670
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,440,309
27% of mentions
No Directives
90,387
0% of mentions
Average Crawl Delay
48
(when set)