Amazonbot
Avg Crawl Delay (when set) 48s
Mentions
13,036,362
0.9% of hostnames
Disallow All
9,302,930
71% of mentions
Allow
73,335
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
34,071
0% of mentions
Disallow Some
99,345
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,436,714
26% of mentions
No Directives
89,773
0% of mentions
Average Crawl Delay
48
(when set)