Applebot
Avg Crawl Delay (when set) 45s
Mentions
1,299,117
0.1% of hostnames
Disallow All
770,077
59% of mentions
Allow
110,060
8% of mentions
Disallow All
+ Allow Some
+ Allow Some
4,133
0% of mentions
Disallow Some
73,404
5% of mentions
Disallow Some
+ Allow Some
+ Allow Some
97,386
7% of mentions
No Directives
243,989
18% of mentions
Average Crawl Delay
45
(when set)