Applebot
Avg Crawl Delay (when set) 72s
Mentions
1,727,387
0.12% of hostnames
Disallow All
959,565
55% of mentions
Allow
245,383
14% of mentions
Disallow All
+ Allow Some
+ Allow Some
4,398
0% of mentions
Disallow Some
118,022
6% of mentions
Disallow Some
+ Allow Some
+ Allow Some
172,309
9% of mentions
No Directives
227,546
13% of mentions
Average Crawl Delay
72
(when set)