Applebot
Avg Crawl Delay (when set) 50s
Mentions
1,321,343
0.1% of hostnames
Disallow All
785,105
59% of mentions
Allow
113,392
8% of mentions
Disallow All
+ Allow Some
+ Allow Some
3,361
0% of mentions
Disallow Some
73,160
5% of mentions
Disallow Some
+ Allow Some
+ Allow Some
99,909
7% of mentions
No Directives
246,346
18% of mentions
Average Crawl Delay
50
(when set)