MSNBot
Avg Crawl Delay (when set) 22s
Mentions
5,822,580
0.42% of hostnames
Disallow All
522,103
8% of mentions
Allow
242,566
4% of mentions
Disallow All
+ Allow Some
+ Allow Some
21,817
0% of mentions
Disallow Some
171,769
2% of mentions
Disallow Some
+ Allow Some
+ Allow Some
176,434
3% of mentions
No Directives
4,687,849
80% of mentions
Average Crawl Delay
22
(when set)