msnbot-media
Avg Crawl Delay (when set) 129s
Mentions
148,938
0.01% of hostnames
Disallow All
60,184
40% of mentions
Allow
21,986
14% of mentions
Disallow All
+ Allow Some
+ Allow Some
22,571
15% of mentions
Disallow Some
10,251
6% of mentions
Disallow Some
+ Allow Some
+ Allow Some
12,523
8% of mentions
No Directives
21,412
14% of mentions
Average Crawl Delay
129
(when set)