msnbot-media
Avg Crawl Delay (when set) 129s
Mentions
151,460
0.01% of hostnames
Disallow All
62,251
41% of mentions
Allow
21,695
14% of mentions
Disallow All
+ Allow Some
+ Allow Some
22,864
15% of mentions
Disallow Some
10,242
6% of mentions
Disallow Some
+ Allow Some
+ Allow Some
12,699
8% of mentions
No Directives
21,698
14% of mentions
Average Crawl Delay
129
(when set)