msnbot-media
Avg Crawl Delay (when set) 122s
Mentions
160,470
0.01% of hostnames
Disallow All
66,884
41% of mentions
Allow
22,712
14% of mentions
Disallow All
+ Allow Some
+ Allow Some
23,467
14% of mentions
Disallow Some
11,378
7% of mentions
Disallow Some
+ Allow Some
+ Allow Some
12,905
8% of mentions
No Directives
23,113
14% of mentions
Average Crawl Delay
122
(when set)