msnbot-media
Avg Crawl Delay (when set) 124s
Mentions
157,829
0.01% of hostnames
Disallow All
65,902
41% of mentions
Allow
22,734
14% of mentions
Disallow All
+ Allow Some
+ Allow Some
23,269
14% of mentions
Disallow Some
10,319
6% of mentions
Disallow Some
+ Allow Some
+ Allow Some
12,828
8% of mentions
No Directives
22,766
14% of mentions
Average Crawl Delay
124
(when set)