Yahoo-MMCrawler
Avg Crawl Delay (when set) 257s
Mentions
1,176,642
0.1% of hostnames
Disallow All
1,054,169
89% of mentions
Allow
8,877
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
52
0% of mentions
Disallow Some
9,159
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
9,901
0% of mentions
No Directives
94,477
8% of mentions
Average Crawl Delay
257
(when set)