Yandex
Avg Crawl Delay (when set) 33s
Mentions
9,558,122
0.8% of hostnames
Disallow All
1,017,529
10% of mentions
Allow
184,336
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
6,760,894
70% of mentions
Disallow Some
583,411
6% of mentions
Disallow Some
+ Allow Some
+ Allow Some
665,446
6% of mentions
No Directives
346,320
3% of mentions
Average Crawl Delay
33
(when set)