Yandex
Avg Crawl Delay (when set) 30s
Mentions
9,966,109
0.75% of hostnames
Disallow All
1,050,799
10% of mentions
Allow
219,570
2% of mentions
Disallow All
+ Allow Some
+ Allow Some
7,029,574
70% of mentions
Disallow Some
598,984
6% of mentions
Disallow Some
+ Allow Some
+ Allow Some
687,899
6% of mentions
No Directives
379,089
3% of mentions
Average Crawl Delay
30
(when set)