Yandex
Avg Crawl Delay (when set) 30s
Mentions
9,976,754
0.76% of hostnames
Disallow All
1,038,455
10% of mentions
Allow
207,374
2% of mentions
Disallow All
+ Allow Some
+ Allow Some
7,070,817
70% of mentions
Disallow Some
599,979
6% of mentions
Disallow Some
+ Allow Some
+ Allow Some
682,241
6% of mentions
No Directives
377,696
3% of mentions
Average Crawl Delay
30
(when set)