Yandex
Avg Crawl Delay (when set) 33s
Mentions
9,016,643
0.78% of hostnames
Disallow All
1,001,171
11% of mentions
Allow
138,134
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
6,310,484
69% of mentions
Disallow Some
572,442
6% of mentions
Disallow Some
+ Allow Some
+ Allow Some
649,854
7% of mentions
No Directives
344,369
3% of mentions
Average Crawl Delay
33
(when set)