Baiduspider
Avg Crawl Delay (when set) 42s
Mentions
15,967,696
1.14% of hostnames
Disallow All
9,159,408
57% of mentions
Allow
670,601
4% of mentions
Disallow All
+ Allow Some
+ Allow Some
25,732
0% of mentions
Disallow Some
5,419,537
33% of mentions
Disallow Some
+ Allow Some
+ Allow Some
190,930
1% of mentions
No Directives
501,418
3% of mentions
Average Crawl Delay
42
(when set)