HaosouSpider
Avg Crawl Delay (when set) 22s
Mentions
414,661
0.03% of hostnames
Disallow All
205,168
49% of mentions
Allow
43,702
10% of mentions
Disallow All
+ Allow Some
+ Allow Some
159
0% of mentions
Disallow Some
1,775
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
941
0% of mentions
No Directives
162,914
39% of mentions
Average Crawl Delay
22
(when set)