HaosouSpider
Avg Crawl Delay (when set) 24s
Mentions
348,018
0.03% of hostnames
Disallow All
189,975
54% of mentions
Allow
17,435
5% of mentions
Disallow All
+ Allow Some
+ Allow Some
61
0% of mentions
Disallow Some
1,398
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
477
0% of mentions
No Directives
138,670
39% of mentions
Average Crawl Delay
24
(when set)