YisouSpider
Avg Crawl Delay (when set) 204s
Mentions
736,651
0.05% of hostnames
Disallow All
351,401
47% of mentions
Allow
166,136
22% of mentions
Disallow All
+ Allow Some
+ Allow Some
4,057
0% of mentions
Disallow Some
23,461
3% of mentions
Disallow Some
+ Allow Some
+ Allow Some
13,539
1% of mentions
No Directives
178,029
24% of mentions
Average Crawl Delay
204
(when set)