Yisouspider
Avg Crawl Delay (when set) 466s
Mentions
458,119
0.04% of hostnames
Disallow All
221,462
48% of mentions
Allow
77,161
16% of mentions
Disallow All
+ Allow Some
+ Allow Some
3,831
0% of mentions
Disallow Some
16,507
3% of mentions
Disallow Some
+ Allow Some
+ Allow Some
4,113
0% of mentions
No Directives
135,030
29% of mentions
Average Crawl Delay
466
(when set)