YouBot
Avg Crawl Delay (when set) 11s
Mentions
3,659,568
0.26% of hostnames
Disallow All
870,566
23% of mentions
Allow
47,204
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
1,857
0% of mentions
Disallow Some
3,432
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
2,653,747
72% of mentions
No Directives
82,715
2% of mentions
Average Crawl Delay
11
(when set)