YouBot
Avg Crawl Delay (when set) 12s
Mentions
4,239,438
0.3% of hostnames
Disallow All
996,870
23% of mentions
Allow
75,523
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
2,721
0% of mentions
Disallow Some
40,023
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,074,731
72% of mentions
No Directives
49,517
1% of mentions
Average Crawl Delay
12
(when set)