CCBot
Avg Crawl Delay (when set) 45s
Mentions
13,481,976
0.94% of hostnames
Disallow All
9,651,253
71% of mentions
Allow
232,780
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
29,763
0% of mentions
Disallow Some
103,385
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,401,340
25% of mentions
No Directives
63,324
0% of mentions
Average Crawl Delay
45
(when set)