CCBot
Avg Crawl Delay (when set) 45s
Mentions
13,883,626
0.96% of hostnames
Disallow All
10,035,152
72% of mentions
Allow
251,840
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
30,254
0% of mentions
Disallow Some
104,009
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,400,151
24% of mentions
No Directives
62,085
0% of mentions
Average Crawl Delay
45
(when set)