CCBot
Avg Crawl Delay (when set) 45s
Mentions
13,976,200
0.97% of hostnames
Disallow All
10,122,781
72% of mentions
Allow
255,789
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
30,274
0% of mentions
Disallow Some
104,666
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,400,523
24% of mentions
No Directives
62,031
0% of mentions
Average Crawl Delay
45
(when set)