Gigabot
Avg Crawl Delay (when set) 132s
Mentions
1,760,923
0.15% of hostnames
Disallow All
1,687,840
95% of mentions
Allow
3,802
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
475
0% of mentions
Disallow Some
3,618
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
570
0% of mentions
No Directives
64,614
3% of mentions
Average Crawl Delay
132
(when set)