Gigabot
Avg Crawl Delay (when set) 134s
Mentions
1,665,579
0.14% of hostnames
Disallow All
1,594,094
95% of mentions
Allow
3,955
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
470
0% of mentions
Disallow Some
3,592
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
517
0% of mentions
No Directives
62,947
3% of mentions
Average Crawl Delay
134
(when set)