BLEXBot
Avg Crawl Delay (when set) 64s
Mentions
9,667,480
0.74% of hostnames
Disallow All
9,468,665
97% of mentions
Allow
1,488
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
2,440
0% of mentions
Disallow Some
30,230
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
2,612
0% of mentions
No Directives
162,010
1% of mentions
Average Crawl Delay
64
(when set)