BLEXBot
Avg Crawl Delay (when set) 71s
Mentions
8,411,997
0.73% of hostnames
Disallow All
8,235,298
97% of mentions
Allow
1,031
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
2,231
0% of mentions
Disallow Some
33,133
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
2,202
0% of mentions
No Directives
138,087
1% of mentions
Average Crawl Delay
71
(when set)