aiHitBot
Avg Crawl Delay (when set) 162s
Mentions
3,585,754
0.25% of hostnames
Disallow All
501,050
13% of mentions
Allow
567
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
1,242
0% of mentions
Disallow Some
823
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,080,929
85% of mentions
No Directives
1,116
0% of mentions
Average Crawl Delay
162
(when set)