AdIdxBot
Avg Crawl Delay (when set) 9s
Mentions
285,773
0.02% of hostnames
Disallow All
15,488
5% of mentions
Allow
20,916
7% of mentions
Disallow All
+ Allow Some
+ Allow Some
1,335
0% of mentions
Disallow Some
36,359
12% of mentions
Disallow Some
+ Allow Some
+ Allow Some
36,635
12% of mentions
No Directives
175,033
61% of mentions
Average Crawl Delay
9
(when set)