AdIdxBot
Avg Crawl Delay (when set) 9s
Mentions
290,608
0.02% of hostnames
Disallow All
16,514
5% of mentions
Allow
22,961
7% of mentions
Disallow All
+ Allow Some
+ Allow Some
1,340
0% of mentions
Disallow Some
37,720
12% of mentions
Disallow Some
+ Allow Some
+ Allow Some
36,812
12% of mentions
No Directives
175,254
60% of mentions
Average Crawl Delay
9
(when set)