IonCrawl
Avg Crawl Delay (when set) 46s
Mentions
7,041,473
0.53% of hostnames
Disallow All
27,250
0% of mentions
Allow
17
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
5
0% of mentions
Disallow Some
7,014,135
99% of mentions
Disallow Some
+ Allow Some
+ Allow Some
2
0% of mentions
No Directives
64
0% of mentions
Average Crawl Delay
46
(when set)