AhrefsSiteAudit
Avg Crawl Delay (when set) 10s
Mentions
3,975,104
0.33% of hostnames
Disallow All
76,432
1% of mentions
Allow
46,033
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
119
0% of mentions
Disallow Some
3,775,686
94% of mentions
Disallow Some
+ Allow Some
+ Allow Some
42,684
1% of mentions
No Directives
34,140
0% of mentions
Average Crawl Delay
10
(when set)