DataForSeoBot
Avg Crawl Delay (when set) 93s
Mentions
1,361,803
0.1% of hostnames
Disallow All
1,313,925
96% of mentions
Allow
16,357
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
447
0% of mentions
Disallow Some
2,274
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
5,611
0% of mentions
No Directives
23,173
1% of mentions
Average Crawl Delay
93
(when set)