WebCopier
Avg Crawl Delay (when set) 232s
Mentions
1,605,901
0.11% of hostnames
Disallow All
1,599,222
99% of mentions
Allow
312
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
703
0% of mentions
Disallow Some
1,001
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
635
0% of mentions
No Directives
4,022
0% of mentions
Average Crawl Delay
232
(when set)