WebCopier
Avg Crawl Delay (when set) 229s
Mentions
1,597,427
0.11% of hostnames
Disallow All
1,590,616
99% of mentions
Allow
293
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
684
0% of mentions
Disallow Some
996
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
643
0% of mentions
No Directives
4,189
0% of mentions
Average Crawl Delay
229
(when set)