LinkWalker
Avg Crawl Delay (when set) 424s
Mentions
1,326,544
0.11% of hostnames
Disallow All
1,323,505
99% of mentions
Allow
186
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
516
0% of mentions
Disallow Some
713
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
123
0% of mentions
No Directives
1,499
0% of mentions
Average Crawl Delay
424
(when set)