Google-Extended
Avg Crawl Delay (when set) 10s
Mentions
12,000,318
0.84% of hostnames
Disallow All
7,922,698
66% of mentions
Allow
342,746
2% of mentions
Disallow All
+ Allow Some
+ Allow Some
71,769
0% of mentions
Disallow Some
79,121
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,525,057
29% of mentions
No Directives
58,828
0% of mentions
Average Crawl Delay
10
(when set)