Google-Extended
Avg Crawl Delay (when set) 10s
Mentions
9,047,722
0.69% of hostnames
Disallow All
4,592,643
50% of mentions
Allow
108,460
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
70,266
0% of mentions
Disallow Some
35,627
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
4,152,043
45% of mentions
No Directives
88,606
0% of mentions
Average Crawl Delay
10
(when set)