GoogleOther
Avg Crawl Delay (when set) 15s
Mentions
3,290,601
0.24% of hostnames
Disallow All
451,103
13% of mentions
Allow
16,634
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
46,301
1% of mentions
Disallow Some
6,615
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
2,685,827
81% of mentions
No Directives
84,099
2% of mentions
Average Crawl Delay
15
(when set)