Google-CloudVertexBot
Avg Crawl Delay (when set) 19s
Mentions
2,427,197
0.18% of hostnames
Disallow All
438,346
18% of mentions
Allow
5,888
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
1,400
0% of mentions
Disallow Some
1,150
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
1,979,212
81% of mentions
No Directives
1,178
0% of mentions
Average Crawl Delay
19
(when set)