Google-CloudVertexBot
Avg Crawl Delay (when set) 38s
Mentions
3,502,202
0.29% of hostnames
Disallow All
363,520
10% of mentions
Allow
4,199
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
2,610
0% of mentions
Disallow Some
903
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,130,514
89% of mentions
No Directives
438
0% of mentions
Average Crawl Delay
38
(when set)