GPTBot
Avg Crawl Delay (when set) 10s
Mentions
8,012,060
0.7% of hostnames
Disallow All
4,254,541
53% of mentions
Allow
72,393
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
69,837
0% of mentions
Disallow Some
61,530
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,123,166
38% of mentions
No Directives
430,444
5% of mentions
Average Crawl Delay
10
(when set)