GPTBot
Avg Crawl Delay (when set) 10s
Mentions
9,348,553
0.79% of hostnames
Disallow All
5,426,295
58% of mentions
Allow
104,296
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
82,275
0% of mentions
Disallow Some
60,691
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,195,109
34% of mentions
No Directives
479,718
5% of mentions
Average Crawl Delay
10
(when set)