GPTBot
Avg Crawl Delay (when set) 16s
Mentions
15,423,391
1.07% of hostnames
Disallow All
10,727,109
69% of mentions
Allow
502,040
3% of mentions
Disallow All
+ Allow Some
+ Allow Some
90,439
0% of mentions
Disallow Some
145,087
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,501,913
22% of mentions
No Directives
456,459
2% of mentions
Average Crawl Delay
16
(when set)