GPTBot
Avg Crawl Delay (when set) 11s
Mentions
11,803,297
0.89% of hostnames
Disallow All
6,893,684
58% of mentions
Allow
164,016
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
85,887
0% of mentions
Disallow Some
70,164
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
4,100,538
34% of mentions
No Directives
488,800
4% of mentions
Average Crawl Delay
11
(when set)