ClaudeBot
Avg Crawl Delay (when set) 18s
Mentions
10,076,262
0.77% of hostnames
Disallow All
5,600,478
55% of mentions
Allow
96,170
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
59,021
0% of mentions
Disallow Some
34,513
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
4,070,744
40% of mentions
No Directives
215,282
2% of mentions
Average Crawl Delay
18
(when set)