ClaudeBot
Avg Crawl Delay (when set) 18s
Mentions
8,139,843
0.68% of hostnames
Disallow All
4,610,679
56% of mentions
Allow
70,245
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
57,224
0% of mentions
Disallow Some
32,782
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,159,824
38% of mentions
No Directives
209,048
2% of mentions
Average Crawl Delay
18
(when set)