DuckDuckBot
Avg Crawl Delay (when set) 23s
Mentions
570,340
0.04% of hostnames
Disallow All
67,738
11% of mentions
Allow
101,121
17% of mentions
Disallow All
+ Allow Some
+ Allow Some
21,708
3% of mentions
Disallow Some
59,394
10% of mentions
Disallow Some
+ Allow Some
+ Allow Some
284,949
49% of mentions
No Directives
35,362
6% of mentions
Average Crawl Delay
23
(when set)