NerdyBot
Avg Crawl Delay (when set) 53s
Mentions
1,399,822
0.11% of hostnames
Disallow All
1,398,684
99% of mentions
Allow
132
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
45
0% of mentions
Disallow Some
99
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
255
0% of mentions
No Directives
607
0% of mentions
Average Crawl Delay
53
(when set)