SurveyBot
Avg Crawl Delay (when set) 1259s
Mentions
1,317,468
0.11% of hostnames
Disallow All
1,315,440
99% of mentions
Allow
257
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
82
0% of mentions
Disallow Some
739
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
136
0% of mentions
No Directives
810
0% of mentions
Average Crawl Delay
1259
(when set)