psbot
Avg Crawl Delay (when set) 119s
Mentions
2,480,410
0.21% of hostnames
Disallow All
2,399,850
96% of mentions
Allow
4,770
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
506
0% of mentions
Disallow Some
4,105
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
1,697
0% of mentions
No Directives
69,476
2% of mentions
Average Crawl Delay
119
(when set)