PetalBot
Avg Crawl Delay (when set) 45s
Mentions
8,978,590
0.63% of hostnames
Disallow All
8,593,478
95% of mentions
Allow
91,220
1% of mentions
Disallow All
+ Allow Some
+ Allow Some
1,808
0% of mentions
Disallow Some
66,695
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
46,398
0% of mentions
No Directives
178,875
1% of mentions
Average Crawl Delay
45
(when set)