omgilibot
Avg Crawl Delay (when set) 17s
Mentions
4,651,426
0.32% of hostnames
Disallow All
1,470,431
31% of mentions
Allow
12,907
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
24,073
0% of mentions
Disallow Some
41,202
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
3,082,684
66% of mentions
No Directives
20,083
0% of mentions
Average Crawl Delay
17
(when set)