Teoma
Avg Crawl Delay (when set) 203s
Mentions
1,491,560
0.13% of hostnames
Disallow All
1,302,550
87% of mentions
Allow
12,822
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
1,567
0% of mentions
Disallow Some
11,035
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
87,618
5% of mentions
No Directives
75,961
5% of mentions
Average Crawl Delay
203
(when set)