Teoma
Avg Crawl Delay (when set) 208s
Mentions
1,372,381
0.12% of hostnames
Disallow All
1,186,565
86% of mentions
Allow
13,128
0% of mentions
Disallow All
+ Allow Some
+ Allow Some
791
0% of mentions
Disallow Some
11,078
0% of mentions
Disallow Some
+ Allow Some
+ Allow Some
86,793
6% of mentions
No Directives
74,019
5% of mentions
Average Crawl Delay
208
(when set)