Facebot
Avg Crawl Delay (when set) 67s
Mentions
462,248
0.03% of hostnames
Disallow All
79,520
17% of mentions
Allow
268,027
57% of mentions
Disallow All
+ Allow Some
+ Allow Some
25,780
5% of mentions
Disallow Some
44,232
9% of mentions
Disallow Some
+ Allow Some
+ Allow Some
27,058
5% of mentions
No Directives
17,624
3% of mentions
Average Crawl Delay
67
(when set)