Facebot
Avg Crawl Delay (when set) 67s
Mentions
443,364
0.03% of hostnames
Disallow All
70,074
15% of mentions
Allow
258,551
58% of mentions
Disallow All
+ Allow Some
+ Allow Some
25,841
5% of mentions
Disallow Some
44,280
9% of mentions
Disallow Some
+ Allow Some
+ Allow Some
26,919
6% of mentions
No Directives
17,692
3% of mentions
Average Crawl Delay
67
(when set)