OpenRobotsTXT

TelegramBot

Avg Crawl Delay (when set) 36s

Mentions

23,501

0.0% of hostnames

Disallow All

12,828

54% of mentions

Allow

6,881

29% of mentions

Disallow All
+ Allow Some

293

1% of mentions

Disallow Some

1,300

5% of mentions

Disallow Some
+ Allow Some

1,374

5% of mentions

No Directives

823

3% of mentions

Average Crawl Delay

36

(when set)