User-Agent Token
anthropic-aiRespects robots.txt
UnknownImpact Level
Notable
10M+ users - Regional/specialized AI companies
Estimated Reach
10M+ users - Regional/specialized AI companies
🎯What is anthropic-ai?
anthropic-ai is an AI training crawler operated by Anthropic. Collects data to train AI models.
🚫 What Happens If You Block
Legacy bot - likely no longer active.
💡 Good to Know
Older user-agent. ClaudeBot is the current primary crawler.
🏢About Anthropic
🛡️anthropic-ai robots.txt Configuration
Control anthropic-ai access to your website using robots.txt directives.
Block anthropic-ai
To completely block anthropic-ai from crawling your site:
User-agent: anthropic-ai
Disallow: /Allow anthropic-ai Full Access
To explicitly allow anthropic-ai to crawl your entire site:
User-agent: anthropic-ai
Allow: /Selective Access for anthropic-ai
To allow anthropic-ai but restrict certain directories:
User-agent: anthropic-ai
Disallow: /private/
Disallow: /api/
Disallow: /admin/
Allow: /anthropic-ai User-Agent String
The user-agent token for anthropic-ai is:
anthropic-ai🌐Who Blocks anthropic-ai?
Blocking (49 sites)
01net.com01rus.ru05bet.me0kkbd.com100layercake.com104nk88.com10best.com112bdt222.com112phsky.vip113bd333.vip116bdt222.com116phsky.vip11freunde.de121bet15.com121bet.cc123rf.com12bajeevip.com12g555.com12no777.com13abc.com13g555.com13wham.com13x555.com157jilipark.com15min.lt166bet.com1777cb.com1777sx.com1777sz.com17bdbajee.com1881.no1xjl.cfd20minutes.fr20minutos.es21viptaka.com222bd2.com228aceph.cc22hero789.com24bdbajee.com24heures.ch24phcash.co24phdream.com24phpark.com24wink.com2521673.com25newsnow.com26superph.com2777cb.com2777cx.com
Explicitly Allowing (1 sites)
🔗Other Anthropic Bots
Check Your Site's AI Policy
See if you're blocking or allowing anthropic-ai and other AI crawlers.