User-Agent Token
anthropic-aiRespects robots.txt
UnknownImpact Level
Notable
10M+ users - Regional/specialized AI companies
Estimated Reach
10M+ users - Regional/specialized AI companies
🎯What is anthropic-ai?
anthropic-ai is an AI training crawler operated by Anthropic. Collects data to train AI models.
🚫 What Happens If You Block
Legacy bot - likely no longer active.
💡 Good to Know
Older user-agent. ClaudeBot is the current primary crawler.
🏢About Anthropic
🛡️anthropic-ai robots.txt Configuration
Control anthropic-ai access to your website using robots.txt directives.
Block anthropic-ai
To completely block anthropic-ai from crawling your site:
User-agent: anthropic-ai
Disallow: /Allow anthropic-ai Full Access
To explicitly allow anthropic-ai to crawl your entire site:
User-agent: anthropic-ai
Allow: /Selective Access for anthropic-ai
To allow anthropic-ai but restrict certain directories:
User-agent: anthropic-ai
Disallow: /private/
Disallow: /api/
Disallow: /admin/
Allow: /anthropic-ai User-Agent String
The user-agent token for anthropic-ai is:
anthropic-ai🌐Who Blocks anthropic-ai?
Blocking (49 sites)
01net.com0kkbd.com10best.com11freunde.de123rf.com12bajeevip.com12g555.com13abc.com13g555.com13wham.com13x555.com15min.lt166bet.com1777sx.com1881.no20minutes.fr20minutos.es21viptaka.com24bdbajee.com24heures.ch24phcash.co2521673.com26superph.com299bet.com29news.com30seconds.com33bd.org355jilipark.vip3dvf.com3rr.com4029tv.com47news.jp499bet.bingo4p.de4players.de4plebs.org4vid.top52216452.com55brlbet.com57bet777.win588ku.com5gbet51.com5gbet.com6523145.com678bd44.com6abc.com72bet.me777jaya.net777tiger888.com
Explicitly Allowing (1 sites)
🔗Other Anthropic Bots
Check Your Site's AI Policy
See if you're blocking or allowing anthropic-ai and other AI crawlers.