User-Agent Token
anthropic-aiRespects robots.txt
UnknownImpact Level
Notable
10M+ users - Regional/specialized AI companies
Estimated Reach
10M+ users - Regional/specialized AI companies
🎯What is anthropic-ai?
anthropic-ai is an AI training crawler operated by Anthropic. Collects data to train AI models.
🚫 What Happens If You Block
Legacy bot - likely no longer active.
💡 Good to Know
Older user-agent. ClaudeBot is the current primary crawler.
🏢About Anthropic
🛡️anthropic-ai robots.txt Configuration
Control anthropic-ai access to your website using robots.txt directives.
Block anthropic-ai
To completely block anthropic-ai from crawling your site:
User-agent: anthropic-ai
Disallow: /Allow anthropic-ai Full Access
To explicitly allow anthropic-ai to crawl your entire site:
User-agent: anthropic-ai
Allow: /Selective Access for anthropic-ai
To allow anthropic-ai but restrict certain directories:
User-agent: anthropic-ai
Disallow: /private/
Disallow: /api/
Disallow: /admin/
Allow: /anthropic-ai User-Agent String
The user-agent token for anthropic-ai is:
anthropic-ai🌐Who Blocks anthropic-ai?
Blocking (49 sites)
01net.com123rf.com15min.lt20minutes.fr20minutos.es21viptaka.com24bdbajee.com24heures.ch299bet.com47news.jp499bet.bingo678bd44.com6abc.com777tiger888.com7sur7.be888pg-ox.vip88betpk.com8newsnow.com9news.com.auaachener-zeitung.deabc11.comabc13.comabc27.comabc4.comabc6onyourside.comabc7chicago.comabc7.comabc7news.comabc7ny.comabc.net.auabcnews.comabendblatt.deabout.comacog.orgactu.fradl.orgad.nladnuntius.comadressa.noadvrider.comadweek.comaerzteblatt.deafp.aiafr.comaftenposten.noaftonbladet.seagrarheute.comagriculture.comahrq.gov
Explicitly Allowing (1 sites)
🔗Other Anthropic Bots
Check Your Site's AI Policy
See if you're blocking or allowing anthropic-ai and other AI crawlers.