Claude-User
Tier 1Fetches pages when Claude users request web content.
Claude-User🎯What is Claude-User?
Claude-User is an AI assistant fetcher operated by Anthropic. Fetches content for AI assistant responses.
On-demand fetching. Not stored or used for training.
Users can't ask Claude to read your pages directly.
User-initiated. Popular for research, coding help, and document analysis.
🏢About Anthropic
Anthropic operates 5 known bots for AI assistant features. Their service reaches 100M+ users.
🛡️Claude-User robots.txt Configuration
Control Claude-User access to your website using robots.txt directives.
Block Claude-User
To completely block Claude-User from crawling your site:
User-agent: Claude-User
Disallow: /Allow Claude-User Full Access
To explicitly allow Claude-User to crawl your entire site:
User-agent: Claude-User
Allow: /Selective Access for Claude-User
To allow Claude-User but restrict certain directories:
User-agent: Claude-User
Disallow: /private/
Disallow: /api/
Disallow: /admin/
Allow: /✓ Claude-User respects robots.txt directives.
Claude-User User-Agent String
The user-agent token for Claude-User is:
Claude-User🌐Who Blocks Claude-User?
🔗Other Anthropic Bots
Check Your Site's AI Policy
See if you're blocking or allowing Claude-User and other AI crawlers.