🤖
Research Bot
Tier 4📚 AI Trainingby Various Universities
Academic research crawlers.
User-Agent Token
ResearchBotRespects robots.txt
YesImpact Level
Niche
Smaller players and developer tools
Estimated Reach
Smaller players and developer tools
🎯What is Research Bot?
Research Bot is an AI training crawler operated by Various Universities. Collects data to train AI models.
🏢About Various Universities
Various Universities
Various Universities operates 1 known bot for AI model training.
🛡️ResearchBot robots.txt Configuration
Control ResearchBot access to your website using robots.txt directives.
Block ResearchBot
To completely block Research Bot from crawling your site:
User-agent: ResearchBot
Disallow: /Allow ResearchBot Full Access
To explicitly allow Research Bot to crawl your entire site:
User-agent: ResearchBot
Allow: /Selective Access for ResearchBot
To allow Research Bot but restrict certain directories:
User-agent: ResearchBot
Disallow: /private/
Disallow: /api/
Disallow: /admin/
Allow: /✓ Research Bot respects robots.txt directives.
ResearchBot User-Agent String
The user-agent token for Research Bot is:
ResearchBot🌐Who Blocks Research Bot?
Check Your Site's AI Policy
See if you're blocking or allowing Research Bot and other AI crawlers.