🤖
User-Agent Token
TwinAgentRespects robots.txt
YesImpact Level
Niche
Smaller players and developer tools
Estimated Reach
Smaller players and developer tools
🎯What is TwinAgent?
TwinAgent is an autonomous AI agent operated by Twin. Autonomous AI browsing for tasks.
📊 How Your Data is Used
Powers workflow automation and AI agent tasks.
🚫 What Happens If You Block
Twin's AI agent can't automate tasks on your site.
🏢About Twin
🛡️TwinAgent robots.txt Configuration
Control TwinAgent access to your website using robots.txt directives.
Block TwinAgent
To completely block TwinAgent from crawling your site:
User-agent: TwinAgent
Disallow: /Allow TwinAgent Full Access
To explicitly allow TwinAgent to crawl your entire site:
User-agent: TwinAgent
Allow: /Selective Access for TwinAgent
To allow TwinAgent but restrict certain directories:
User-agent: TwinAgent
Disallow: /private/
Disallow: /api/
Disallow: /admin/
Allow: /✓ TwinAgent respects robots.txt directives.
TwinAgent User-Agent String
The user-agent token for TwinAgent is:
TwinAgent🌐Who Blocks TwinAgent?
Blocking (50 sites)
123inventatuweb.comaberdeennews.comagweek.comalbertleatribune.coman0ns.ruarkansasonline.comaskmen.comatari.orgbabycenter.cababycenter.combabycentre.co.ukbarcelona.catbattlecreekenquirer.combemidjipioneer.combeurs.nlbrainerddispatch.combuckscountycouriertimes.comburlingtoncountytimes.comcamera-wiki.orgchillicothegazette.comchrispederick.comcityweekly.netcolumbusmonthly.comcomune.prato.itcoshoctontribune.comcoupons.decptdb.cacrosswordheaven.comdailyamerican.comdailycommercial.comdaily-jeff.comdanspapers.comdiabetesdaily.comdiariodeavila.esdiariopalentino.esdicebreaker.comechopress.comeldiadevalladolid.comemmasdiary.co.ukeurogamer.eseurogamer.neteurogamer.pteveningsun.comexaminer-enterprise.comextremetech.comfdlreporter.comforeverdreaming.orggadsdentimes.comgamesindustry.bizgeek.com
Check Your Site's AI Policy
See if you're blocking or allowing TwinAgent and other AI crawlers.