User-Agent Token
xAI-GrokRespects robots.txt
UnknownImpact Level
Major
100M+ users - Meta, Apple, Microsoft, Perplexity, xAI
Estimated Reach
100M+ users - Meta, Apple, Microsoft, Perplexity, xAI
🎯What is xAI-Grok?
xAI-Grok is an AI training crawler operated by xAI. Collects data to train AI models.
🚫 What Happens If You Block
Same as GrokBot. Block both for complete coverage.
💡 Good to Know
Alternate user-agent for xAI crawling.
🏢About xAI
🛡️xAI-Grok robots.txt Configuration
Control xAI-Grok access to your website using robots.txt directives.
Block xAI-Grok
To completely block xAI-Grok from crawling your site:
User-agent: xAI-Grok
Disallow: /Allow xAI-Grok Full Access
To explicitly allow xAI-Grok to crawl your entire site:
User-agent: xAI-Grok
Allow: /Selective Access for xAI-Grok
To allow xAI-Grok but restrict certain directories:
User-agent: xAI-Grok
Disallow: /private/
Disallow: /api/
Disallow: /admin/
Allow: /xAI-Grok User-Agent String
The user-agent token for xAI-Grok is:
xAI-Grok🌐Who Blocks xAI-Grok?
Blocking (27 sites)
abendblatt.deaid.nobbv-net.deblog.euroka.moeblv.nobraunschweiger-zeitung.dedallasnews.comdorstenerzeitung.deha-halden.nohellwegeranzeiger.delaagendalsposten.nolofotposten.nomorgenpost.denamdalsavisa.nonationen.noop.noranablad.norappler.comreebok.com.trringblad.noruhrnachrichten.desahibinden.comsandnesposten.nothueringer-allgemeine.detlz.dewaz.dewr.de
🔗Other xAI Bots
Check Your Site's AI Policy
See if you're blocking or allowing xAI-Grok and other AI crawlers.