User-Agent Token
TikTokSpiderRespects robots.txt
UnknownImpact Level
Major
100M+ users - Meta, Apple, Microsoft, Perplexity, xAI
Estimated Reach
100M+ users - Meta, Apple, Microsoft, Perplexity, xAI
🎯What is TikTokSpider?
TikTokSpider is an AI training crawler operated by ByteDance. Collects data to train AI models.
🚫 What Happens If You Block
May help prevent content use in TikTok's AI features.
💡 Good to Know
Same concerns as Bytespider. Block both for complete ByteDance coverage.
🏢About ByteDance
🛡️TikTokSpider robots.txt Configuration
Control TikTokSpider access to your website using robots.txt directives.
Block TikTokSpider
To completely block TikTokSpider from crawling your site:
User-agent: TikTokSpider
Disallow: /Allow TikTokSpider Full Access
To explicitly allow TikTokSpider to crawl your entire site:
User-agent: TikTokSpider
Allow: /Selective Access for TikTokSpider
To allow TikTokSpider but restrict certain directories:
User-agent: TikTokSpider
Disallow: /private/
Disallow: /api/
Disallow: /admin/
Allow: /TikTokSpider User-Agent String
The user-agent token for TikTokSpider is:
TikTokSpider🌐Who Blocks TikTokSpider?
Blocking (47 sites)
15min.lt4p.de4players.deabendblatt.deaccountingtools.comacronymfinder.comadameteve.fradnuntius.comafp.aiairfind.comairplane-pictures.netalertustech.comallgaeuer-zeitung.deall-in.deamarillo.comapp.comardaudiothek.deard.deardmediathek.deard-text.deargusleader.comarkansasonline.comartvee.comaugsburger-allgemeine.deaugustachronicle.comautobild.deaxonvibe.comazcentral.combaamboozle.combabbel.combarcelona.catbaseballchannel.jpbeaconjournal.combfmtv.combild.debirdsoftheworld.orgblink.comblinkforhome.combolshoyvopros.rubraunschweiger-zeitung.debr.debrightline.tvbrigitte.debsc.esbucksfreepress.co.ukbusinessinsider.debz-berlin.de
Explicitly Allowing (3 sites)
🔗Other ByteDance Bots
Check Your Site's AI Policy
See if you're blocking or allowing TikTokSpider and other AI crawlers.