Webz.io Extended
Tier 4Crawls content for Webz.io's data repository sold for AI training.
webzio-extended🎯What is Webz.io Extended?
Webz.io Extended is an AI training crawler operated by Webz.io. Collects data to train AI models.
Webz.io sells structured web data to AI companies for training.
Content won't be included in datasets sold to AI companies.
Commercial data provider. Your content may end up training various AI models.
🏢About Webz.io
🛡️webzio-extended robots.txt Configuration
Control webzio-extended access to your website using robots.txt directives.
Block webzio-extended
To completely block Webz.io Extended from crawling your site:
User-agent: webzio-extended
Disallow: /Allow webzio-extended Full Access
To explicitly allow Webz.io Extended to crawl your entire site:
User-agent: webzio-extended
Allow: /Selective Access for webzio-extended
To allow Webz.io Extended but restrict certain directories:
User-agent: webzio-extended
Disallow: /private/
Disallow: /api/
Disallow: /admin/
Allow: /✓ Webz.io Extended respects robots.txt directives.
webzio-extended User-Agent String
The user-agent token for Webz.io Extended is:
webzio-extended🌐Who Blocks Webz.io Extended?
🔗Other Webz.io Bots
Check Your Site's AI Policy
See if you're blocking or allowing Webz.io Extended and other AI crawlers.