Site History
Track how adweek.com's technology stack and AI policy have changed over time
📍
Feb 9, 2026, 08:36 AMPolicy SnapshotLatest
CLOSED34 AI bots blocked (29 training)
robots.txt
Policy changes:
Started blocking:
Internet Archive BotCohere AICohere Training CrawlerInternet ArchiveiAsk SpiderLAION HuggingFace ProcessorLeipzig Corpora CollectionMeta External AgentOmgiliVelen Public Web CrawlerWebz.io Extended
Stopped blocking:
archive.org_botcohere-aicohere-training-data-crawleria_archiveriaskspiderlaion-huggingface-processorLCCmeta-externalagentomgiliVelenPublicWebCrawlerwebzio-extended
📍
Feb 9, 2026, 08:34 AMTech ScanLatest
View Details40 technologies detected
Changes from previous scan:
+ Added:
core-js
📦
Feb 8, 2026, 03:48 AMTech Scan
View Details39 technologies detected
Changes from previous scan:
+ Added:
reCAPTCHAGoogle AdSenseLightboxSalesforce
- Removed:
AI Powered
📦
🛡️
Jan 29, 2026, 08:27 AMPolicy Snapshot
CLOSED34 AI bots blocked (28 training)
robots.txt
First recorded policy snapshot
Summary
3
Tech Scans
2
Policy Snapshots
40
Current Techs
5
Techs Added (All Time)