Site History
Track how dailymail.co.uk's technology stack and AI policy have changed over time
📍
Feb 9, 2026, 04:35 AMPolicy SnapshotLatest
CLOSED47 AI bots blocked (29 training)
robots.txt
Policy changes:
Started blocking:
Bedrock BotCohere AIGemini Deep Researchimg2dataset (LAION)Meta External AgentMeta External FetcherMistral AI UserOmgiliOmgilibotVelen Public Web CrawlerWebz.io Extended
Stopped blocking:
bedrockbotcohere-aiGemini-Deep-Researchimg2datasetmeta-externalagentMeta-ExternalFetcherMistralAI-UseromgiliomgilibotVelenPublicWebCrawlerwebzio-extended
📍
Feb 9, 2026, 04:35 AMTech ScanLatest
View Details14 technologies detected
Changes from previous scan:
+ Added:
core-js
📦
🛡️
Jan 30, 2026, 11:08 PMPolicy Snapshot
CLOSED47 AI bots blocked (29 training)
robots.txt
🛡️
Jan 29, 2026, 06:44 AMPolicy Snapshot
CLOSED47 AI bots blocked (29 training)
robots.txt
🛡️
Jan 29, 2026, 06:27 AMPolicy Snapshot
CLOSED47 AI bots blocked (29 training)
robots.txt
🛡️
Jan 29, 2026, 06:19 AMPolicy Snapshot
CLOSED47 AI bots blocked (29 training)
robots.txt
First recorded policy snapshot
Summary
2
Tech Scans
5
Policy Snapshots
14
Current Techs
1
Techs Added (All Time)