Site History
Track how thueringer-allgemeine.de's technology stack and AI policy have changed over time
📍
Feb 14, 2026, 02:53 PMPolicy SnapshotLatest
CLOSED55 AI bots blocked (30 training)
robots.txt
Policy changes:
Started blocking:
Amazon KendraInternet Archive BotBedrock BotCloudflare AutoRAGCohere Training CrawlerCohere AIFirecrawlGrok DeepSearchInternet Archiveimg2dataset (LAION)Meta External AgentOmgiliOmgilibotVelen Public Web CrawlerResearch BotGemini Deep ResearchGoogle NotebookLMProject MarinerMeta External FetcherMistral AI User
Stopped blocking:
amazon-kendraarchive.org_botbedrockbotCloudflare-AutoRAGcohere-training-data-crawlercohere-aiFirecrawlAgentGrok-DeepSearchia_archiverimg2datasetmeta-externalagentomgiliomgilibotVelenPublicWebCrawlerGemini-Deep-ResearchGoogle-NotebookLMGoogleAgent-MarinerMeta-ExternalFetcherMistralAI-User
📍
🛡️
Jan 29, 2026, 05:05 PMPolicy Snapshot
CLOSED54 AI bots blocked (28 training)
robots.txt
First recorded policy snapshot
Summary
1
Tech Scans
2
Policy Snapshots
12
Current Techs
0
Techs Added (All Time)