Site History
Track how l1.nl's technology stack and AI policy have changed over time
📍
Apr 29, 2026, 07:21 AMPolicy SnapshotLatest
CLOSED56 AI bots blocked (29 training)
robots.txt
Policy changes:
Started blocking:
OmgilibotAtlassian BotCloudflare AutoRAGCohere AICohere Training CrawlerGemini Deep ResearchGoogle NotebookLMiAsk SpiderLAION HuggingFace ProcessorLeipzig Corpora CollectionMeta External AgentMeta External FetcherMistral AI UserOmgiliVelen Public Web CrawlerWebz.io Extended
Stopped blocking:
omgilibotatlassian-botBravebotCloudflare-AutoRAGcohere-aicohere-training-data-crawlerDevinDiffbotGemini-Deep-ResearchGoogle-NotebookLMiaskspiderlaion-huggingface-processorLCCmeta-externalagentMeta-ExternalFetcherMistralAI-UseromgiliTavilyBotVelenPublicWebCrawlerwebzio-extendedYouBot
📍
🛡️
Feb 2, 2026, 04:55 AMPolicy Snapshot
CLOSED61 AI bots blocked (30 training)
robots.txt
First recorded policy snapshot
Summary
1
Tech Scans
2
Policy Snapshots
8
Current Techs
0
Techs Added (All Time)