Site History
Track how thespec.com's technology stack and AI policy have changed over time
📍
Feb 14, 2026, 09:28 PMPolicy SnapshotLatest
UNKNOWN0 AI bots blocked
Policy changes:
CLOSED→UNKNOWN
Stopped blocking:
AI2BotAi2Bot-DolmaApplebot-ExtendedBytespiderCCBotChatGPT-UserClaude-WebClaudeBotDiffbotFacebookBotGPTBotGoogle-Extendedmeta-externalagentOAI-SearchBotPerplexityBotPetalBotScrapyTimpibotVelenPublicWebCrawlerwebzio-extendedYouBotanthropic-aicohere-aiimg2datasetomgiliomgilibotcohere-training-data-crawlerKangaroo BotPanguBotAmazonbotDuckAssistBotMeta-ExternalFetcherPerplexity-UserClaude-SearchBotClaude-UserCotoyogiDevinGemini-Deep-ResearchMistralAI-UserAddSearchBotGoogleOtherLinerBotQualifiedBotAI2Bot-DeepResearchEvalChatGLM-SpiderDeepSeekBotGoogle-NotebookLMZanistaBotatlassian-botBravebotCloudflare-AutoRAGCrawl4AIGoogle-CloudVertexBotimageSpiderKlaviyoAIBotlaion-huggingface-processorLCCLinkupBotPhindBotSBIntuitionsBotTavilyBot
Removed llms.txt
📍
🛡️
Jan 29, 2026, 05:43 PMPolicy Snapshot
CLOSED61 AI bots blocked (31 training)
robots.txtllms.txt
First recorded policy snapshot
Summary
1
Tech Scans
2
Policy Snapshots
25
Current Techs
0
Techs Added (All Time)