Site History
Track how blackfriday.com's technology stack and AI policy have changed over time
📍
Feb 9, 2026, 06:56 AMPolicy SnapshotLatest
OPEN0 AI bots blocked
robots.txt
Policy changes:
CLOSED→OPEN
Stopped blocking:
AddSearchBotAI2BotAI2Bot-DeepResearchEvalAi2Bot-DolmaAmazonbotanthropic-aiApplebotApplebot-Extendedatlassian-botBravebotBytespiderCCBotChatGLM-SpiderChatGPT-UserClaude-SearchBotClaude-UserClaude-WebClaudeBotcohere-aicohere-training-data-crawlerCotoyogiDeepSeekBotDevinDiffbotDuckAssistBotFacebookBotGPTBotKangaroo BotKlaviyoAIBotLinerBotmeta-externalagentMeta-ExternalFetcherMistralAI-UserNovaActOAI-SearchBotomgiliPanguBotPerplexity-UserPerplexityBotPetalBotQualifiedBotTimpibotVelenPublicWebCrawlerwebzio-extendedYouBotZanistaBotGoogle-Extended
📍
Feb 9, 2026, 06:55 AMTech ScanLatest
View Details16 technologies detected
📦
Feb 8, 2026, 01:48 AMTech Scan
View Details16 technologies detected
📦
🛡️
Jan 29, 2026, 07:55 AMPolicy Snapshot
CLOSED47 AI bots blocked (22 training)
robots.txt
First recorded policy snapshot
Summary
3
Tech Scans
2
Policy Snapshots
16
Current Techs
0
Techs Added (All Time)