AI Bot Policies
See how websites handle AI crawlers like GPTBot and ClaudeBot
#650
sohu.com
2 blocked
Unknown
#651
time.com
No AI bot rules
Allows AI
#653
google.com.hk
No AI bot rules
Allows AI
#654
bsky.app
No AI bot rules
Allows AI
#655
rutube.ru
1 blocked
Allows AI
#656
xhamster.com
1 blocked
Unknown
#658
paloaltonetworks.com
No AI bot rules
Allows AIllms.txt
#660
blogger.com
No AI bot rules
Allows AI
#661
ft.com
17 blocked
Blocks AI
#663
usatoday.com
47 blocked
Blocks AI
#664
gosuslugi.ru
No AI bot rules
Unknown
#669
docker.com
No AI bot rules
Allows AI
#670
t-online.de
8 blocked
Unknown
#671
shein.com
No AI bot rules
Unknown
#672
amazon.co.jp
33 blocked
Blocks AI
#674
ya.ru
2 blocked
Allows AI
#676
deviantart.com
No AI bot rules
Allows AI
#681
intel.com
No AI bot rules
Allows AI
#683
jquery.com
No AI bot rules
Allows AI
#684
sophos.com
1 blocked·12 allowed
Unknownllms.txt
#687
capcut.com
9 allowed
Allows AI
#688
tplinkcloud.com
No AI bot rules
Allows AI
#689
dynatrace.com
No AI bot rules
Allows AIllms.txt
#691
sharethrough.com
17 blocked
Blocks AI
#692
sina.com.cn
No AI bot rules
Allows AI
#693
fast.com
No AI bot rules
Allows AI
#697
branch.io
7 allowed
Allows AI
#699
trendmicro.com
18 blocked
Blocks AIllms.txt
#703
noaa.gov
No AI bot rules
Unknown
#705
tripadvisor.com
20 blocked
Blocks AI
#706
hotjar.com
No AI bot rules
Allows AI
#708
consultant.ru
No AI bot rules
Allows AI
#709
tandfonline.com
1 blocked
Unknown
#710
wix.com
No AI bot rules
Allows AI
#711
playstation.com
1 blocked
Allows AIllms.txt
#715
target.com
No AI bot rules
Allows AIllms.txt
#717
braze.com
No AI bot rules
Allows AIllms.txt
#718
berkeley.edu
No AI bot rules
Allows AI
#723
verisign.com
No AI bot rules
Allows AI
#724
digitalocean.com
No AI bot rules
Allows AI
#730
britannica.com
No AI bot rules
Allows AI
#731
rakuten.co.jp
No AI bot rules
Unknown
#732
2gis.com
No AI bot rules
Unknown
#733
okta.com
No AI bot rules
Allows AIllms.txt
#736
dreamhost.com
No AI bot rules
Allows AIllms.txt
#737
loc.gov
No AI bot rules
Allows AI
#739
ca.gov
No AI bot rules
Allows AI
#741
intercom.io
No AI bot rules
Allows AIllms.txt
#744
fandom.com
No AI bot rules
Allows AI
#748
arxiv.org
2 blocked
Allows AI
Showing 251-300 of 500 sites
Check any website's AI policy
See if they block GPTBot, ClaudeBot, and other AI crawlers.
Check AI Policy