Question 1

Does sec.gov block AI crawlers?

Accepted Answer

No, sec.gov does not block any AI crawlers in its robots.txt.

Question 2

Does sec.gov block ChatGPT?

Accepted Answer

No, sec.gov allows GPTBot (OpenAI's crawler) to access the site.

Question 3

Does sec.gov block Claude?

Accepted Answer

No, sec.gov allows ClaudeBot (Anthropic's crawler) to access the site.

Question 4

Does sec.gov have an AI policy?

Accepted Answer

sec.gov has a open AI policy as determined by its robots.txt rules. It blocks 0 AI crawlers.

sec.gov