What is ClaudeBot?
ClaudeBot is Anthropic's AI training crawler. Training crawlers collect pages that may be used to train or improve AI models.
| Operator | Anthropic |
|---|---|
| Purpose (operator wording) | training |
| robots.txt token | ClaudeBot |
| User agent string | Not stated by the operator |
| Respects robots.txt | Yes |
| IP ranges | https://claude.com/crawling/bots.json |
| Last verified |
What else should I know about ClaudeBot?
What is the ClaudeBot user agent string?
Anthropic does not publish a full user agent string for ClaudeBot. The robots.txt token to use is ClaudeBot. To tell whether a visit is real, compare the visitor's IP address with the published list.
How do I block ClaudeBot in robots.txt?
Add this group to the robots.txt file at the root of your site. It asks ClaudeBot to stay out of every page.
User-agent: ClaudeBot
Disallow: /To allow it explicitly, for example while a broader rule blocks other bots:
User-agent: ClaudeBot
Allow: /How do I opt out of Anthropic training but stay in its search?
Each Anthropic user agent has its own token, so you can set them one by one. This file blocks ClaudeBot and allows Claude-User, Claude-SearchBot. Every line comes from the rows in this dataset. Whether allowing a bot keeps you in a product is Anthropic's call, so check each crawler's page.
User-agent: ClaudeBot
Disallow: /
User-agent: Claude-User
Allow: /
User-agent: Claude-SearchBot
Allow: /Does blocking ClaudeBot affect AI answers?
Anthropic: restricting ClaudeBot 'signals that the site's future materials should be excluded from our AI model training datasets.'
Each other Anthropic user agent has its own robots.txt token, so a rule for ClaudeBot does not cover them. See the related crawlers below.
Does ClaudeBot follow robots.txt?
Yes. Anthropic: 'Anthropic's Bots respect "do not crawl" signals by honoring industry standard directives in robots.txt.' Also supports Crawl-delay.
Robots.txt is a request, not a lock (RFC 9309).
What IP addresses does ClaudeBot use?
Anthropic publishes a list at https://claude.com/crawling/bots.json. We did not download or parse it. A request that claims to be ClaudeBot but comes from outside that list is not ClaudeBot.
Which other Anthropic crawlers are there?
What should I read next?
Next: give AI tools something to quote
Paperboy writes articles that answer the questions your buyers ask, with every fact linked to its source, FAQ and Article structured data, and publishes them to your site every week. Your first article is free.