Knocklog

ClaudeBot

Anthropic's AI training crawler.

Operator
Anthropic
Type
AI training crawler
In the user agent
ClaudeBot
robots.txt token
ClaudeBot
Follows robots.txt
says it follows robots.txt
Official documentation
support.claude.com
Checked on
2026-11-04

What is ClaudeBot?

ClaudeBot is Anthropic's AI training crawler. It collects pages that may be used to train AI models. A visit does not answer anyone right now; it decides what future models know about you.

The operator's description: Collects web content that could contribute to training Anthropic's models. (support.claude.com)

How do I block or allow ClaudeBot?

With robots.txt, using the token ClaudeBot. Anthropic says it follows robots.txt. To block the whole site:

User-agent: ClaudeBot
Disallow: /

To allow it, use Allow instead of Disallow. Before blocking, weigh what it brings: blocking does not remove you from AI search, but future models will know less about you.

How do I see whether ClaudeBot visits my site?

In your server or CDN access logs, look for ClaudeBot in the user agent. Google Analytics does not show these visits: agents do not run its JavaScript.

Anyone can copy the name in a user agent, so counting log lines is not enough. Knocklog reads your site's traffic on any host, confirms who is who, and shows how often ClaudeBot came, which pages it read and what that brought you.

How do I know it is really ClaudeBot?

Anthropic publishes ClaudeBot's IP addresses at https://claude.com/crawling/bots.json. A visit that claims to be ClaudeBot from outside those ranges is spoofed.

Knocklog runs this check on every visit and separates the genuine agent from impostors.

Does ClaudeBot visit your site?

See in seconds how your site treats AI agents. Then connect your traffic to learn who really came.

Other Anthropic agents

All AI agents