Cloudflare-AI-Search
Cloudflare's AI search crawler.
- Operator
- Cloudflare
- In the user agent
Cloudflare-AI-Search- robots.txt token
- none
- Follows robots.txt
- does not say whether it follows robots.txt
- Official documentation
- developers.cloudflare.com
- Checked on
- 2026-11-11
What is Cloudflare-AI-Search?
Cloudflare-AI-Search is Cloudflare's AI search crawler. It builds the index an AI assistant searches when it answers. If it cannot read your pages, the assistant cannot cite or recommend them.
The operator's description: Indexes sites a Cloudflare customer owns and selects as a source for its AI Search.
(developers.cloudflare.com)
How do I block or allow Cloudflare-AI-Search?
Cloudflare-AI-Search has no robots.txt token of its own. Cloudflare does not say whether it follows robots.txt. Controlling its access means recognising its verified identity at the edge (CDN or server).
How do I see whether Cloudflare-AI-Search visits my site?
In your server or CDN access logs, look for Cloudflare-AI-Search in the user agent. Google Analytics does not show these visits: agents do not run its JavaScript.
Anyone can copy the name in a user agent, so counting log lines is not enough. Knocklog reads your site's traffic on any host, confirms who is who, and shows how often Cloudflare-AI-Search came, which pages it read and what that brought you.
How do I know it is really Cloudflare-AI-Search?
Cloudflare does not publish a way to verify Cloudflare-AI-Search. Any program can use that name, so Knocklog shows these visits as “unverified”, never as genuine.
Does Cloudflare-AI-Search visit your site?
See in seconds how your site treats AI agents. Then connect your traffic to learn who really came.