Knocklog

Googlebot

Google's search engine.

Operator
Google
Type
Search engine
In the user agent
Googlebot
robots.txt token
Googlebot
Follows robots.txt
says it follows robots.txt
Official documentation
developers.google.com
Checked on
2026-11-04

What is Googlebot?

Googlebot is Google's search engine. It indexes pages for a search engine, whose results also feed AI answers.

The operator's description: Google Search crawler; its index also feeds Google's AI features. (developers.google.com)

How do I block or allow Googlebot?

With robots.txt, using the token Googlebot. Google says it follows robots.txt. To block the whole site:

User-agent: Googlebot
Disallow: /

To allow it, use Allow instead of Disallow. Before blocking, weigh what it brings: without it, the assistant cannot find your pages to cite or recommend them.

How do I see whether Googlebot visits my site?

In your server or CDN access logs, look for Googlebot in the user agent. Google Analytics does not show these visits: agents do not run its JavaScript.

Anyone can copy the name in a user agent, so counting log lines is not enough. Knocklog reads your site's traffic on any host, confirms who is who, and shows how often Googlebot came, which pages it read and what that brought you.

How do I know it is really Googlebot?

Google publishes Googlebot's IP addresses at https://developers.google.com/static/crawling/ipranges/common-crawlers.json. A visit that claims to be Googlebot from outside those ranges is spoofed.

The IP's reverse DNS, confirmed forward and back, must end in .googlebot.com or .google.com.

Knocklog runs this check on every visit and separates the genuine agent from impostors.

Does Googlebot visit your site?

See in seconds how your site treats AI agents. Then connect your traffic to learn who really came.

Other Google agents

All AI agents