What is Googlebot?
Googlebot is Google's search crawler. Search crawlers build the index a search or answer product draws on.
| Operator | |
|---|---|
| Purpose (operator wording) | search index |
| robots.txt token | Googlebot |
| User agent string | Desktop: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/W.X.Y.Z Safari/537.36 |
| Respects robots.txt | Yes |
| IP ranges | https://developers.google.com/static/crawling/ipranges/common-crawlers.json |
| Last verified |
What else should I know about Googlebot?
What is the Googlebot user agent string?
This is the user agent string Google documents for Googlebot, as of October 29, 2026. Version numbers and details can change, so match on the token in robots.txt, not on the whole string.
Desktop: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/W.X.Y.Z Safari/537.36How do I block Googlebot in robots.txt?
Add this group to the robots.txt file at the root of your site. It asks Googlebot to stay out of every page.
User-agent: Googlebot
Disallow: /To allow it explicitly, for example while a broader rule blocks other bots:
User-agent: Googlebot
Allow: /How do I opt out of Google training but stay in its search?
Each Google user agent has its own token, so you can set them one by one. This file blocks Google-Extended and allows Googlebot. Every line comes from the rows in this dataset. Whether allowing a bot keeps you in a product is Google's call, so check each crawler's page.
User-agent: Google-Extended
Disallow: /
User-agent: Googlebot
Allow: /Does blocking Googlebot affect AI answers?
Google lists affected products: Google Search (including Discover and Google Search features), Google Images, Google Video, Google News. Blocking it affects those, so do not block it to opt out of AI training.
Each other Google user agent has its own robots.txt token, so a rule for Googlebot does not cover them. See the related crawlers below.
Does Googlebot follow robots.txt?
Yes. Google lists a Googlebot robots.txt token on its common-crawlers page.
Robots.txt is a request, not a lock (RFC 9309).
What IP addresses does Googlebot use?
Google publishes a list at https://developers.google.com/static/crawling/ipranges/common-crawlers.json. We did not download or parse it. A request that claims to be Googlebot but comes from outside that list is not Googlebot.
Which other Google crawlers are there?
What should I read next?
Next: give AI tools something to quote
Paperboy writes articles that answer the questions your buyers ask, with every fact linked to its source, FAQ and Article structured data, and publishes them to your site every week. Your first article is free.