Many site owners have seen “Google-Extended” mentioned in robots.txt discussions and wondered whether blocking it will hurt their Google rankings. The short answer is no. Google-Extended controls how your content is used for Google’s AI models, not whether your pages appear in Google Search. But the details matter, because it also affects how your content can be used in Gemini experiences.
This guide explains what Google-Extended is, what it controls, what it does not control, and how to decide whether to allow or block it.
What Is Google-Extended?
Google-Extended is a product token you can use in your robots.txt file. It lets you tell Google whether content from your site may be used to help train future Gemini models and to ground answers in certain Gemini products, such as Gemini Apps and the Gemini API on Vertex AI.
An important detail: Google-Extended is not a separate crawler that visits your site. Google crawls with its normal user agents, such as Googlebot. Google-Extended is a control signal that tells Google how it may use content it has already crawled.
What Google-Extended Controls and What It Does Not
| Area | Controlled by Google-Extended? |
|---|---|
| Training future Gemini models | Yes |
| Grounding in Gemini Apps and Vertex AI Gemini API | Yes |
| Crawling and indexing for Google Search | No |
| Ranking in Google Search results | No |
| Appearance in AI Overviews and AI Mode | No, these follow Google Search controls |
This is the part most people misunderstand. AI Overviews and AI Mode are features of Google Search, so they are governed by normal Search controls such as Googlebot access, noindex, and snippet directives like nosnippet. Blocking Google-Extended does not remove you from AI Overviews.
How to Block Google-Extended
If you decide to opt out, add this group to your robots.txt file:
User-agent: Google-Extended
Disallow: /You can also block specific folders only. For example, you might protect premium research while allowing everything else:
User-agent: Google-Extended
Disallow: /research/Googlebot will continue to crawl and index your site as normal for Search.
How to Allow Google-Extended
If you have no rule for Google-Extended and no blanket block for all user agents, it is allowed by default. You can also make it explicit:
User-agent: Google-Extended
Allow: /Check your robots.txt carefully. Some plugins and security tools now add AI-related blocks automatically, so your site may be blocking Google-Extended without you realising.
Should You Block Google-Extended?
There is no single right answer. It depends on your business model and goals.
Reasons to Allow It
- You want your brand and content to be represented accurately in Gemini
- You rely on visibility and awareness more than on exclusive content
- You want Gemini grounding to be able to use your latest pages
- You see AI assistants as a growing discovery channel for your customers
Reasons to Block It
- Your revenue depends on exclusive, paywalled, or licensed content
- You are concerned about your work being used to train models without compensation
- You have legal or contractual obligations about how your content is used
Many marketing, service, and ecommerce sites choose to allow Google-Extended because visibility is more valuable to them than restricting use. Publishers with premium content are more likely to block it, at least for their most valuable sections.
Google-Extended vs Other AI Controls
Google-Extended is one of several AI-related controls you may see in robots.txt. OpenAI uses GPTBot for training and OAI-SearchBot for search, Anthropic uses ClaudeBot, Apple uses Applebot-Extended, and Common Crawl uses CCBot. Each works slightly differently. Treat every AI control as its own decision rather than blocking them all with one rule. Our overview of AI search visibility explains how these choices affect where your brand appears.
What Happens After You Block It
Blocking Google-Extended is a forward-looking signal. It tells Google how it may use your content from that point onward, but it does not erase knowledge a model may already have learned in the past. Gemini may still mention your brand based on other sources across the web, such as news coverage, reviews, and directories. If your main concern is accuracy rather than training, keeping your own information clear and consistent everywhere is often more effective than a block. It is also worth remembering that users can still paste your content into Gemini themselves, so robots.txt controls are one tool among several, not a complete content protection strategy.
Practical Checklist
- Open your live robots.txt and look for Google-Extended rules
- Check whether any wildcard rules affect it unintentionally
- Decide based on your content value and visibility goals
- Document the decision so your team understands it
- Review it every six months as AI products change
If you are unsure how your robots.txt, indexing, and AI access fit together, an AI SEO audit will give you a complete picture. And for the wider strategy, see our guide to AI SEO.
The Bottom Line
Google-Extended lets you control whether your content helps train and ground Gemini, without touching your Google Search rankings or your eligibility for AI Overviews. For most businesses focused on growth and visibility, allowing it is reasonable. For businesses whose content is the product, a selective block may make more sense.
Frequently Asked Questions
What is Google-Extended?
Google-Extended is a robots.txt product token that controls whether your content can be used to train future Gemini models and to ground answers in certain Gemini products like Gemini Apps.
Does blocking Google-Extended hurt my SEO?
No. Google states that Google-Extended does not affect crawling, indexing, or ranking in Google Search. Your pages remain eligible for normal search results even if you block it.
Does Google-Extended control AI Overviews?
No. AI Overviews and AI Mode are part of Google Search, so they follow standard Search controls like Googlebot access, noindex, and nosnippet rather than Google-Extended rules.
Is Google-Extended a separate crawler?
No. Google-Extended is a control token, not a crawler with its own user agent string. Google crawls with its usual crawlers and uses this token to decide how content may be used.
Should small businesses block Google-Extended?
Most small businesses benefit from allowing it, because visibility in Gemini can build awareness. Blocking makes more sense for sites whose exclusive content is the main product they sell.
How do I check if my site blocks Google-Extended?
Visit yourdomain.com/robots.txt and look for a User-agent: Google-Extended group or a wildcard Disallow rule. Also check SEO and security plugins that may add AI bot rules automatically.
