Home AI crawlers

Meta / User fetch

What is Meta-ExternalFetcher, and how do I block it?

Meta-ExternalFetcher is Meta's user-triggered fetcher. The two block lines below are valid, but Meta says it may not follow robots.txt, so stopping it takes a server or CDN rule. Checked October 29, 2026.

What is Meta-ExternalFetcher?

Meta-ExternalFetcher is Meta's user-triggered fetcher. A user-triggered fetcher runs when a person asks an AI product to open a page, not on a schedule.

OperatorMeta
Purpose (operator wording)user-triggered fetch
robots.txt tokenMeta-ExternalFetcher
User agent stringmeta-externalfetcher/1.1 (+/documentation/sharing/webmasters/web-crawlers) (as printed on the doc page; the link is relative) ; also meta-externalfetcher/1.1
Respects robots.txtNo (operator: may bypass robots.txt)
IP rangesNot stated by the operator
Last verified

What else should I know about Meta-ExternalFetcher?

What is the Meta-ExternalFetcher user agent string?

This is the user agent string Meta documents for Meta-ExternalFetcher, as of October 29, 2026. Version numbers and details can change, so match on the token in robots.txt, not on the whole string.

Meta-ExternalFetcher user agent string
meta-externalfetcher/1.1 (+/documentation/sharing/webmasters/web-crawlers) (as printed on the doc page; the link is relative) ; also meta-externalfetcher/1.1

How do I block Meta-ExternalFetcher in robots.txt?

Add this group to the robots.txt file at the root of your site. It asks Meta-ExternalFetcher to stay out of every page. Meta says the crawler may not follow robots.txt, so this may not stop it.

Block Meta-ExternalFetcher
User-agent: meta-externalfetcher
Disallow: /

To allow it explicitly, for example while a broader rule blocks other bots:

Allow Meta-ExternalFetcher
User-agent: meta-externalfetcher
Allow: /

How do I opt out of Meta training but stay in its search?

Each Meta user agent has its own token, so you can set them one by one. This file blocks Meta-ExternalAgent and allows Meta-ExternalFetcher, Meta-WebIndexer. Every line comes from the rows in this dataset. Whether allowing a bot keeps you in a product is Meta's call, so check each crawler's page. Meta says Meta-ExternalFetcher may not follow robots.txt.

Meta: block training, allow search and user fetch
User-agent: meta-externalagent
Disallow: /

User-agent: meta-externalfetcher
Allow: /

User-agent: meta-webindexer
Allow: /

Does blocking Meta-ExternalFetcher affect AI answers?

Meta does not say what blocking Meta-ExternalFetcher changes. That is not stated.

Each other Meta user agent has its own robots.txt token, so a rule for Meta-ExternalFetcher does not cover them. See the related crawlers below.

Does Meta-ExternalFetcher follow robots.txt?

No. Operator: may bypass robots.txt. Meta: 'this crawler may bypass robots.txt rules' / 'may bypass robots.txt because it performs fetches that were requested by the user.'

Robots.txt is a request, not a lock (RFC 9309).

What IP addresses does Meta-ExternalFetcher use?

Not stated by the operator. Without a published list you cannot verify Meta-ExternalFetcher by address, so a rule at your server cannot rely on one.

Which other Meta crawlers are there?

What should I read next?

Next: give AI tools something to quote

Paperboy writes articles that answer the questions your buyers ask, with every fact linked to its source, FAQ and Article structured data, and publishes them to your site every week. Your first article is free.