What is Applebot-Extended, and how do I block it?

Applebot-Extended is Apple's robots.txt control token. A control token does not fetch pages. It sets how pages another crawler already collected may be used. From the operator's wording: control token for Apple foundation models.

No separate UA string (it does not crawl; it controls use of data crawled by Applebot). The robots.txt token to use is Applebot-Extended. To tell whether a visit is real, compare the visitor's IP address with the published list.

Add this group to the robots.txt file at the root of your site. It asks Applebot-Extended to stay out of every page.

To allow it explicitly, for example while a broader rule blocks other bots:

Each Apple user agent has its own token, so you can set them one by one. This file blocks Applebot-Extended and allows Applebot. Every line comes from the rows in this dataset. Whether allowing a bot keeps you in a product is Apple's call, so check each crawler's page.

Apple: 'Webpages that disallow Applebot-Extended can still be included in search results.' It is only used to determine how crawled data is used.

Each other Apple user agent has its own robots.txt token, so a rule for Applebot-Extended does not cover them. See the related crawlers below.

Yes. Apple: 'In addition to following all robots.txt rules and directives, Apple has a secondary user agent, Applebot-Extended.'

This is a control token, so there is no fetching for it to obey. Googlebot or Applebot reads your robots.txt and applies the token.

Robots.txt is a request, not a lock (RFC 9309).

Apple publishes a list at https://search.developer.apple.com/applebot.json. We did not download or parse it. A request that claims to be Applebot-Extended but comes from outside that list is not Applebot-Extended.