What is Diffbot, and how do I block it?

Diffbot is Diffbot's search crawler. Search crawlers build the index a search or answer product draws on. From the operator's wording: general crawling for Diffbot Knowledge Graph and web search.

Diffbot does not publish a full user agent string for Diffbot. The robots.txt token to use is Diffbot.

Add this group to the robots.txt file at the root of your site. It asks Diffbot to stay out of every page.

To allow it explicitly, for example while a broader rule blocks other bots:

Diffbot: crawling allows sites 'to be discovered and cited from the Diffbot Knowledge Graph and web search services'. Effect of blocking on that: not stated beyond this.

Each other Diffbot user agent has its own robots.txt token, so a rule for Diffbot does not cover them. See the related crawlers below.

Yes. Diffbot docs: 'By default Diffbot's web crawls adhere to a site's robots.txt instructions, including the disallow and crawl-delay directives.'

Robots.txt is a request, not a lock (RFC 9309).

Not stated by the operator. Without a published list you cannot verify Diffbot by address, so a rule at your server cannot rely on one.