Rattlesnakes By Mail

Applebot-Extended vs Claude-SearchBot

Applebot-Extended and Claude-SearchBot compared on 11 documented fields, from 17 claims, last verified 2026-09-13.

How many of the compared fields have the same answer for both crawlers, and how many differ.

Applebot-Extended and Claude-SearchBot, fields comparedApplebot-Extended and Claude-SearchBot, fields compared. Different is the largest with 10 fields, out of 11 fields in total.Same: 1 fieldSame1Different: 10 fieldsDifferent10
Applebot-Extended and Claude-SearchBot, fields compared. Different is the largest with 10 fields, out of 11 fields in total. Source: each vendor's own crawler documentation, quoted and dated on every claim page.
FieldApplebot-ExtendedClaude-SearchBotComparison
cites_sourcesnot documentednot_documenteddifferent
honours_crawl_delayn/ayesdifferent
publishes_ip_listnoyesdifferent
purpose_documentedpolicy_onlysearchdifferent
reads_llms_txtnot documentednot_documenteddifferent
respects_robots_txtn/ayesdifferent
robots_token_differs_from_uayesnot documenteddifferent
separate_search_and_training_tokensyesyessame
user_agent_string_fullnot documentednot_documenteddifferent
verifiable_by_rdnsn/anot_documenteddifferent
what_it_controlsAI training use of content, not search indexingnot documenteddifferent

Differences

FieldApplebot-Extended statementClaude-SearchBot statement
cites_sourcesnot documentedAnthropic does not publish whether Claude search responses cite pages crawled by Claude-SearchBot.
honours_crawl_delayApple does not publish a Crawl-delay behaviour for Applebot-Extended.Claude-SearchBot respects the Crawl-delay directive in robots.txt when crawling domains.
publishes_ip_listApple publishes no IP list specific to Applebot-Extended and documents one shared crawl identity for Applebot.Anthropic publishes the IP address ranges used by Claude-SearchBot as a JSON file at https://claude.com/crawling/bots.json.
purpose_documentedApplebot-Extended controls whether website content is used to train Apple general purpose foundation models.Claude-SearchBot analyses online content to improve the relevance and accuracy of Claude search responses.
reads_llms_txtnot documentedAnthropic does not publish whether Claude-SearchBot reads llms.txt files.
respects_robots_txtApplebot-Extended is a robots.txt token that publishers disallow to opt out of generative model training.Claude-SearchBot honours industry standard robots.txt directives that signal do not crawl.
robots_token_differs_from_uaApplebot-Extended is a robots.txt control token and does not correspond to a separate crawler user agent string.not documented
user_agent_string_fullnot documentedAnthropic does not publish a full user agent string for Claude-SearchBot.
verifiable_by_rdnsApple does not publish a reverse-DNS verification method specific to Applebot-Extended.Anthropic does not publish a reverse-DNS verification method for Claude-SearchBot.
what_it_controlsApplebot-Extended controls the AI training use of crawled content and does not control search indexing.not documented

Sources