Rattlesnakes By Mail

ClaudeBot vs Googlebot

ClaudeBot and Googlebot compared on 15 documented fields, from 22 claims, last verified 2026-09-15.

How many of the compared fields have the same answer for both crawlers, and how many differ.

ClaudeBot and Googlebot, fields comparedClaudeBot and Googlebot, fields compared. Different is the largest with 12 fields, out of 15 fields in total.Same: 3 fieldsSame3Different: 12 fieldsDifferent12
ClaudeBot and Googlebot, fields compared. Different is the largest with 12 fields, out of 15 fields in total. Source: each vendor's own crawler documentation, quoted and dated on every claim page.
FieldClaudeBotGooglebotComparison
anti_circumvention_stancedocumentednot documenteddifferent
cites_sourcesnot documentedyesdifferent
honours_crawl_delayyesnodifferent
observed_fetches_jsonyesnot documenteddifferent
observed_fetches_markdownyesnot documenteddifferent
observed_first_request_path/sitemap.xmlnot documenteddifferent
observed_format_pass_orderhtml_json_mdnot documenteddifferent
observed_verified_request_ratio_censusnot documented0.991different
publishes_ip_listyesyessame
purpose_documentedtrainingsearchdifferent
reads_llms_txtnot documentednodifferent
respects_robots_txtyesyessame
separate_search_and_training_tokensyesyessame
user_agent_string_fullnot_documentedMozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html)different
verifiable_by_rdnsnot_documentedyesdifferent

Differences

FieldClaudeBot statementGooglebot statement
anti_circumvention_stanceClaudeBot respects anti-circumvention technologies and does not bypass CAPTCHAs on crawled sites.not documented
cites_sourcesnot documentedGoogle AI Overviews and AI Mode return AI generated responses with links to supporting websites.
honours_crawl_delayAnthropic supports the non-standard robots.txt Crawl-delay extension for ClaudeBot.Google does not support the robots.txt Crawl-delay directive for Googlebot.
observed_fetches_jsonClaudeBot made 177 JSON requests to Rattlesnakes By Mail between 2026-09-13T20:00Z and 2026-09-15T09:00Z, covering the .json twin of 170 base paths. The .json URLs of Rattlesnakes By Mail appear in no entry of the 171 URL sitemap.not documented
observed_fetches_markdownClaudeBot fetched the .md twin of 170 base paths on Rattlesnakes By Mail between 2026-09-13T20:00Z and 2026-09-15T09:00Z, 170 Markdown requests of 542 ClaudeBot requests in that window. The .md URLs of Rattlesnakes By Mail appear in no entry of the 171 URL sitemap.not documented
observed_first_request_pathClaudeBot's first request to Rattlesnakes By Mail between 2026-09-13T20:00Z and 2026-09-15T09:00Z was /sitemap.xml at 2026-09-14T19:53:45Z. ClaudeBot fetched /robots.txt on the second request to Rattlesnakes By Mail.not documented
observed_format_pass_orderClaudeBot fetched Rattlesnakes By Mail in three format passes between 2026-09-13T20:00Z and 2026-09-15T09:00Z, HTML from 2026-09-14T20:52Z to 2026-09-14T20:57Z, then JSON from 2026-09-14T21:37Z to 2026-09-14T21:56Z, then Markdown from 2026-09-14T21:37Z to 2026-09-14T21:57Z. The same order holds on 170 of 170 base paths that ClaudeBot fetched in all three formats on Rattlesnakes By Mail.not documented
observed_verified_request_ratio_censusnot documentedGooglebot requests to Rattlesnakes By Mail matched Google's published IP ranges on 108 of 109 requests between 2026-09-13T20:00Z and 2026-09-15T09:00Z. Googlebot's first request to Rattlesnakes By Mail in that window was /robots.txt at 2026-09-14T11:53:00Z.
purpose_documentedClaudeBot collects web content for training Anthropic generative AI models.Googlebot crawls pages for Google Search, Google Images, Google Video, Google News, and Discover.
reads_llms_txtnot documentedGoogle Search does not use llms.txt files.
user_agent_string_fullAnthropic does not publish a full user agent string for ClaudeBot.Googlebot sends the user agent string Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html).
verifiable_by_rdnsAnthropic does not publish a reverse-DNS verification method for ClaudeBot.Googlebot requests are verifiable by reverse DNS lookup resolving to googlebot.com, google.com, or googleusercontent.com.

Sources