Rattlesnakes By Mail

ClaudeBot vs GPTBot

ClaudeBot and GPTBot compared on 13 documented fields, from 20 claims, last verified 2026-09-15.

How many of the compared fields have the same answer for both crawlers, and how many differ.

ClaudeBot and GPTBot, fields comparedClaudeBot and GPTBot, fields compared. Different is the largest with 8 fields, out of 13 fields in total.Same: 5 fieldsSame5Different: 8 fieldsDifferent8
ClaudeBot and GPTBot, fields compared. Different is the largest with 8 fields, out of 13 fields in total. Source: each vendor's own crawler documentation, quoted and dated on every claim page.
FieldClaudeBotGPTBotComparison
anti_circumvention_stancedocumentednot documenteddifferent
honours_crawl_delayyesnot_documenteddifferent
observed_fetches_jsonyesnot documenteddifferent
observed_fetches_markdownyesnot documenteddifferent
observed_first_request_path/sitemap.xmlnot documenteddifferent
observed_format_pass_orderhtml_json_mdnot documenteddifferent
publishes_ip_listyesyessame
purpose_documentedtrainingtrainingsame
respects_robots_txtyesyessame
robots_txt_fetch_markernot documentedyesdifferent
separate_search_and_training_tokensyesyessame
user_agent_string_fullnot_documentedMozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbotdifferent
verifiable_by_rdnsnot_documentednot_documentedsame

Differences

FieldClaudeBot statementGPTBot statement
anti_circumvention_stanceClaudeBot respects anti-circumvention technologies and does not bypass CAPTCHAs on crawled sites.not documented
honours_crawl_delayAnthropic supports the non-standard robots.txt Crawl-delay extension for ClaudeBot.OpenAI does not publish whether GPTBot honours the robots.txt Crawl-delay directive.
observed_fetches_jsonClaudeBot made 177 JSON requests to Rattlesnakes By Mail between 2026-09-13T20:00Z and 2026-09-15T09:00Z, covering the .json twin of 170 base paths. The .json URLs of Rattlesnakes By Mail appear in no entry of the 171 URL sitemap.not documented
observed_fetches_markdownClaudeBot fetched the .md twin of 170 base paths on Rattlesnakes By Mail between 2026-09-13T20:00Z and 2026-09-15T09:00Z, 170 Markdown requests of 542 ClaudeBot requests in that window. The .md URLs of Rattlesnakes By Mail appear in no entry of the 171 URL sitemap.not documented
observed_first_request_pathClaudeBot's first request to Rattlesnakes By Mail between 2026-09-13T20:00Z and 2026-09-15T09:00Z was /sitemap.xml at 2026-09-14T19:53:45Z. ClaudeBot fetched /robots.txt on the second request to Rattlesnakes By Mail.not documented
observed_format_pass_orderClaudeBot fetched Rattlesnakes By Mail in three format passes between 2026-09-13T20:00Z and 2026-09-15T09:00Z, HTML from 2026-09-14T20:52Z to 2026-09-14T20:57Z, then JSON from 2026-09-14T21:37Z to 2026-09-14T21:56Z, then Markdown from 2026-09-14T21:37Z to 2026-09-14T21:57Z. The same order holds on 170 of 170 base paths that ClaudeBot fetched in all three formats on Rattlesnakes By Mail.not documented
robots_txt_fetch_markernot documentedOpenAI may add a robots.txt marker to the GPTBot user agent string when GPTBot fetches robots.txt files.
user_agent_string_fullAnthropic does not publish a full user agent string for ClaudeBot.GPTBot sends the user agent string Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot.

Sources