Rattlesnakes By Mail

GoogleOther vs GPTBot

GoogleOther and GPTBot compared on 11 documented fields, from 17 claims, last verified 2026-09-15.

How many of the compared fields have the same answer for both crawlers, and how many differ.

GoogleOther and GPTBot, fields comparedGoogleOther and GPTBot, fields compared. Different is the largest with 9 fields, out of 11 fields in total.Same: 2 fieldsSame2Different: 9 fieldsDifferent9
GoogleOther and GPTBot, fields compared. Different is the largest with 9 fields, out of 11 fields in total. Source: each vendor's own crawler documentation, quoted and dated on every claim page.
FieldGoogleOtherGPTBotComparison
honours_crawl_delaynonot_documenteddifferent
observed_fetches_robots_txtnonot documenteddifferent
observed_requested_mcp_endpointyesnot documenteddifferent
publishes_ip_listyesyessame
purpose_documentedmixedtrainingdifferent
respects_robots_txtyesyessame
robots_txt_fetch_markernot documentedyesdifferent
robots_txt_tokenGoogleOthernot documenteddifferent
separate_search_and_training_tokensnot_documentedyesdifferent
user_agent_string_fullnot documentedMozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbotdifferent
verifiable_by_rdnsyesnot_documenteddifferent

Differences

FieldGoogleOther statementGPTBot statement
honours_crawl_delayGoogle does not support the robots.txt Crawl-delay directive for GoogleOther.OpenAI does not publish whether GPTBot honours the robots.txt Crawl-delay directive.
observed_fetches_robots_txtGoogleOther made 93 requests to Rattlesnakes By Mail between 2026-09-13T20:00Z and 2026-09-15T09:00Z and fetched /robots.txt on none of those requests. GoogleOther's first request to Rattlesnakes By Mail was the home page at 2026-09-14T12:03:30Z.not documented
observed_requested_mcp_endpointGoogleOther requested the /mcp endpoint of Rattlesnakes By Mail twice between 2026-09-13T20:00Z and 2026-09-15T09:00Z, at 2026-09-14T15:07:30Z and at 2026-09-14T18:27:49Z. Rattlesnakes By Mail returned status 405 to both GoogleOther requests for /mcp.not documented
purpose_documentedGoogleOther is a generic crawler that Google product teams use to fetch publicly accessible content from sites.GPTBot crawls content that may be used to train OpenAI generative AI foundation models.
robots_txt_fetch_markernot documentedOpenAI may add a robots.txt marker to the GPTBot user agent string when GPTBot fetches robots.txt files.
robots_txt_tokenThe robots.txt token for GoogleOther is GoogleOther.not documented
separate_search_and_training_tokensGoogle does not document whether content fetched by GoogleOther is used for Gemini training.OpenAI separates search and training controls into two robots.txt tokens: OAI-SearchBot for search and GPTBot for training.
user_agent_string_fullnot documentedGPTBot sends the user agent string Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot.
verifiable_by_rdnsGoogleOther requests are verifiable by reverse DNS lookup resolving to googlebot.com, google.com, or googleusercontent.com.OpenAI does not publish a reverse-DNS verification method for GPTBot.

Sources