Rattlesnakes By Mail

respects robots txt across crawlers

respects robots txt across 15 crawlers from 6 vendors, from 15 claims, last verified 2026-09-14.

How the crawlers answer this one question, counted by answer. Every answer links to the vendor quote behind it.

Respects robots txtRespects robots txt. yes is the largest with 10 crawlers, out of 15 crawlers in total.yes: 10 crawlersyes10n/a: 2 crawlersn/a2partial: 1 crawlerpartial1not_documented: 1 crawlernot_documented1no: 1 crawlerno1
Respects robots txt. yes is the largest with 10 crawlers, out of 15 crawlers in total. Source: each vendor's own crawler documentation, quoted and dated on every claim page.
CrawlerVendorValueMethodVerifiedClaim
ApplebotAppleyesvendor_doc2026-09-13claim 9
Applebot-ExtendedApplen/avendor_doc2026-09-13claim 1
BingbotMicrosoftyesvendor_doc2026-09-14claim 121
ChatGPT-UserOpenAIpartialvendor_doc2026-09-13claim 19
Claude-SearchBotAnthropicyesvendor_doc2026-09-13claim 28
Claude-UserAnthropicyesvendor_doc2026-09-13claim 37
ClaudeBotAnthropicyesvendor_doc2026-09-13claim 47
Google-ExtendedGooglen/avendor_doc2026-09-13claim 55
GooglebotGoogleyesvendor_doc2026-09-13claim 63
GoogleOtherGoogleyesvendor_doc2026-09-13claim 71
GPTBotOpenAIyesvendor_doc2026-09-13claim 78
OAI-AdsBotOpenAInot_documentedvendor_doc2026-09-13claim 86
OAI-SearchBotOpenAIyesvendor_doc2026-09-13claim 94
Perplexity-UserPerplexitynovendor_doc2026-09-13claim 103
PerplexityBotPerplexityyesvendor_doc2026-09-13claim 112

Not documented

Every documented crawler has a claim for this field.

Sources

Source
https://support.apple.com/en-us/119829
https://developers.openai.com/api/docs/bots
https://support.claude.com/en/articles/8896518-does-anthropic-crawl-content-from-the-web-and-how-can-site-owners-block-the-crawler
https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers
https://docs.perplexity.ai/docs/resources/perplexity-crawlers
https://www.bing.com/webmasters/help/ai-performance-9f8e7d6c