AI bots · Training
Amazonbot
Used to improve Amazon products and services, and may be used to train Amazon AI models.
- Run by
- Amazon
- Type
- Training bot
- robots.txt token
Amazonbot- User-agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1) Chrome/W.X.Y.Z Safari/537.36- Honours robots.txt
- Yes. Amazon says its crawlers honour robots.txt user-agent, Allow and Disallow lines, but not Crawl-delay. If robots.txt cannot be fetched, Amazon says they behave as if it does not exist.
- Official docs
- developer.amazon.com
- Published IP list
- developer.amazon.com/amazonbot/ip-addresses/
Block it
User-agent: Amazonbot Disallow: /
Allow it
User-agent: Amazonbot Allow: /
A named group replaces the * group for this bot, so repeat any private-path Disallow lines in it. Firewalls (Cloudflare AI Crawl Control, Vercel AI Bots ruleset) can block it even when robots.txt allows it. See fixes per stack.
Other Amazon bots
- Amzn-SearchBot (search)
- Amzn-User (user fetch)
Other training bots
- GPTBot (OpenAI)
- ClaudeBot (Anthropic)
- Google-Extended (Google)
- Applebot-Extended (Apple)
- CCBot (Common Crawl)
- Meta-ExternalAgent (Meta)
- MistralAI-Training (Mistral AI)
- Bytespider (ByteDance)