AI bots · Training
MistralAI-Training
Crawls web content to build datasets for training Mistral's generative models.
- Run by
- Mistral AI
- Type
- Training bot
- robots.txt token
MistralAI-Training- User-agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; MistralAI-Training/1.0; +https://docs.mistral.ai/robots)- Honours robots.txt
- Yes. Mistral says webmasters can disallow it in robots.txt.
- Official docs
- docs.mistral.ai
Block it
User-agent: MistralAI-Training Disallow: /
Allow it
User-agent: MistralAI-Training Allow: /
A named group replaces the * group for this bot, so repeat any private-path Disallow lines in it. Firewalls (Cloudflare AI Crawl Control, Vercel AI Bots ruleset) can block it even when robots.txt allows it. See fixes per stack.
Other Mistral AI bots
- MistralAI-Index (search)
- MistralAI-User (user fetch)
Other training bots
- GPTBot (OpenAI)
- ClaudeBot (Anthropic)
- Google-Extended (Google)
- Applebot-Extended (Apple)
- CCBot (Common Crawl)
- Meta-ExternalAgent (Meta)
- Amazonbot (Amazon)
- Bytespider (ByteDance)