AI bots · Search
OAI-SearchBot
Surfaces websites in ChatGPT search results. Sites that block it will not be shown in ChatGPT search answers, though they can still appear as navigational links.
- Run by
- OpenAI
- Type
- Search bot
- robots.txt token
OAI-SearchBot- User-agent string
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot- Honours robots.txt
- Yes. OpenAI says robots.txt changes can take about 24 hours to apply.
- Official docs
- platform.openai.com
- Published IP list
- openai.com/searchbot.json
Block it
User-agent: OAI-SearchBot Disallow: /
Allow it
User-agent: OAI-SearchBot Allow: /
A named group replaces the * group for this bot, so repeat any private-path Disallow lines in it. Firewalls (Cloudflare AI Crawl Control, Vercel AI Bots ruleset) can block it even when robots.txt allows it. See fixes per stack.
Other OpenAI bots
- GPTBot (training)
- ChatGPT-User (user fetch)
Other search bots
- Claude-SearchBot (Anthropic)
- PerplexityBot (Perplexity)
- Applebot (Apple)
- Amzn-SearchBot (Amazon)
- MistralAI-Index (Mistral AI)
- DuckAssistBot (DuckDuckGo)
- Meta-WebIndexer (Meta)