GPTBot
OpenAI · Trains AI models
What OpenAI says it does
“crawl content that may be used in training our generative AI foundation models”
The facts
| Operator | OpenAI |
| robots.txt token | GPTBot |
| Appears in access logs | Yes |
| User-agent string | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot |
| robots.txt | Honors robots.txt. OpenAI states that disallowing GPTBot signals the site's content should not be used for model training. |
| IP list | https://openai.com/gptbot.json |
| Documentation | https://developers.openai.com/api/docs/bots, read 4 Aug 2026 |
Allow or block it in robots.txt
To block GPTBot entirely:
User-agent: GPTBot Disallow: /
To allow it everywhere:
User-agent: GPTBot Allow: /
Verify a request is really GPTBot
Match the requesting IP against the published list.
Observed on this property
| First seen here | 4 Aug 2026 |
| Most recent | 4 Aug 2026 |
| Requests, last 30 days | 2007 |
| Requests, all time | 2007 |
| robots.txt fetches | 0 |
Most requested paths:
/api/tools(14)/(1)/about/(1)/about/index.md(1)/agents.md(1)/ai-visibility-tools-under-100/(1)/ai-visibility-tools-under-100/index.md(1)/alternatives/ahrefs-brand-radar/(1)
Also run by OpenAI
- ChatGPT-User | User-requested fetches
- OAI-SearchBot | AI search index