OAI-SearchBot, GPTBot and ChatGPT-User: which should your website allow?

Three names do not mean one shared index

According to OpenAI’s documentation, allowing the search crawler and restricting the training bot are compatible choices. You therefore do not need to allow training merely to be eligible to appear in search. ChatGPT-User, in turn, is not an automatic search crawl: it may request a page while handling a question or another user task.

It is useful to log the client name, time, URL and how the request’s origin was verified. A User-Agent string alone is not enough: an unrelated script can reproduce it. A user reading a page through an assistant does not prove that the page has become a source for future customers.

How to choose access rules

First define your own policy: what is allowed for public search, what is acceptable for training and which private sections must remain closed. Then check robots.txt for the relevant paths and your CDN rules. Do not copy a broad list of Allow directives without understanding the existing groups.

OpenAI’s official IP lists help verify real requests. Use the current documentation rather than an old post. The roughly one-day interval mentioned by OpenAI concerns a robots.txt change taking effect, not a deadline for indexing or receiving a recommendation. Save the edit time separately from the subsequent observation time.

Practical verification and the limits of the result

In AURA GEO, the agent launches a technical check and retrieves results for accessible URLs. This helps spot a policy error before writing new content. If a change is needed, the agent prepares it for your repository or hosting environment when separate access is available.

After deployment, repeat the check on the relevant page. Review actual visits separately in server observations. Auri helps assemble the task brief, but its name should not replace the provider actually used in a report.

An access rule removes a specific technical obstacle. It does not confirm that a source was discovered, selected for a question or recommended. The setup outcome is a clear policy and a verified server response. The next substantive step is to examine real answers to customer questions, rather than add more bot names without a defined purpose.