GPTBot

OpenAI's crawler used for training ChatGPT models.

GPTBot is OpenAI's user agent for crawling the web to gather training data for ChatGPT and other OpenAI models. It respects robots.txt and can be allowed or disallowed per path.

GPTBot is distinct from OAI-SearchBot (the live ChatGPT Search crawler). Blocking GPTBot prevents your content from being included in future training data but does not remove you from ChatGPT Search results.

Related terms

OAI-SearchBot

OpenAI's crawler for ChatGPT Search, separate from GPTBot.

robots.txt

A text file telling crawlers which parts of a site they can and can't access.

ClaudeBot

Anthropic's crawler used for training Claude models.