AI Search Glossary
GPTBot
GPTBot is a web crawler operated by OpenAI that collects content from public websites to train ChatGPT and improve OpenAI's language models.
In detail
GPTBot uses the user-agent string "GPTBot" and follows the robots.txt standard. Sites can allow or block it using standard robots.txt rules. GPTBot is separate from OAI-SearchBot, which handles real-time ChatGPT Search. Blocking GPTBot prevents OpenAI from using your content as training data but does not affect Google Search rankings.
Example
User-agent: GPTBot
Allow: / — allows GPTBot to crawl all pagesUser-agent: GPTBot
Disallow: / — blocks GPTBot from all pagesSee how your site handles GPTBot.
Free AI search readiness check — no account required.
Related terms