AI Search Glossary
ClaudeBot
ClaudeBot is a web crawler operated by Anthropic that collects web content to help train and update Claude, Anthropic's AI assistant.
In detail
ClaudeBot uses the user-agent string "ClaudeBot" and follows the robots.txt standard. Allowing ClaudeBot increases the chance that Claude has accurate, up-to-date knowledge about your site. Blocking ClaudeBot means Anthropic cannot include your content in Claude's knowledge base.
See how your site handles ClaudeBot.
Free AI search readiness check — no account required.
Related terms