# Claude-SearchBot

Anthropic's crawler: what it does, robots.txt handling per the operator, and observed behaviour on this property.

- Operator: Anthropic
- Purpose: AI search index | operator's words: "Claude-SearchBot navigates the web to improve search result quality for users. It analyzes online content specifically to enhance the relevance and accuracy of search responses."
- robots.txt token: `Claude-SearchBot`
- Appears in access logs: yes
- User-agent: not published by the operator
- robots.txt: Honors robots.txt. Anthropic states its crawlers respect 'do not crawl' signals by honoring industry standard directives in robots.txt.
- IP list: https://claude.com/crawling/bots.json
- Verification: Match the requesting IP against the published list, shared across Anthropic's three agents.
- Documentation: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (read 2026-08-04)

Block it:
```
User-agent: Claude-SearchBot
Disallow: /
```

Observed on this property (logged since 2026-08-04):
- Not observed here yet.

---

Source: https://agentvisibilitytools.com/crawlers/claude-searchbot/ · HTML version of this page.
Full index: https://agentvisibilitytools.com/agents.md · JSON: https://agentvisibilitytools.com/api/tools
Free to quote with attribution to The Agent Visibility Directory, https://agentvisibilitytools.com.