# Claude-User

Anthropic's crawler: what it does, robots.txt handling per the operator, and observed behaviour on this property.

- Operator: Anthropic
- Purpose: User-requested fetches | operator's words: "Claude-User supports Claude AI users. When individuals ask questions to Claude, it may access websites using a Claude-User agent."
- robots.txt token: `Claude-User`
- Appears in access logs: yes
- User-agent: not published by the operator
- robots.txt: Honors robots.txt. Anthropic states its crawlers respect 'do not crawl' signals by honoring industry standard directives in robots.txt. Unlike OpenAI and Perplexity, it claims this for its user-triggered agent too.
- IP list: https://claude.com/crawling/bots.json
- Verification: Match the requesting IP against the published list, shared across Anthropic's three agents.
- Documentation: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (read 2026-08-04)
- Note: A hit from Claude-User means a person asked Claude something and it opened this page to answer.

Block it:
```
User-agent: Claude-User
Disallow: /
```

Observed on this property (logged since 2026-08-04):
- Not observed here yet.

---

Source: https://agentvisibilitytools.com/crawlers/claude-user/ · HTML version of this page.
Full index: https://agentvisibilitytools.com/agents.md · JSON: https://agentvisibilitytools.com/api/tools
Free to quote with attribution to The Agent Visibility Directory, https://agentvisibilitytools.com.