Meta-ExternalAgent
Meta · Trains AI models; indexing
What Meta says it does
“crawls the web for use cases such as training foundation AI models or improving products by indexing content”
The facts
| Operator | Meta |
| robots.txt token | meta-externalagent |
| Appears in access logs | Yes |
| User-agent string | meta-externalagent/1.1 |
| robots.txt | Controlled via robots.txt. Meta's crawler page documents the meta-externalagent robots.txt token and notes no bypass conditions for it, unlike its user-triggered sibling Meta-ExternalFetcher. |
| IP list | Not published |
| Documentation | https://developers.facebook.com/docs/sharing/webmasters/web-crawlers, read 4 Aug 2026 |
Allow or block it in robots.txt
To block meta-externalagent entirely:
User-agent: meta-externalagent Disallow: /
To allow it everywhere:
User-agent: meta-externalagent Allow: /
Verify a request is really Meta-ExternalAgent
Meta publishes no IP list for it on the crawler page.
Observed on this property
| First seen here | 15 Aug 2026 |
| Most recent | 2 Nov 2026 |
| Requests, last 30 days | 22 |
| Requests, all time | 3400 |
| robots.txt fetches | 0 |
Most requested paths:
/api/tools(22)/tracks/claude/(5)/tool/ahrefs-brand-radar/(4)/tool/amicited/(4)/tool/brightedge/(4)/tool/checkthat/(4)/tool/rankability/(4)/tools/(4)
Also run by Meta
- Meta-ExternalFetcher | User-requested fetches