Meta-ExternalAgent
Meta · Trains AI models; indexing
What Meta says it does
“crawls the web for use cases such as training foundation AI models or improving products by indexing content”
The facts
| Operator | Meta |
| robots.txt token | meta-externalagent |
| Appears in access logs | Yes |
| User-agent string | meta-externalagent/1.1 |
| robots.txt | Controlled via robots.txt. Meta's crawler page documents the meta-externalagent robots.txt token and notes no bypass conditions for it, unlike its user-triggered sibling Meta-ExternalFetcher. |
| IP list | Not published |
| Documentation | https://developers.facebook.com/docs/sharing/webmasters/web-crawlers, read 4 Aug 2026 |
Allow or block it in robots.txt
To block meta-externalagent entirely:
User-agent: meta-externalagent Disallow: /
To allow it everywhere:
User-agent: meta-externalagent Allow: /
Verify a request is really Meta-ExternalAgent
Meta publishes no IP list for it on the crawler page.
Observed on this property
| First seen here | 15 Aug 2026 |
| Most recent | 17 Sep 2026 |
| Requests, last 30 days | 88 |
| Requests, all time | 3107 |
| robots.txt fetches | 0 |
Most requested paths:
/api/tools(19)/tool/ahrefs-brand-radar/(4)/tools/(4)/about/(3)/ai-visibility-tools-under-100/(3)/compare/demandsphere-vs-frase/(3)/compare/gauge-vs-promptmonitor/(3)/compare/gauge-vs-rankscale/(3)
Also run by Meta
- Meta-ExternalFetcher | User-requested fetches