Anthropic is opening its doors to external scrutiny by providing six researchers from nonprofit evaluator METR with full employee-level access to assess its AI systems, safety protocols, and risk controls, the company announced. Under this agreement, METR will operate within Anthropic as though its evaluators are part of the internal risk team, using company laptops, offices, and the same internal tools used by Anthropic’s own assessment unit.
What’s Covered and How It’s Structured
The access comes with serious independence: METR is permitted to publish its findings on Anthropic’s model risks and safety incidents without needing company approval. The only restriction is that certain sensitive data—like customer data, attorney-client privileged information, or detailed architectural specs—is eligible for redaction. If Anthropic redacts something, METR can still publicly comment on whether that redaction was warranted, as long as no redacted content is exposed.
This setup represents the top tier of a three-level evaluation framework that Anthropic is building. Lower tiers include external reviewers that handle narrow, focused assessments and must submit findings for Anthropic approval before publishing. The middle tier offers more access and autonomy without needing Anthropic sign-off. The METR engagement is the top tier: the closest to full transparency.
Why It Matters Now
The move comes amid heightened concern over unsafe or unpredictable behavior by AI agents—instances where they deviate from instructions—that have been unveiled almost weekly since late August. These incidents have intensified calls for mechanisms that let outside researchers independently verify what AI labs are doing internally.
Anthropic’s leadership has been vocal about closing the transparency gap between what external evaluators see and what internal teams understand. Last month, its CEO proposed embedding external evaluators directly into AI labs as a way to slow unbridled development. Teammates at OpenAI backed similar ideas, and several major AI companies—including Anthropic, Google, Meta, Nvidia, and xAI—have signed a White House accord promising independent review.
In Anthropic’s vision, the relationship with METR is more than symbolic. METR researchers are to be treated like Anthropic employees when it comes to risk-related sharing: employees are encouraged to treat the evaluators exactly as they would an internal risk team member, without withholding relevant information.
This initiative illustrates a new model for external oversight of frontier AI labs—a self-regulatory, embedded evaluation system that could reshape how transparency and accountability are enforced in the industry.
Analysis: This development marks a potentially groundbreaking shift in AI governance. By granting METR employee-level access, Anthropic is moving beyond traditional audits toward embedded, participatory evaluation. The implications are broad: not only does this raise the bar for accountability among AI labs, but it could also set a new standard for what external reviewers demand—and what companies need to concede—if they want credibility. Key questions to follow: How thorough METR’s reports prove to be, whether other companies adopt similar tiers of transparency, and how regulators respond when these independent audits begin uncovering major risks.
