Anthropic taps nonprofits METR and SecureBio for independent AI safety evaluation

Anthropic announced on Thursday it is collaborating with the nonprofits METR (Model Evaluation and Threat Research) and SecureBio, and holding discussions with other organizations including Hugging Face, as part of its pledge to embed independent evaluators within the company. The AI lab committed to working closely with third-party evaluators when CEO Dario Amodei published a proposal last month to slow down the pace of model development. Amodei said the evaluators would help "verify" Anthropic's adherence to safety practices. The company selected Accenture as its first partner days after Amodei's proposal.

Independent evaluators like METR had occupied a quiet corner of the multi-trillion-dollar AI industry before Amodei's proposal, but concerns over whether AI companies can safeguard their models have pushed them into the spotlight. President Donald Trump in late September encouraged AI companies to "partner with an independent external auditor or evaluator" as part of a voluntary accord he presented. Amodei signed the accord, along with top executives from Google, Meta, OpenAI, SpaceX and Nvidia.

"We're embedding evaluators voluntarily, and we believe every frontier AI company should do the same," Anthropic said on Thursday.

How it's landing

clem 🤗 @ClementDelangue

Hugging Face CEO

We're sending a bunch of Microducks to assess @AnthropicAI! Kidding aside, it hasn't been approved yet but in my opinion, it's very important that open-source engineers and scientists are included in these important endeavors for transparency. That's a big part of why @Thom_Wolf is starting the open alignment initiative. We're all in this together!

16 Oct, 6:50 am View on X

Posts on X from people the desk follows. Their words, unedited.

Analysis

Why This Matters

The move signals a shift toward voluntary third-party oversight in frontier AI development, potentially setting a precedent for the industry.
It follows growing public concern about AI safety and a White House-backed voluntary accord that encourages independent auditing.
How seriously Anthropic and its peers implement these evaluations will determine whether voluntary measures suffice or if regulation becomes necessary.

Background

Anthropic, founded by former OpenAI researchers, has positioned itself as a safety-conscious AI company. CEO Dario Amodei's proposal to slow model development, published last month, included a commitment to independent evaluators. The Trump administration promoted a voluntary accord in late September that encouraged AI companies to work with external auditors. Anthropic's new partnerships with METR and SecureBio, along with discussions with Hugging Face, follow its earlier selection of Accenture as a partner.

Key Perspectives

Anthropic: The company argues that embedding evaluators voluntarily is the right approach and that every frontier AI lab should follow suit. The Trump administration: The White House has encouraged voluntary partnerships with independent evaluators but has not mandated them. Critics/Skeptics: Some observers question whether voluntary measures go far enough, especially given the rapid pace of AI development and the commercial incentives to release powerful models quickly.

What to Watch

Whether other frontier AI labs, such as OpenAI, Google and Meta, announce similar embedded evaluator partnerships.
The scope of METR and SecureBio's findings and whether they are made public.
Any push for mandatory independent auditing from lawmakers or regulators.