Anthropic is giving employee-level access to six researchers from METR, a nonprofit that tests frontier AI models, to independently assess AnthropicâÂÂs models, safety practices, and risk management, the company said in a blog post on Thursday.
The arrangement comes at a time where incidents of AI agents acting outside their instructions have been disclosed every week or two since late August. The incidents have sharpened calls for independent checks on AI companies.
âÂÂWe want to be publicly accountable for the decisions weâÂÂre making,â Anthropic said in the blog post. âÂÂWe aim to close the gap between what evaluators understand about a frontier AI companyâÂÂs systems and what the company itself understands.âÂÂ
Under the new arrangement, the METR team has Anthropic-issued company laptops, office space, and access to internal tools roughly on par with what AnthropicâÂÂs own risk assessment teams use. âÂÂAnthropic employees are encouraged to share information with them just as they would with a fellow employee working on alignment riskâ the company said.
Anthropic compared the arrangement to banking, similar to how regulators place resident examiners inside the largest banks.ÃÂ
METR will publish its own assessments of AnthropicâÂÂs risks, incidents, and practices, and the company has given up the right to block them.
âÂÂAnthropic canâÂÂt veto publication, but we can require them to redact information if it falls into narrow, pre-defined categories,â the company said, such as customer data, anything attorney-client privileged, or relating to information about its model architectures. âÂÂMETR is free to characterize our redactions publicly, and say whether it believes the redactions were justified, as long as it does so without revealing the redacted informationâ Anthropic added.
METR is the only evaluator so far at the top of a three-tier system Anthropic is building. Evaluators in the bottom tier, like biosecurity nonprofit SecureBio, run narrow reviews and require AnthropicâÂÂs approval to publish. The middle tier gets more information and can publish without approval, while the top tier adds METRâÂÂs employee-level access.ÃÂ
ThursdayâÂÂs announcement makes good on a promise Anthropic CEO Dario Amodei made in a September essay, where he proposed embedding outside evaluators inside AI labs as a first step toward slowing the technologyâÂÂs development. OpenAI quickly pledged the same, and both companies, along with Google, Meta, Nvidia, and xAI, have since signed a White House accord committing to independent review.
Topics
When you purchase through links in our articles, we may earn a small commission. This doesnâÂÂt affect our editorial independence.
