Skip to content

Topic

Third-party AI audits

Independent review of AI systems and companies by evaluators outside the developer, including safety testing, red-teaming, and compliance checks.

Current stories

leadershipConfirmed9 publishers

Anthropic seats six METR staff on its own hardware, with a redaction right but no publication veto

Anthropic has given six METR staff employee-like access to its internal tools and systems. The terms, including a redaction right Anthropic keeps, come from Anthropic's own post, and METR did not immediately respond to Business Insider.

Perspective Coverage

9 publishers
Builder
Builder 16%
Operator
Operator 49%
Investor
Investor 35%

Reality

Evidence68
Adoption15
Hype gap+15
Incentives60
Confidence72
buildConfirmed4 publishers

Anthropic plans to put outside evaluators inside its labs, starting with Accenture

Anthropic named Accenture and its Faculty unit as embedded evaluators, backed by at least $1 billion in joint investment over five years. Employee-like access could make third-party findings on Claude more informative, though METR's role is still a planned pilot.

Perspective Coverage

4 publishers
Builder
Builder 20%
Operator
Operator 52%
Investor
Investor 28%

Reality

Evidence58
Adoption30
Hype gap+20
Incentives65
Confidence60
investConfirmed5 publishers

Newsom's kill switch order accelerates California's AI auditor registry by 13 months

The executive order signed on September 18 gives a panel of experts two months to say what an AI emergency stop should be, and the firm date inside it moves the state's audit registry from January 2029 to December 2027.

Perspective Coverage

5 publishers
Builder
Builder 27%
Operator
Operator 33%
Investor
Investor 40%

Reality

Evidence70
Adoption18
Hype gap+45
Incentives68
Confidence72
productOne report1 publisher

Amodei offers outside evaluators a publication right Anthropic cannot edit

Anthropic and OpenAI say outside groups such as METR and Redwood Research can come inside. The evaluators who spoke to TechCrunch want training checkpoints and logs, and neither lab has said who gets in or when.

Publishers:techcrunch.com

Reality

Evidence58
Adoption18
Hype gap+32
Incentives72
Confidence55
investOne report1 publisher

OpenAI endorses the FRONTIER Act provision that puts licensed verifiers inside frontier labs

The FRONTIER Act would license independent verifiers to inspect the largest frontier labs, and OpenAI has endorsed that provision days after Sam Altman promised the same access voluntarily. The bill's heaviest duties start far above startup scale.

Reality

Evidence42
Adoption
Not scored
Hype gap+28
Incentives72
Confidence44
productOne report1 publisher

Amodei's call to "pace" AI development rattles chip stocks and OpenAI's IPO plans

Dario Amodei's call to slow AI development sold off the chipmakers on Monday and hit an OpenAI shareholder harder still, while analysts said the models already serving customers will keep serving them.

Publishers:latimes.com

Reality

Evidence38
Adoption30
Hype gap+35
Incentives74
Confidence42
investOne report1 publisher

Amodei asks the frontier labs to seat permanent outside reviewers inside their risk processes

Days after Anthropic's threat intelligence report and a researcher's resignation, Dario Amodei asked AI companies to moderate capability gains and to put permanent third-party reviewers inside frontier labs.

Publishers:livemint.com

Reality

Evidence45
Adoption15
Hype gap+25
Incentives70
Confidence50
productOne report1 publisher

The next tier of AI audit money is priced off the valuations it exists to check

Grant money buys the only independent look inside frontier labs today. The projection for the next round of that money rises and falls with the valuations of the companies being looked at.

Publishers:thenextweb.com

Reality

Evidence48
Adoption40
Hype gap+27
Incentives76
Confidence45