Skip to content

Topic

Third-Party AI Evaluation and Audit

Outside review of frontier labs, from public-evidence scorecards to evaluators granted access to internal models and nonpublic information.

Current stories

buildConfirmed17 publishers

OpenAI fired three safety researchers over a 'breach of trust' it has not described

OpenAI says it fired three safety researchers for mishandling sensitive information, citing a further breach of trust it has not described. For staff working with outside evaluators, the sharing line stays unsettled until OpenAI publishes the assessor contracts it says are coming.

Perspective Coverage

18 publishers
Builder
Builder 29%
Operator
Operator 43%
Investor
Investor 28%

Reality

Evidence55
Adoption
Not scored
Hype gap+15
Incentives72
Confidence60
investConfirmed11 publishers

OpenAI endorses the audit plan of three safety researchers it fired for what it calls leaks

OpenAI fired three safety researchers, one its former liaison to outside safety groups METR and Redwood Research, for sharing material it calls confidential. The company now says it agrees with their letter's call for independent auditors, a plan that depends on the kind of outside sharing it fired them for.

Perspective Coverage

12 publishers
Builder
Builder 32%
Operator
Operator 43%
Investor
Investor 25%

Reality

Evidence62
Adoption
Not scored
Hype gap+15
Incentives70
Confidence60
leadershipConfirmed8 publishers

Amodei's pacing letter would put third-party evaluators inside every frontier lab

Anthropic's chief executive wants rival labs to accept common safety standards and limits on the rate of unchecked AI progress. Altman, Musk, Hassabis and Nadella have said they support the letter.

Perspective Coverage

8 publishers
Builder
Builder 25%
Operator
Operator 56%
Investor
Investor 19%

Reality

Evidence72
Adoption18
Hype gap+35
Incentives70
Confidence62
investConfirmed23 publishers

Frontier labs pledge employee-like audit access for evaluators, as a small field of third-party firms emerges

Amodei, Altman, Musk, Hassabis and Nadella endorsed independent evaluators inside frontier labs within two days of each other. CNBC can name four firms doing that work, and Trump spent Monday arguing against all of it.

Perspective Coverage

25 publishers
Builder
Builder 30%
Operator
Operator 32%
Investor
Investor 38%

Reality

Evidence72
Adoption25
Hype gap+30
Incentives70
Confidence65
buildConfirmed9 publishers

Amodei commits Anthropic to an embedded evaluator that can publish what it finds

The one step Anthropic is adopting on its own binds it to access and disclosure. His three-step framework for pacing frontier development leaves the capability ceiling and the enforcement mechanism undefined.

Perspective Coverage

9 publishers
Builder
Builder 27%
Operator
Operator 44%
Investor
Investor 29%

Reality

Evidence68
Adoption18
Hype gap+35
Incentives70
Confidence62
buildConfirmed6 publishers

Ruling out 2026 keeps OpenAI's infrastructure obligations out of a prospectus

OpenAI submitted a draft S-1 confidentially in June, and Sam Altman has now told Fortune the safety work is unfinished, so for another year the figures on OpenAI are the ones OpenAI reports about itself.

Perspective Coverage

6 publishers
Builder
Builder 23%
Operator
Operator 39%
Investor
Investor 38%

Reality

Evidence68
Adoption55
Hype gap+15
Incentives62
Confidence64
leadershipConfirmed6 publishers

FTC's rogue-agent probe extends to the group OpenAI and Anthropic used to investigate agent incidents

FTC confirmed Wednesday it is investigating Anthropic, OpenAI and other AI labs, the first US enforcement action aimed at rogue AI agents. Its reported plan to question Metr, which investigated the labs' agent incidents, puts outside incident reviews within the regulator's reach.

Perspective Coverage

7 publishers
Builder
Builder 34%
Operator
Operator 37%
Investor
Investor 29%

Reality

Evidence72
Adoption
Not scored
Hype gap+15
Incentives45
Confidence68
productConfirmed16 publishers

Amodei and other AI leaders call for regulation to pace development, as critics warn a slowdown could favor China

Amodei, Altman, Musk and Nadella have all called for pacing frontier development, and Trump's team told the labs to do it themselves. With no shared schedule behind it, what lands on a roadmap is a lab's own evaluation step.

Perspective Coverage

16 publishers
Builder
Builder 29%
Operator
Operator 46%
Investor
Investor 25%

Reality

Evidence70
Adoption20
Hype gap+35
Incentives70
Confidence60
productOne report1 publisher

Huang answers Anthropic's slowdown plan with the reliability laws already on the books

Dario Amodei wants governments to help labs slow self-improving models in step. Jensen Huang told CNBC the rules on product reliability already cover it, and Mark Zuckerberg says Meta held its Muse model back on its own.

Publishers:pcmag.com uk.pcmag.com

Reality

Evidence40
Adoption
Not scored
Hype gap+15
Incentives
Not scored
Confidence45
leadershipOne report1 publisher

Anthropic's own index rates Claude as leading 26% of its AI R&D work

The lab published a prototype index of how much of its own AI research Claude now leads, alongside a promise to embed outside evaluators with access comparable to what its internal risk teams get. The figures are dated August 2026.

Publishers:anthropic.com

Reality

Evidence38
Adoption32
Hype gap+22
Incentives74
Confidence46
productOne report1 publisher

Hugging Face, a company Nvidia is buying for $12.93bn, asks to join Anthropic's third-party evaluator program

Anthropic and OpenAI have promised outside evaluators permanent, employee-level access to their systems. Neither has published the terms, and the first volunteer is mid-acquisition by a would-be anchor investor in Anthropic's IPO.

Publishers:thenextweb.com

Reality

Evidence56
Adoption18
Hype gap+38
Incentives82
Confidence58