Safety evaluators set four terms for embedding inside frontier labs
More than 100 signatories, Geoffrey Hinton and METR among them, told frontier AI companies that third-party testing needs ownership, payment and retaliation terms the evaluators say they do not have today.
The Investor · Invest desk
What happened
- More than 100 AI researchers and evaluators published a letter on Friday saying third-party safety testers lack the independence, resources and legal protections to credibly assess frontier model risk.
- The document is titled "Minimum Conditions for Embedding Evaluators", and the coalition behind it is the AI Evaluator Forum, whose chair is Conrad Stosz.
- Its conditions include that evaluators not be owned or governed by the companies they assess, not be paid contingent on findings, and get access equivalent to senior internal employees.
- Sam Altman, Elon Musk and Satya Nadella have each backed Dario Amodei's employee-like access proposal, and none of them has said how evaluators would be chosen or how much access they would get.
Why it matters
- Who's exposed exposure The signatories are asking the labs to absorb a legal risk that currently sits on academic and nonprofit evaluators, who can be sued over a conclusion a company dislikes.
- What it limits constraint With qualified organisations numbering in the handful, a lab that rejects one evaluator's terms has few substitutes, and an evaluator that loses a lab has few buyers.
- Who has to decide decision Each executive who endorsed employee-like access now chooses between publishing selection criteria and access depth or leaving the endorsement itself as the only record.
Next story · Invest · Sep 19
Washington's kill switch has to defeat the redundancy hyperscalers spent billions buildingConfirmed2 publishers
A third-party evaluation is worth whatever the access agreement behind it allows, and the conditions in this letter describe an agreement the signatories say they do not currently have [1][3]. CNBC was given the document exclusively, and Quartz took the letter's terms and quotes from CNBC, so what the public can see is that one newsroom's copy [4][15].
There are very few sellers in this market. "There's a very small number of groups that are actually sufficiently technically credible and have the scale and the ability" to perform the kind of work, Stosz said [9]. The hundred-plus signatures are individuals, some of them at Johns Hopkins, Stanford and METR, and how many organisations could actually be embedded is a separate number [5][16].
An evaluator paid a flat fee can still lose the next contract. Neither published account says what evaluators are paid, or by whom [17]. The letter asks for protection from retaliation, including retaliatory litigation [1].
The letter addresses the government and the public. Vinh Nguyen, a Council on Foreign Relations senior fellow for AI and the National Security Agency's former chief AI officer, said in a statement: "When a few powerful labs control capabilities that can endanger the cybersecurity, critical infrastructure, and the systems our national security and economy run on, the government and the public cannot be dependent on those labs' own account of what's secure and safe" [11].
President Donald Trump and David Sacks, his former AI czar, have opposed federal efforts to regulate AI development [13]. No statute requires any of this. Stosz acknowledged that the companies could ignore the letter, and said their credibility is at stake [10].
So an outside reader should price a frontier safety evaluation as the output of an access agreement whose terms remain unpublished [2]. The argument against this is that the letter is a bid for standing by the few groups that would be embedded, and that the labs' internal testing already finds most of what an embedded evaluator would; Stosz said third-party evaluators are not intended to "be a replacement for any internal efforts to evaluate, let alone mitigate issues that that developers find" [14]. The evidence pointing the other way is a single example: Stosz cited the unreleased OpenAI model used in the attack on Hugging Face as a system that independent scrutiny would have reached [6]. A published agreement naming the evaluator, the access level and the non-retaliation term would settle which of the two is right.
What to watch
- The safety talks among OpenAI, Anthropic and Google DeepMind, and whether they yield a shared standard for choosing evaluators.
- Any legal action by a frontier lab against an evaluator over a published finding.
- Whether Amodei's employee-like access acquires a first named evaluator.
Information, not investment advice. This story is general information about companies and markets, drawn from cited reporting. It is not a recommendation to buy, sell or hold any security; check the cited sources before acting.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence62
- Adoption15
- Hype gap+8
- Incentives66
- Confidence58
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
The letter sets out minimum conditions: evaluators must not be owned or governed by the companies they assess, must not receive payment contingent on their findings, must be shielded from retaliation including retaliatory litigation when conclusions are unfavourable, and must receive access equivalent to that of senior internal employees, including the ability to speak candidly with staff and review unreleased systems.
- [2]
OpenAI CEO Sam Altman, Elon Musk of xAI and Microsoft CEO Satya Nadella have each voiced support for Amodei's proposal, yet none has offered specifics on how evaluators would be chosen or what level of access they would receive.
- [3]
More than 100 AI researchers and evaluators published a public letter on Friday warning that third-party safety evaluators lack the independence, resources and legal protections needed to credibly assess the risks posed by frontier AI models.
- [4]
The letter was organised by the AI Evaluator Forum, a consortium chaired by Conrad Stosz, and was shared exclusively with CNBC.
- [5]
Signatories include Geoffrey Hinton and members of organisations such as Johns Hopkins University, Stanford University and the nonprofit evaluator METR.
- [6]
Stosz cited an unreleased OpenAI model used in an attack on Hugging Face as an example of a system that would benefit from independent scrutiny.
- [7]
Anthropic CEO Dario Amodei floated the idea of providing some evaluators "employee-like access" to inspect and audit bleeding-edge foundation models and their development processes.
- [8]
The letter published by the coalition is titled "Minimum Conditions for Embedding Evaluators".
- [9]
"There's a very small number of groups that are actually sufficiently technically credible and have the scale and the ability" to perform the kind of work.
- [10]
Stosz acknowledged that it is possible the foundation model companies ignore the public letter and the call to action, but said their credibility is at stake.
- [11]
"When a few powerful labs control capabilities that can endanger the cybersecurity, critical infrastructure, and the systems our national security and economy run on, the government and the public cannot be dependent on those labs' own account of what's secure and safe"
- [12]
The letter's minimum conditions as reported by Quartz number four: ownership and governance separation, non-contingent payment, protection from retaliation, and senior-employee-equivalent access.
- [13]
President Donald Trump and his former AI czar, David Sacks, have adamantly opposed government efforts to regulate AI development.
- [14]
Third-party evaluators are not intended to "be a replacement for any internal efforts to evaluate, let alone mitigate issues that that developers find".
- [15]
Both published accounts of the letter trace to CNBC: CNBC received the letter exclusively, and Quartz attributes the letter's terms and the Stosz quotes to CNBC.
- [16]
The 100-plus signature count and the count of organisations capable of embedding describe different populations: the signatures are individuals, while the letter's own chair describes the set of technically credible, sufficiently large groups as very small.
- [17]
Neither published account states what third-party evaluators are paid, or who funds them.
- [18]
OpenAI, Anthropic and Google DeepMind have been engaged in talks on AI safety.
Sources
2 independent publishers whose own reporting we read for this story.
- cnbc.comAnthropic and OpenAI need truly independent safety evaluators, experts say in public letter
1 article · September 18, 2026
- qz.comAI experts warn safety evaluators lack independence and protections
1 article · September 18, 2026
Topics and entities
Topics
- Third-Party Auditing
- US AI Policy and Politics
- AI Safety Evaluation
- Frontier Model Governance