Frontier labs pledge employee-like audit access for evaluators, as a small field of third-party firms emerges

Amodei called for model evaluators to embed with frontier AI companies, and said Anthropic is committed to giving evaluators "employee-like access" to monitor and verify the safety of its models.

Anthropic CEO Dario Amodei published an essay titled 'We Must Pace the Frontier' on Saturday, September 12, arguing AI was advancing too quickly for researchers to continue building it safely.

Amodei pointed to two key warning signs: early indications of self-improving AI and the recent OpenAI-Hugging Face incident.

Trump wrote on Monday: "The only control or 'guardrails' that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!"

Trump signed an executive order in June asking AI companies to voluntarily submit their models to the government before release, and is adamantly opposing any federal regulation.

OpenAI paused training on GPT-6 Astra for a little more than two weeks before rolling it out to a limited set of users, and its largest planned frontier training run reportedly remains on hold while new guardrails are put in place.

Altman wrote on X on Saturday: "I agree with Dario [Amodei] that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same."

Altman wrote on X on Sunday, September 13: "We welcome a federal framework that sets consistent safety requirements for frontier AI. But we do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence."

Trump said, according to the Financial Times: "Look, we're leading China in AI . . . and, frankly, I want to keep it that way, because whoever wins AI, wins."

The third-party evaluator Amodei cited as an example was METR, a nonprofit founded by a former OpenAI and DeepMind researcher that says its mission is to "develop scientific methods to assess catastrophic risks stemming from AI systems' autonomous capabilities and enable good decision-making about their development".

Only a handful of other firms are in the third-party frontier evaluation space, including FAR.AI, AVERI and Apollo Research.

AI in the US is regulated for the most part at the state level; California Governor Gavin Newsom recently signed multiple AI bills into law, including Adam's Law, which requires AI chatbot makers to protect minors from harmful interactions.

Representatives from Anthropic and OpenAI didn't immediately respond to CNBC's requests for comment.

Goldman Sachs estimates global AI investment will reach around $1 trillion in 2026, including roughly $581 billion in the US.

S&P Global says combined capital expenditure from the six largest hyperscalers, Alphabet, Amazon, Microsoft, Meta, Oracle and SpaceX, is expected to exceed $1.3 trillion by 2027.

AI researcher and lecturer Eli David said: "Perfectly explains Dario's motivation: Slow down research to cut compute spending that is spiraling out of control, so he can IPO."

Flock co-founder and CEO Ed Leon Klinger said: "For it to be true, Sam, Dario, Demis, and Elon all have to be lying, along with a big chunk of their execs, chief scientists, and resigning employees... A much simpler explanation at this point: they think the risk is real."

David Sacks, Trump's former AI czar and a venture capitalist with investments in AI startups, has repeatedly criticised what he calls Anthropic's "regulatory capture" campaign, and said the two leading model developers should police themselves rather than asking the government to do it.

Sacks wrote: "If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible."

AI-linked Asian stocks fell sharply on Monday following the slowdown calls, with SoftBank down 13.2%, Kioxia 9.8% and SK Hynix 5.3%.

Anthropic and OpenAI are trying to grow fast enough to justify their almost trillion-dollar valuations while responding to concerns their technology could spin out of human control.

Tom Wheeler, a former chair of the Federal Communications Commission and now a visiting fellow at Brookings, said the companies can't be responsible for effectively regulating themselves, and that the latest proposal for industry oversight was so vague that it amounted to an "empty vessel".

"Who makes the decision about standards? Who makes the decision about who's going to do the evaluation? Who makes the decision about enforcement?" Wheeler said.

OpenAI has said AI research is becoming increasingly autonomous and that coding agents were materially accelerating researchers' work, using 3.1 agent workdays for every workday of human labor by mid-August.

Shlomit Wagman, a Harvard Kennedy School fellow, wrote that the first workable global AI safety pact should be designed for a world without trust.

Wagman wrote that perfect verification is impossible and the objective should be to make cheating sufficiently detectable, and sufficiently costly, that compliance becomes strategically rational.

Wagman wrote that evaluators inside companies solve the problem inside the lab, not between strategic rivals.

Agents accounted for about three quarters (roughly 76%) of the combined agent-plus-human workday total OpenAI reported for mid-August.

The US share of Goldman Sachs' 2026 global AI investment estimate is about 58%.

Four organisations are named in CNBC's account of the third-party frontier evaluation field: METR, FAR.AI, AVERI and Apollo Research.

Spending on AI safety and alignment is drastically eclipsed by spending on AI development and capabilities.