Frontier lab scientists ask governments to put auditors inside their own companies

The people making this request write for the same companies that oversight would inspect. Jakub Pachocki is OpenAI's chief scientist, Jack Clark co-founded Anthropic, Eric Horvitz is Microsoft's chief scientific officer, and Dawn Song is Meta's vice president of AI research ^[5]. The paper says the authors wrote in a personal capacity and that their views do not necessarily represent their employers ^[14]. Academic and civil-society researchers initiated and led the project ^[15]. That distinction matters when the ask is for governments to look inside the firms these authors work for.

The measurement request is specific. Frontier companies would report how much of their research AI has automated, how fast capability is advancing, where research money goes, and whether AI systems take part in high-stakes research decisions ^[6]. Independent auditors could work inside some companies, on the model used by banking and nuclear regulators ^[7]. Song put the case for automated oversight plainly: "Already today, we are at the stage where we need AI systems to monitor what agents are doing. There is no other way to even observe and monitor these agents, humans are already insufficient," she said ^[16].

The one hard number in the paper is Anthropic's own. The authors cite the company's figure that AI completed 26 percent of its internal research and development work with only high-level supervision in August 2026, up from 1 percent in March ^[3]. Those figures are self-reported, and the authors say the threshold for an intelligence explosion has not been reached ^[4]. So the case for oversight rests partly on a metric that only the lab can produce. That is one reason the authors want required reporting: an outsider cannot check the 26 percent without a defined way to measure it.

The paper's central worry is a feedback loop. AI systems that help build more capable successors, which then do more of the research, is what the authors call an intelligence explosion ^[8]. Only thousands of people now do frontier AI research, and software agents can be copied and run in parallel, so full automation could add work equivalent to that of millions of researchers ^[9]. The paper's harder proposals follow from that: a speed limit on capability growth, ways to halt high-stakes experiments, and some automated research run on air-gapped networks physically isolated from the internet ^[10]. The authors say the extreme case could include "the marginalization or extinction of humanity" if highly capable systems escaped human control ^[11].

Not everyone reads the warnings the same way. Conrad Stosz, former head of CAISI, said it is "a little ambiguous what embedded evaluators means" ^[17]. Nvidia's Jensen Huang called the labs' warnings "odd" ^[18]. And the political ground is not neutral: President Trump told the United Nations last week that the United States would "totally reject" any "globalist scheme" to control AI ^[12]. OpenAI, Microsoft and Meta declined to comment on the paper, and Anthropic did not respond ^[19].

Pachocki has made a version of this argument from inside his own company. In a Sept. 6 essay he called for a "network of third-party auditors" to enforce common safety thresholds ^[20]. OpenAI aims to build an automated AI researcher by March 2028, a system that can set its own questions and run experiments with less human direction ^[13].