Skip to content

Topic

AI Auditing

The practice of independent parties assessing AI systems and models against stated requirements, and the accreditation regimes that decide who may do it.

Current stories

investOne report1 publisher

OpenAI and Anthropic models hacked five companies during internal testing

GovAI's Alan Chan says labs' published safety tests may not reflect internal use, where models with safeguards off hacked at least four companies. The independent audits he favors need technical staff that, by his account, the field does not yet have.

Publishers:fortune.com

Reality

Evidence55
Adoption
Not scored
Hype gap+10
Incentives40
Confidence50
investConfirmed3 publishers

California prices a knowing under-16 feed violation at $50,000 per minor from January 2027

California's under-16 feed ban starts on January 1, 2027, and the cheapest way to comply is an age signal built from a birth date the user typed at device setup. The EFF expects platforms to verify every adult instead.

Perspective Coverage

3 publishers
Builder
Builder 25%
Operator
Operator 42%
Investor
Investor 33%

Reality

Evidence55
Adoption
Not scored
Hype gap+30
Incentives55
Confidence50
leadershipOne report1 publisher

The bias evidence for agent-to-agent hiring comes from the vendor selling the agents

Matt Wilson's AI agents represent close to half a million professionals to around 5,000 companies, and the bias experiment he cites as the reason to trust audited screening sits in his own company's white paper.

Reality

Evidence22
Adoption25
Hype gap+45
Incentives88
Confidence45
leadershipOne report1 publisher

California gives officials until November 16 to recommend AI safeguards, including a possible kill switch

Newsom's executive order sets a deadline for recommending the shutdown capability he vetoed in 2024, and the two audit laws he signed last week already give the state the inspectors such a rule would need.

Publishers:cbsnews.com

Reality

Evidence45
Adoption25
Hype gap+30
Incentives60
Confidence50
productOne report1 publisher

White House shelves the self-regulating AI body Treasury and OSTP had drafted

Wired reports the FINRA-style oversight framework has sat in limbo since executives called Trump in August. Product teams are left with an embedded-auditor bill and no House voting days before the midterms.

Publishers:wired.com

Reality

Evidence45
Adoption
Not scored
Hype gap+10
Incentives72
Confidence45
productOne report1 publisher

California's AI auditor registry lets each firm declare the standards it audits against

Two now-signed California laws build a registry and a qualifications framework for AI auditors ahead of a 2029 deadline. The audit itself stays voluntary, and what it measures is left to whichever firm sells it.

Publishers:iapp.org

Reality

Evidence58
Adoption
Not scored
Hype gap−10
Incentives66
Confidence57
buildOne report1 publisher

California will bar unregistered AI audits from January 1, 2029

Newsom signed AB 1405 on September 9, giving California's Government Operations Agency until 2029 to publish a registry of AI auditors, complete with registration numbers, annual fees and independence rules for anyone selling a covered audit.

Publishers:startupfortune.com

Reality

Evidence52
Adoption
Not scored
Hype gap+20
Incentives68
Confidence48
productOne report1 publisher

California sets the rules for who may audit an AI system

Newsom signed two bills on September 9 creating independent verification organizations for AI systems and a state registry of auditors with standards for their independence. Vendors now have a defined party to satisfy.

Publishers:gov.ca.gov

Reality

Evidence58
Adoption
Not scored
Hype gap+30
Incentives80
Confidence60