House Democrats ask the GAO what metrics the FAA uses to judge its 200-stream AI tool
House Democrats asked the GAO to identify the metrics the FAA uses to judge SMART, an AI tool that pulls 200 data streams into one view. What the sources describe are lawmakers' requests for test evidence.
Context for builders, not their beat.
Three figures. Ocasio-Cortez says the FAA launched SMART over three National Capital Region airports with plans to expand nationwide in the following 90 days. SMART aims to give aviation specialists a centralized view of 200 data streams.
| Figure | Value | Note | Claim |
|---|---|---|---|
| Planned window to expand nationwide, per Ocasio-Cortez | 90 days | 7 | |
| National Capital Region airports at launch | 3 airports | per Ocasio-Cortez | 7 |
| Data streams in one view | 200 data streams | 1 |
What happened
- More than two dozen House Democrats wrote Transportation Secretary Duffy and FAA Administrator Bedford last week, urging a pause until questions on safety testing, implementation and oversight are answered.
- Rep. Alexandria Ocasio-Cortez said the FAA launched the tool without publishing a safety evaluation or sufficiently consulting air traffic controllers and safety experts.
- Rep. Don Beyer said the FAA must show through controller feedback and independently verified testing that the technology is safe and reliable, and tell the public what it finds.
- Lt. Gen. Christopher Eubank of Army Cyber Command said Monday that none of the command's AI agents have strayed, citing a tight framework of guardrails.
Why it matters
- Who has to decide decision The GAO has to decide whether to take the request, and the sources do not show how the FAA has answered the earlier pause letter.
- Who's exposed exposure Controllers would be the people working with SMART output, and the letter says frontline workers were told days ahead or not at all.
- What it limits constraint Congress is still developing legislation, so the testing requests in these sources are letters, and neither the FAA nor the White House is required to act on them.
Most of the Democrats on the House Transportation and Infrastructure Committee asked the Government Accountability Office to identify the quantifiable performance metrics the FAA uses to evaluate SMART [2]. The tool is meant to give aviation specialists one view of 200 data streams, with the aim of cutting flight delays and cancellations [1]. A count of streams does not show how often the tool is right. The House letter goes after that gap.
The FAA's public defence is that a person stays in charge. "This tool won't manage airspace, but it gives us better information, better data to manage the airspace," Transportation Secretary Sean Duffy said at the launch event [6]. It does not answer the members' concern. Their letter cites the tool's potential to "interfere with decision-making by experienced air traffic controllers or to hallucinate safety-critical information" [3]. In our view, a controller who makes the final call still makes it on whatever SMART shows.
Timing sharpens the request. Rep. Alexandria Ocasio-Cortez said the FAA launched SMART "with plans to expand nationwide in the following 90 days" [7]. The 90 days is her account, because the FAA's own schedule does not appear in the sources. This quarter's choice is whether the FAA names a measure before any expansion. If the tool has spread by next quarter without one, the first evidence of its performance would come from mistakes it makes in operation.
Testing requests are coming from outside aviation and from both parties. Sen. Maria Cantwell, a Democrat, and Sen. Mike Rounds, a Republican, wrote to two White House officials asking for federal capacity to test advanced AI models independently, with qualified third-party testers alongside [9][10]. On the Army side, Lt. Gen. Christopher Eubank described humans checking each agent's work before the human-machine pair moves to the next task [12]. The sources describe that process but give no published error rate, either for the Army's agents or for SMART.
What to watch
- Whether the GAO accepts the request and the FAA publishes a safety evaluation or performance metrics for SMART before any expansion beyond the three Washington-area airports.
- Whether the White House task force and OSTP respond to the Cantwell-Rounds letter on independent testing capacity.
- Whether the Army's Project Griffin awards, due to be announced on the Friday after Eubank spoke, come with any stated measure of agent performance.
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
The FAA's SMART tool aims to give aviation specialists a centralized view of 200 data streams to help cut down on flight delays and cancellations.
- [2]
In a letter sent Tuesday, most of the Democrats on the House Committee on Transportation and Infrastructure asked the Government Accountability Office to investigate the FAA's safety risk management activities tied to SMART, identify the FAA's quantifiable performance metrics used for evaluations, and assess the development and deployment strategy.
- [3]
The members wrote that the use of AI within SMART has raised serious questions about its reliability and safety benefits, particularly on its potential to "interfere with decision-making by experienced air traffic controllers or to hallucinate safety-critical information."
- [4]
Last week, more than two dozen House Democrats sent a letter to Transportation Secretary Sean Duffy and FAA Administrator Bryan Bedford urging a pause in use of the tool until questions around safety testing, implementation and oversight were answered.
- [5]
Rep. Don Beyer said the FAA must demonstrate, through air traffic controller feedback and independently verified testing, that the technology is safe and reliable, and must clearly communicate those findings to the public before deploying SMART.
- [6]
Duffy said at the launch event: "This tool won't manage airspace, but it gives us better information, better data to manage the airspace."
- [7]
Rep. Alexandria Ocasio-Cortez said the FAA launched the AI tool over three National Capital Region airports "with plans to expand nationwide in the following 90 days."
- [8]
Ocasio-Cortez said the FAA launched the tool without publishing a safety evaluation or sufficiently consulting air traffic controllers and safety experts.
- [9]
Sens. Cantwell and Rounds urged that CAISSI, the Department of Energy and the National Laboratories, and other appropriate federal facilities have the capacity to independently test advanced AI models for serious national security risks, complemented by qualified third-party testing.
- [10]
Maria Cantwell (D-Wash.) and Mike Rounds (R-S.D.) sent the letter to Jay Clayton, Director of National Intelligence and chair of the White House Task Force for Super Intelligence, and Michael Kratsios, director of the White House Office of Science and Technology Policy.
- [11]
Lt. Gen. Christopher Eubank, head of Army Cyber Command, said Monday at the AUSA conference that none of ARCYBER's AI agents "have strayed to this point," citing the command's "tight framework."
- [12]
Eubank described a process in which human operators check agent work in the network and retrain the agents before the human-machine pair moves on to the next task.
- [13]
The senators wrote: "As Congress continues to develop legislative solutions, we urge you to make them a central part of the Administration's AI and cybersecurity strategy now."
- [14]
The House Democrats' letter says they heard from several sources that frontline aviation safety workers were notified of the SMART rollout mere days in advance or not at all, giving them no time to learn how the system works or how it fits their workflow.
- [15]
The Army was set to announce two awards for its Project Griffin agent-ecosystem initiative on Friday, according to Brandon Pugh, the service's principal cyber advisor.
Sources & coverage · 4 publishers
The reporting this story was synthesized from, earliest first. Every link goes to the original.
- defensescoop.com4d agoARCYBER expanding use of AI agents for network operations | DefenseScoop