Skip to content

Build Reports disagree4 publishers 3 min read Published

Nvidia says the harness, not the model, took Claude Opus 5 from 30.2% to 100% on ARC-AGI-3

A five-person Nvidia team published the architecture behind its AVO agent alongside a perfect public-set score. The model did not change. The system around it did.

The Engineer · Build desk

Send a correction

What happened

  • Nvidia first introduced its Agentic Variation Operators (AVO) general-purpose coding agent system in late March 2026, and has now published the architecture and system-level mechanisms behind it, applied to the ARC-AGI-3 benchmark.
  • In a team blog released on Friday, a five-person team of Nvidia software engineers, machine learning specialists and AI research interns, headed by principal engineer Terry Chen, described how AVO elevated Claude Opus 5's ARC-AGI-3 performance.
  • ARC Prize, a non-profit AI research and benchmarking body, used an analysis post this July to report a 30.2% score for Claude Opus 5 at high reasoning effort on the public set of environments and tasks in ARC-AGI-3.
  • AVO achieved a 100.00 RHAE score across all 25 environments in the ARC-AGI-3 public set, completing all 183 levels.
  • ARC-AGI-3 uses Relative Human Action Efficiency (RHAE), a metric combining task completion with per-level action efficiency relative to first-time human baselines, aggregated across different levels and environments.

Why it matters

Nvidia has published the architecture of Agentic Variation Operators (AVO), the general-purpose coding agent system it introduced in late March 2026, and applied it to ARC-AGI-3 [s1 c1]. In a blog posted Friday by a five-person team of engineers, machine learning specialists and research interns led by principal engineer Terry Chen, the company reports that Claude Opus 5 went from a 30.2% baseline on the benchmark to a 100.00 RHAE score, with the harness as the only thing that changed [10][2][3].

The numbers first. ARC Prize, the non-profit that maintains the benchmark, reported 30.2% for Claude Opus 5 at high reasoning effort on the ARC-AGI-3 public set in an analysis post this July [2]. Nvidia reports 100.00 RHAE across all 25 environments in that public set, with all 183 levels completed [3]. That is a gap of 69.8 points, or roughly 3.3 times the baseline [22]. RHAE combines task completion with per-level action efficiency measured against first-time human baselines, aggregated across levels and environments [12], so it is not a simple pass rate and a perfect score means the agent was also action-efficient.

What makes this worth reading rather than filing is the claim about where the capability came from. Nvidia's framing is that the harness determines how a model receives context, uses tools, maintains state, responds to feedback, recovers from failure and sustains progress on long-running work [21]. The team writes that the result shows system design, not model capability alone, can unlock frontier-level long-horizon performance, and that "model capability matters enormously, but the surrounding system determines how effectively that capability can be converted into sustained autonomous progress" [20][19].

The transfer story is the part operators should look at. AVO was built for software engineering and GPU-kernel optimization, where it replaces the predefined variation step of conventional evolutionary search with an agent that decides what to inspect, change, test and commit [6]. For ARC-AGI-3, Nvidia says it connected the same agent to a different task interface, changing only the environment-specific tools and the evaluation [5]. ARC-AGI-3 drops agents into unfamiliar environments with no instructions, stated rules or stated goals [11], which on the surface has nothing to do with compilers and throughput, but Nvidia argues the loop is the same: build hypotheses from incomplete evidence, act through an interface, observe consequences, preserve state, revise, recover from wrong assumptions, keep going [13].

Two mechanisms carry that loop past a single context window: persistent memory and supervision [7]. Memory holds prior implementations, evaluation results, compiler and profiler output and accumulated reasoning so the agent resumes instead of rebuilding the search [9]. A supervisor watches the trajectory for stagnation and repeated unproductive cycles and can redirect the main agent [4]. In the kernel work, AVO ran continuously for seven days, explored more than 500 optimization directions and committed 40 kernel versions, with the supervisor stepping in when the search plateaued [15][8]. The resulting multihead attention kernels beat cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5% on DGX B200 across the configurations evaluated [16], and the agent then adapted the kernel to grouped-query attention in about 30 minutes [17].

Two things to watch. The 100.00 is a public-set result [3]; ARC Prize's own comparative figures for Claude Opus 5 include a semi-private ARC-AGI-2 score of 90.4% [18], and neither Nvidia post reports a semi-private ARC-AGI-3 number. And the baseline and the agent result were produced by different parties, ARC Prize and Nvidia respectively [2][3], which is the usual reason to wait for a third-party run before treating a 69.8-point delta as a property of the harness rather than of the setup.

Clarity's read

What the record supports and how the coverage leans. The claims behind it follow.

Reality

Evidence58
Adoption20
Hype gap+34
Incentives82
Confidence72

Perspective Coverage

4 publishers
Builder
Builder 50%
Operator
Operator 29%
Investor
Investor 21%
Why these scores

Claim ledger

Ranked by verification strength, evidence, and original report placement.

  1. [1]

    Nvidia first introduced its Agentic Variation Operators (AVO) general-purpose coding agent system in late March 2026, and has now published the architecture and system-level mechanisms behind it, applied to the ARC-AGI-3 benchmark.

    Reported SupportedView cited source
  2. [2]

    ARC Prize, a non-profit AI research and benchmarking body, used an analysis post this July to report a 30.2% score for Claude Opus 5 at high reasoning effort on the public set of environments and tasks in ARC-AGI-3.

    Reported SupportedView cited source
  3. [3]

    AVO achieved a 100.00 RHAE score across all 25 environments in the ARC-AGI-3 public set, completing all 183 levels.

    Reported SupportedView cited source
  4. [4]

    The supervisor monitors the broader trajectory for stagnation or repeated unproductive cycles and can redirect the main agent toward alternative strategies when needed.

    Reported SupportedView cited source
  5. [5]

    For ARC-AGI-3, Nvidia connected the same general-purpose agent to a different task interface; the underlying agent remains the same and only the environment-specific tools and evaluation change.

    Reported SupportedView cited source
  6. [6]

    In Nvidia's GPU-kernel optimization work, AVO replaces the predefined variation step of conventional evolutionary-search systems with an autonomous agent that decides how to generate the next candidate: what to inspect, what to change, what to test, and what to commit.

    Reported SupportedView cited source
  7. [7]

    AVO is designed to preserve progress beyond a single model context, and two mechanisms are particularly important: persistent memory and supervision.

    Reported SupportedView cited source
  8. [8]

    During the seven-day attention-kernel run, the main agent remained responsible for deciding what to inspect, change, test and evaluate, while the supervisor helped maintain forward progress when the search plateaued.

    Reported SupportedView cited source
  9. [9]

    Persistent memory carries forward prior implementations, evaluation results, compiler and profiler outputs, and accumulated reasoning, allowing the agent to resume from the current state rather than repeatedly reconstructing the search.

    Reported SupportedView cited source
  10. [10]

    In a team blog released on Friday, a five-person team of Nvidia software engineers, machine learning specialists and AI research interns, headed by principal engineer Terry Chen, described how AVO elevated Claude Opus 5's ARC-AGI-3 performance.

    Reported SupportedView cited source
  11. [11]

    ARC-AGI-3 is an interactive reasoning benchmark in which agents enter unfamiliar environments without instructions, stated rules, or stated goals.

    Reported SupportedView cited source
  12. [12]

    ARC-AGI-3 uses Relative Human Action Efficiency (RHAE), a metric combining task completion with per-level action efficiency relative to first-time human baselines, aggregated across different levels and environments.

    Reported SupportedView cited source
  13. [13]

    Nvidia argues GPU-kernel optimization and ARC-AGI-3 share an underlying computational pattern: the agent must build hypotheses from incomplete evidence, take actions through an external interface, observe the consequences, preserve useful state, revise its model of the problem, recover from incorrect assumptions, and continue making progress over a long horizon.

    Reported SupportedView cited source
  14. [14]

    AVO can inspect and edit code, run commands, consult documentation, and validate its work through execution; its distinguishing focus is sustained autonomous operation across long horizons.

    Reported SupportedView cited source
  15. [15]

    In Nvidia's attention-kernel study, AVO operated continuously for seven days, explored more than 500 optimization directions, and produced 40 committed kernel versions.

    Reported SupportedView cited source
  16. [16]

    On NVIDIA DGX B200 systems, the resulting multihead attention kernels outperformed cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5% across the evaluated configurations.

    Reported SupportedView cited source
  17. [17]

    The agent subsequently adapted the evolved kernel to grouped-query attention in approximately 30 minutes of additional autonomous work.

    Reported SupportedView cited source
  18. [18]

    According to ARC Prize, Claude Opus 5 run at Max reasoning effort scores 97.5% on ARC-AGI-1 and 90.4% on ARC-AGI-2 semi-private.

    Reported SupportedView cited source
  19. [19]

    The Nvidia blog team wrote: "Model capability matters enormously, but the surrounding system determines how effectively that capability can be converted into sustained autonomous progress."

    Reported Contested Source: Nvidia AVO blog teamView cited source
  20. [20]

    The Nvidia team wrote that the result shows system design, not model capability alone, can unlock frontier-level long-horizon performance.

    Reported Contested Source: Nvidia AVO blog teamView cited source
  21. [21]

    Nvidia states that a frontier language model is only one component of an AI agent, and that the surrounding agent system, often called a harness, determines how the model receives context, uses tools, maintains state, responds to feedback, recovers from failure, and sustains progress over long-running tasks.

    Reported ContestedView cited source
  22. [22]

    The move from the reported 30.2% Claude Opus 5 baseline to AVO's 100.00 RHAE score is a gap of 69.8 points, or approximately 3.3 times the baseline figure.

    Derived Contested

Sources

4 independent publishers whose own reporting we read for this story.

  1. developer.nvidia.com

    1 article · August 21, 2026

    NVIDIA AVO Reaches 100% on ARC-AGI-3, Demonstrating a Frontier-Level General-Purpose Architecture for Long-Horizon Autonomous Agents
  2. mezha.net

    1 article · August 21, 2026

    Nvidia показала, як harness підняв Claude Opus 5 до 100% на ARC-AGI-3
  3. runtimewire.com

    1 article · August 21, 2026

    NVIDIA's AVO scores 100% on ARC-AGI-3's public set
  4. thenewstack.io

    1 article · August 21, 2026

    Claude Opus 5 scored 30% on ARC-AGI-3. Wrapped in Nvidia’s AVO, it hit 100%.

Topics and entities

Topics

Entities