NVIDIA AVO Hits 100% on ARC-AGI-3, Proving the AI Harness May Matter More Than the Model

Professor Scott Durant
50 minutes ago
9 min read

The artificial intelligence industry has spent years treating the underlying model as the central measure of progress. Bigger models, stronger reasoning, larger context windows, and better benchmark scores have dominated the competitive narrative. NVIDIA’s latest research on its Agentic Variation Operators, or AVO, suggests that this model-centric view is becoming incomplete.

The important development is not simply that AVO achieved a perfect 100.00 RHAE score across the 25-environment public set of ARC-AGI-3. The deeper lesson is that the architecture surrounding an AI model can dramatically change what that model is capable of accomplishing over long periods of autonomous interaction.

NVIDIA reports that Claude Opus 5, which achieved approximately 30% under a separate model-level evaluation at high reasoning effort, reached 100.00 RHAE when integrated into the AVO agent system. AVO completed all 183 levels in the ARC-AGI-3 public set using 6,624 environment actions. A comparable VISTA configuration using Claude Opus 5 reportedly completed the same levels using 7,542 actions.

The distinction matters because an AI model and an AI agent are no longer interchangeable concepts. The model provides reasoning and generation capabilities, while the agent harness determines how those capabilities are connected to memory, tools, feedback, execution, supervision, and recovery.

The AI Model Is Only One Layer of an Agent

A foundation model can generate code, reason about a problem, interpret information, or recommend an action. But long-horizon autonomy requires considerably more than generating a useful response.

An agent working on a complex task must repeatedly perform a cycle:

Understand the current state.
Form a hypothesis about what to do next.
Execute an action.
Observe the resulting feedback.
Update its understanding.
Preserve useful information.
Recover from errors.
Decide whether to continue, change strategy, or backtrack.

This transforms AI from a question-answering system into an operational system.

The software layer that manages this process is commonly described as a harness or agent scaffold. It can determine which tools the model can access, what information remains in context, how previous work is stored, when new information is retrieved, how failed attempts are handled, and when another reasoning cycle should begin.

That means two systems using exactly the same foundation model can behave very differently.

Component	Primary function
Foundation model	Reasoning, generation and interpretation
Tools	Enable interaction with external systems
Memory	Preserves useful information across iterations
Runtime	Executes actions and manages state
Feedback	Grounds decisions in real-world or environment outcomes
Supervisor	Detects stagnation and redirects strategy
Harness	Coordinates the complete agent workflow

The strategic consequence is significant. AI performance increasingly depends on the engineering of the complete system rather than simply selecting the strongest available model.

NVIDIA AVO Demonstrates the Power of System Design

NVIDIA developed AVO initially around autonomous software engineering and GPU-kernel optimization. In that environment, an AI system cannot simply write code and declare success.

GPU optimization involves a difficult search space where seemingly minor changes can influence correctness, memory access, scheduling, compiler behavior, and throughput. Reliable optimization requires repeated experimentation against actual execution results.

In NVIDIA’s reported attention-kernel experiment, AVO operated continuously for seven days, explored more than 500 optimization directions, and produced 40 committed kernel versions. On NVIDIA DGX B200 systems, the resulting multihead attention kernels reportedly exceeded cuDNN performance by as much as 3.5% and FlashAttention-4 performance by as much as 10.5% across the evaluated configurations.

The significance goes beyond those performance figures. AVO was able to maintain an engineering process over a prolonged period rather than relying on a single model response.

It could inspect implementations, generate modifications, execute tests, analyze performance, learn from unsuccessful approaches, and continue searching.

That same architecture was then applied to a radically different environment.

From GPU Optimization to ARC-AGI-3

ARC-AGI-3 is designed to test interactive reasoning in unfamiliar environments. Agents receive limited information and must discover the rules and objectives through interaction.

This is fundamentally different from conventional benchmarks where a model receives a question and produces an answer.

An ARC-AGI-3 agent must learn from consequences. An action can provide information, eliminate a hypothesis, expose a rule, or move the agent closer to completing a level. Efficiency matters because excessive experimentation can consume valuable actions.

NVIDIA’s AVO configuration used a text-only representation of the environment, supplying the model with an exact 64 by 64 text grid rather than images or image tokens. The system was not simply handed the rules of each environment. It had to infer how the environment behaved through interaction.

AVO reportedly completed all 25 environments in the public set, solving all 183 levels and achieving a 100.00 RHAE score.

The result illustrates an important distinction between model capability and agent capability.

A model benchmark asks, in effect, what the model can accomplish within a particular evaluation setup. An agent benchmark asks what the entire system can accomplish when the model has tools, memory, feedback mechanisms, execution capabilities, and an architecture for sustained action.

Why Persistent Memory Changes Long-Horizon AI

One of the central problems with autonomous AI is that useful information can disappear when the active context changes.

A long-running engineering project may generate thousands of observations. A software agent may discover that one optimization fails because of a compiler limitation, another produces better throughput, and a third introduces a subtle correctness problem.

If the system repeatedly forgets those discoveries, it wastes resources rediscovering the same information.

AVO addresses this through persistent memory that can preserve previous implementations, evaluation results, compiler and profiler outputs, and accumulated understanding.

Memory therefore becomes more than a convenience. It functions as a mechanism for compounding experience.

This is particularly important for agents operating across hours or days. The objective is not simply to make each individual model invocation smarter. The objective is to make the entire sequence of decisions increasingly informed by what happened previously.

The Supervisor Introduces a Second Layer of Intelligence

NVIDIA’s architecture also highlights another important development, supervisory agents.

A conventional agent may become trapped in an unproductive strategy. It can repeatedly investigate the same direction, spend excessive resources on an unpromising hypothesis, or fail to recognize that its current approach has reached a dead end.

A supervisor provides another layer of control.

Rather than performing the primary task itself, the supervisory component monitors the broader trajectory. It can recognize stagnation and encourage the main agent to explore a different strategy.

This resembles organizational decision-making. A worker may execute the detailed task, while a manager monitors whether the overall project is moving toward its objective.

The architecture consequently becomes hierarchical:

Model → Main Agent → Tools and Memory → Supervisor → Overall Search Trajectory

This structure could become increasingly important as AI agents begin performing complex research, software engineering, scientific experimentation, cybersecurity operations, and business workflows.

Why the Same Architecture Can Transfer Across Domains

The most compelling aspect of AVO is not necessarily its performance on one benchmark. It is the apparent transfer of its operating principles between unrelated tasks.

GPU-kernel optimization and interactive reasoning games have almost nothing in common at the domain level.

One requires knowledge of software, hardware and performance engineering. The other requires discovering unknown rules through interaction.

Yet their underlying computational pattern is similar:

hypothesis → action → observation → evaluation → memory → revision

This suggests that general-purpose agent architecture may become a reusable layer between foundation models and specialized applications.

Instead of building an entirely new agent architecture for every task, developers could potentially retain a common framework while changing the tools, environment interface, evaluation mechanism, and domain-specific skills.

That would represent a major shift in AI engineering.

The Cost Dimension: Better Harnesses Can Mean Better Economics

Agent architecture is not only a question of accuracy.

Every unnecessary action consumes inference capacity, tool calls, compute resources, and time. An inefficient agent may use a powerful model repeatedly when a better memory system, context strategy, or supervisory mechanism could reduce the number

of required steps.

Research cited around agentic systems has increasingly pointed toward this problem. Different harnesses can produce substantially different costs even when they use the same underlying model.

For businesses deploying agents at scale, this creates a new optimization problem.

The relevant question is no longer simply:

Which model is cheapest or smartest?

It becomes:

Which combination of model, harness, tools, memory and runtime produces the required result at the lowest total cost?

This could make agent architecture a major source of competitive advantage.

Open Models Need Open Agent Stacks

The development also strengthens the strategic argument for openness.

If users can control only the model but not the surrounding agent infrastructure, they may remain dependent on decisions made by the model provider.

An open agent stack can expose additional controls over:

Memory management
Tool access
Context construction
Runtime behavior
Supervisory logic
Evaluation
Security policies
Infrastructure
Model selection

This flexibility matters particularly for enterprises that need predictable behavior, auditability, privacy, and integration with internal systems.

It also changes the competitive landscape. The future of AI may not be determined exclusively by whoever develops the most capable foundation model. Companies that build superior orchestration, memory, evaluation and runtime systems could capture substantial value around those models.

Security Becomes More Important as Autonomy Increases

Greater autonomy also creates greater risk.

An agent capable of taking hundreds or thousands of actions has a much larger operational surface than a chatbot that produces text. A mistaken assumption can propagate through multiple subsequent decisions.

The risks can include unauthorized file changes, incorrect software modifications, excessive resource consumption, unsafe tool calls, or persistent execution of a flawed strategy.

A sophisticated harness therefore needs more than memory and performance optimization. It requires safeguards, permissions, monitoring, rollback mechanisms, action boundaries, and reliable evaluation.

The same architectural layer that makes agents more capable can also become the layer responsible for preventing dangerous behavior.

This is why agentic AI security cannot be treated as an afterthought.

The New AI Competition Is Moving Up the Stack

For years, the central AI competition focused on training larger and more capable models. That competition remains important, but AVO illustrates why the next phase may involve the entire AI stack.

The emerging hierarchy looks increasingly like this:

Foundation model + Agent harness + Memory + Tools + Runtime + Supervisor + Infrastructure + Evaluation

Performance is produced by the interaction of all these layers.

This does not mean the model is becoming unimportant. A weak model cannot necessarily be transformed into a frontier system simply through orchestration. Rather, the lesson is that model capability represents only part of the final performance envelope.

A highly capable model can waste resources without effective memory. It can lose direction without supervision. It can produce theoretically strong answers without tools that allow those ideas to be tested. And it can fail to complete long tasks without a runtime designed to preserve state and recover from errors.

What NVIDIA AVO Means for the Future of AI Agents

NVIDIA’s AVO research points toward an AI industry increasingly focused on persistent, autonomous systems rather than isolated model interactions.

Future agents are likely to be judged less by how impressive a single response appears and more by whether they can complete complicated objectives reliably.

That means benchmarks will increasingly need to measure:

Long-horizon task completion
Action efficiency
Error recovery
Memory retention
Tool reliability
Cost per completed objective
Security and controllability
Transfer across unfamiliar environments

The strongest systems may ultimately be those that can turn a model’s reasoning ability into sustained, measurable progress.

For researchers and businesses, the practical lesson is straightforward. Choosing a frontier model is only the beginning. The surrounding architecture can determine how much of that model’s theoretical capability becomes useful performance in the real world.

The Agent, Not Just the Model, Is the Product

NVIDIA’s AVO results represent an important moment in the evolution of agentic AI because they shift attention from isolated model scores toward system-level intelligence.

The reported jump from roughly 30% model-level performance to a 100.00 RHAE result under a substantially different agent architecture does not prove that harnesses universally matter more than models. The experimental configurations are not identical, and the contribution of individual architectural components requires further controlled research.

But the broader lesson is difficult to ignore.

Memory, supervision, tools, feedback, runtime design and context management can fundamentally alter what an AI model accomplishes over a long horizon.

The next generation of AI competition may therefore be fought not only inside neural networks, but around them.

For organizations exploring autonomous AI, the strategic question is no longer simply which model to buy. It is how to engineer the complete system that turns model intelligence into reliable action.

As research from NVIDIA and the wider AI ecosystem continues to develop, the architecture surrounding the model may become one of the most valuable layers in artificial intelligence. For analysts such as Dr. Shahid Masood and research organizations such as 1950.ai, this system-level transition offers a particularly important lens for understanding where AI capability is heading next.

The future of AI agents may belong to the systems that remember, supervise, evaluate, recover and adapt, not merely the models that generate the most impressive answer.