Google Harness² Explained: How Recursive AI Optimization Delivers Up to 11.5 Points on WorkBuddy-Bench

8 minutes ago

12 min read

Artificial intelligence progress is increasingly shaped by more than the capabilities of an underlying language model. The software architecture surrounding that model, including its instructions, tools, memory, verification procedures, and task-management logic, can determine whether an agent produces reliable work or repeatedly makes preventable mistakes.

Google Cloud AI Research’s open-source Harness² (Harness2) framework explores this opportunity through recursive agent harnessing. Rather than updating a model’s learned parameters, it automatically experiments with the system components that guide the model’s behavior. It evaluates alternative configurations on actual tasks, retains changes supported by observed results, and records lessons that can inform subsequent work.

The central idea is significant for developers building AI agents for software engineering, professional services, research, and business operations. An existing model may deliver better results when equipped with a more effective operating environment, potentially reducing the need to retrain or replace it for every improvement.

Harness² does not eliminate the need to verify results, and its benchmark performance does not guarantee equivalent gains in production. Nevertheless, it offers a practical research direction for making AI agents more adaptive, efficient, and capable of learning from experience.

What Is Harness² and Why Does AI Agent Infrastructure Matter?

Harness² is an open-source framework designed to optimize an AI agent’s harness while tasks are being executed. A harness is the operational layer that connects a language model to its working environment and determines how it receives instructions, accesses information, invokes tools, delegates subtasks, and handles results.

A modern AI agent typically depends on several interacting components:

System prompts: Establish the agent’s role, objectives, constraints, and expected behavior.
Guardrails: Define persistent rules governing actions and acceptable procedures.
Memory: Preserve relevant information and lessons across interactions.
Tool guidance: Explain how and when to use external functions, applications, or command-line utilities.
Subagents: Delegate specialized tasks to other agents or model instances.
Skills: Provide reusable procedures for recurring workflows.
Scripts: Automate deterministic operations, checks, and transformations.
Plugins or middleware: Intercept tool interactions and enforce behavior at the integration layer.

These components influence how effectively a model converts its underlying reasoning capabilities into completed work. A model may understand a task but still fail because it overlooks a file, uses the wrong tool, misinterprets a spreadsheet, or neglects to inspect its output.

Harness engineering addresses these operational weaknesses without necessarily changing the model itself.

Harness² takes the concept further by automating part of the engineering process. Instead of requiring a developer to diagnose every failure and manually rewrite the relevant instructions, the framework proposes changes, tests them, compares observed behavior, and combines promising modifications.

The project is available under the Apache-2.0 open-source license. Its research implementation uses model-backed execution through Google Cloud Vertex AI, while its local demonstration can run without model credentials.

How Recursive Agent Harnessing Works

Harness² uses two connected feedback loops: one that improves the harness for an individual task and another that preserves useful lessons for future tasks.

1. Task-Level Optimization Through Propose, Probe, and Compose

The first loop examines a particular task and experiments with alternative ways of completing it.

Propose: The agent initially attempts the task using its existing harness. This produces a reference execution record, or trajectory, that captures how the agent approached the problem, used tools, and generated its deliverables.

An editor model then proposes two different modifications.

The first is an invariant candidate, which targets general weaknesses such as poor planning, ineffective tool use, inadequate context management, or insufficient verification. The second is a domain-specific candidate, which focuses on the knowledge, procedures, and constraints associated with the current task.

This separation matters because agent failures can arise from different sources. A system might need a universal rule to inspect its outputs, but it may also need specialized instructions for reconciling financial records or preparing legal documents.

Probe: Both candidate configurations execute the same task under comparable conditions. Their results can then be compared with the original trajectory to identify behavioral improvements.

For example, if an agent creates a spreadsheet but never checks whether its formulas work, a candidate harness could introduce an explicit output-inspection procedure. The subsequent execution reveals whether the additional instruction changes the agent’s behavior.

Compose: A composition stage examines the reference and candidate trajectories, then constructs a revised harness. It can preserve successful modifications, discard changes that conflict with each other, or retain the original configuration when the evidence does not justify an edit.

This approach is more structured than simply asking an AI model to improve its own instructions. The framework introduces explicit experimentation and comparison into the process.

However, the comparison is not equivalent to proving that an answer is correct. Without an authoritative verifier, the framework observes how behavior changes rather than establishing that every resulting claim, calculation, or deliverable is accurate.

2. Harness-Level Recursion and Persistent Learning

The second loop extends optimization beyond an individual task.

After a task is completed, a reflection stage can record the adopted harness configuration and document relevant lessons in an EXPERIENCE.md file. The record describes the situation, the observed evidence, and the procedures that appeared useful.

Future tasks can draw on this accumulated experience when proposing new modifications.

Importantly, each new task starts from the shared base harness rather than automatically inheriting every task-specific change. This helps distinguish reusable improvements from modifications that only make sense in a particular context.

The resulting architecture separates three functions:

Improve behavior for the current task.
Evaluate which changes have sufficient supporting evidence.
Preserve useful experience for future tasks.

This creates a mechanism for cross-task adaptation without updating the model’s weights.

The distinction is important for organizations that need agents to improve their procedures over time while keeping the underlying model fixed. It also introduces a new engineering requirement: the experience library must be managed carefully so that ineffective or overly specialized lessons do not become persistent sources of error.

Harness² Benchmark Results: What the Numbers Reveal

The supplied research reports experiments across three professional benchmarks: WorkBuddy-Bench, JobBench, and Harvey’s Legal Agent Benchmark, known as LAB. The evaluated executors were Gemini 3.5 Flash and GLM-5, paired with OpenCode or Codex. Claude Opus 4.8 served as the default editor in the described experimental setup.

Results below represent averages over three runs, with one refinement step and a search budget of (k=1).

Model and execution environment	WorkBuddy-Bench	JobBench	LAB Partial
Gemini 3.5 Flash with OpenCode	72.4 → 77.5	31.5 → 36.0	79.9 → 81.9
Gemini 3.5 Flash with Codex	74.5 → 77.5	33.9 → 35.3	71.6 → 75.6
GLM-5 with OpenCode	64.2 → 75.7	20.6 → 24.2	59.6 → 71.4
GLM-5 with Codex	64.4 → 75.3	23.6 → 24.8	58.1 → 65.9

Source context: Harness² research results supplied in the background material. Values show baseline performance followed by performance after refinement.

The reported experiments show improvements across all four model and execution-environment combinations on all three benchmarks.

The largest listed gain on WorkBuddy-Bench occurs with GLM-5 and OpenCode, which rises from 64.2 to 75.7, an increase of 11.5 points. On LAB, the same combination improves from 59.6 to 71.4, a gain of 11.8 percentage points. Gemini 3.5 Flash with OpenCode rises from 31.5 to 36.0 on JobBench, an increase of 4.5 points.

These findings suggest that the quality of the surrounding infrastructure can materially influence the practical performance of a fixed model. They also indicate that weaker starting configurations may have more room for improvement.

One particularly interesting comparison is GLM-5 with OpenCode reaching 75.7 on WorkBuddy-Bench after harness refinement, compared with Gemini 3.5 Flash’s baseline score of 72.4. In this experiment, improving the operating framework allowed one configuration to outperform a stronger baseline configuration on that benchmark.

That does not establish a universal ranking between the models. It demonstrates that model selection alone may not explain the quality of an agent’s output.

What Happens When the Search Budget Increases?

Harness² uses a parameter called (k) to control the amount of refinement performed.

In parallel mode, the framework explores multiple invariant and domain-specific candidates before producing compositions. In sequential mode, it chains successive refinement steps, allowing later experiments to build on earlier changes.

The research reports additional gains when parallel search increases to (k=5). Relative to a single refinement step, reported improvements range from 2.9 to 3.9 points on WorkBuddy-Bench, 3.0 to 4.2 points on JobBench, and 2.6 to 6.7 percentage points on LAB.

More experimentation, however, requires additional model executions and editorial work. A larger search budget can improve the likelihood of finding useful changes, but it also increases computational consumption and financial cost.

The practical question is therefore not simply whether more refinement improves benchmark scores. It is whether the incremental improvement justifies the extra expense for a particular workload.

For a low-risk internal task, a single refinement pass may be sufficient. For an expensive legal review or a complex engineering workflow, additional experimentation might be worthwhile if the resulting gains can be measured reliably.

Harness² Compared With Other AI Agent Optimization Methods

Harness² belongs to a broader research effort focused on improving agent behavior through infrastructure, memory, and automated configuration.

Its distinguishing characteristic is the combination of task-specific experimentation with a persistent experience library.

Approach	Main optimization strategy	Key distinction
Harness²	Refines harness components during task execution and retains supported lessons	Combines task-level experiments with cross-task experience
Meta-Harness	Searches for a strong harness using separate training tasks	Focuses on offline optimization
ReasoningBank	Retrieves strategies from previous experience while keeping the harness fixed	Emphasizes memory rather than changing the full harness
Harness Scaling	Revises the harness for individual instances	Does not provide the same persistent cross-task learning mechanism described for Harness²

These approaches address related problems but make different trade-offs.

Offline optimization can produce a stable configuration before deployment, which may be attractive in controlled environments. Memory-based methods can reuse previous experience without repeatedly changing system instructions or tool definitions. Per-task optimization can adapt to individual requirements but may repeatedly rediscover lessons if experience is not retained.

Harness² attempts to combine adaptation with cumulative learning. Its value will depend on whether the resulting improvements transfer to real workloads, remain stable across tasks, and justify their computational overhead.

The benchmark comparisons supplied with the research also show that Harness² does not win every individual comparison. Different optimization methods can perform differently depending on the model, benchmark, execution environment, and task type.

A Practical Example: Turning Procedural Mistakes Into Reusable Improvements

A JobBench example illustrates why harness refinement may be useful in professional workflows.

In the reported case, an agent using Gemini 3.5 Flash and OpenCode had to reconcile production calendar, staffing, and budget files into linked documents. The original attempt received a score of 43.1.

The execution record revealed procedural weaknesses. Figures were not consistently reconciled across the source files, and the agent did not reopen its completed outputs to verify them.

The framework tested changes targeting these problems. One candidate introduced guardrails requiring the agent to inventory its inputs, trace figures to authoritative sources, and reopen generated files. It also added a verification script to inspect spreadsheets and identify formulas that might appear blank to a reader.

The domain-specific candidate supplied procedures for reconciling calendar, venue, labor, and budget information. It also improved tool guidance for handling numerical values stored as text.

The composition stage retained the changes, and the reported score increased from 43.1 to 77.8.

This is an illustrative case rather than an average result, but it highlights an important principle: many agent failures stem from missing procedures rather than insufficient subject knowledge.

The same lesson applies to research agents, business analysts, and coding assistants. An agent can benefit from explicit rules to inspect source material, validate calculations, reopen generated files, and confirm that its deliverables satisfy the original requirements.

These procedures are often straightforward for developers to implement manually. Harness² explores whether the process of discovering and refining them can be automated.

Business Applications of Recursive AI Agent Improvement

Harness² points toward several potential applications where agents perform multi-step work and can be evaluated through observable behavior.

Software development: Coding agents can improve repository navigation, testing procedures, debugging workflows, and verification of generated code. Harness modifications may help agents remember to run tests, inspect failures, and validate changes before reporting completion.

Legal and professional services: Agents working with contracts, case materials, or regulatory documents can benefit from more disciplined source tracking, structured review procedures, and explicit checks for omitted sections. Because mistakes in these domains can have serious consequences, independent verification remains essential.

Business operations: Financial reconciliation, document preparation, scheduling, and reporting often require multiple files to remain consistent. Specialized skills and validation scripts can reduce recurring procedural errors.

Research and knowledge work: Research agents can improve their workflows for gathering evidence, maintaining traceability, comparing sources, and distinguishing supported conclusions from uncertain interpretations.

For organizations, the potential economic benefit is reduced manual intervention and better use of existing models. If a carefully designed harness improves task completion, a company may be able to extract more value from its current AI infrastructure before investing in a more expensive model.

Nevertheless, a benchmark gain is not automatically a financial gain. Businesses need to evaluate total cost, including model calls, orchestration, engineering time, security controls, error correction, and human review.

The Critical Limitation: Better Behavior Does Not Guarantee Correct Answers

The central technical challenge is the absence of a definitive verifier for many open-ended tasks.

In conventional machine learning, a system can be evaluated against known labels or expected outputs. Professional work is often more ambiguous. A legal analysis, research report, or business memo may have multiple valid approaches, making correctness difficult to reduce to a single score.

Harness² compares execution trajectories to identify promising changes, but behavioral improvement can be misleading. A modified agent may produce a more organized report without making it more accurate. It may follow instructions more consistently while continuing to rely on incorrect assumptions.

Shared biases between models can further weaken comparisons. Randomness between repeated executions can also make a successful change appear more reliable than it actually is.

The research supplied for this article reports evaluation on three benchmarks using two executor models and a limited set of configurations. Independent replication and testing across broader real-world workloads are important before generalizing the results.

Other challenges include:

Inference cost: Each refinement introduces additional model executions.
Operational complexity: More components create more configurations to maintain and audit.
Transfer risk: A modification that helps one task may be ineffective or harmful elsewhere.
Memory quality: Incorrect lessons can influence future proposals if they are retained.
Security: Agents that execute code or access external services require carefully controlled permissions and isolated environments.

The supplied project documentation also warns about the risks of unrestricted network access and permissive execution settings in experimental configurations. Organizations should isolate test environments, restrict outbound connections, use dedicated credentials, and avoid exposing production data or secrets to untrusted agent workflows.

For high-stakes work, harness refinement should complement independent evaluation, not replace it.

How Developers Can Experiment With Harness²

Developers can begin with the local demonstration before committing to model-backed experiments.

The described setup targets Linux, Python 3.12, and the uv package manager. A basic installation and demonstration sequence is:

git clone https://github.com/google-research/harness2
cd harness2

uv venv --python 3.12 .venv
source .venv/bin/activate

uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2

This demonstration runs the recursion modes without requiring model credentials.

Real benchmark executions require additional dependencies and a configured Google Cloud Vertex AI project. The described environment also uses Node.js 20 or later, while specific benchmarks require their own tools, including Docker for WorkBuddy-Bench and Pandoc and Poppler for LAB.

Model-backed experimentation can incur charges, so developers should begin with a small, controlled workload and monitor usage.

Before adapting the framework to a business application, teams should establish a baseline, define measurable success criteria, record failure cases, and independently validate the outputs. A useful experiment should measure not only task scores but also completion time, tool calls, token consumption, error rates, and the amount of human correction required.

The project’s published workflow provides a research foundation, but adapting it to a custom agent may require additional engineering.

The Future of AI Agents: From Static Configurations to Adaptive Systems

Harness² reflects a broader shift in AI development. As language models become increasingly capable, the challenge is moving from producing a single intelligent response toward building systems that reliably complete complex tasks.

Future agent architectures may combine several complementary capabilities: model reasoning, persistent memory, deterministic scripts, specialized tools, policy enforcement, and automated evaluation. The strongest systems will likely distinguish between what can be learned from experience and what must be checked against an authoritative source.

A particularly promising direction is the integration of stronger verification mechanisms into recursive optimization. If an agent can test generated code, validate calculations against source records, check document requirements, or compare outputs with trusted databases, it gains a firmer basis for deciding which harness modifications actually work.

Organizations may also develop task-specific harness libraries that encode proven procedures for particular departments, industries, or workflows. Such libraries could reduce repeated engineering effort, provided that teams control versioning, evaluate transferability, and remove outdated or unreliable lessons.

The long-term objective is not unrestricted self-modification. It is controlled adaptation, where changes are observable, testable, reversible, and aligned with explicit requirements.

Why Harness² Matters for the Next Generation of AI Agents

Google’s Harness² explores an important possibility: AI agents can improve their practical performance by refining the instructions, tools, safeguards, and workflows around an existing model.

Its reported benchmark results demonstrate meaningful gains in the tested configurations, including an 11.5-point increase for GLM-5 with OpenCode on WorkBuddy-Bench after one refinement step. The combination of task-level experimentation and a persistent experience library distinguishes the framework from approaches that optimize only offline or rely exclusively on retrieved memory.

The unanswered questions concern reliability, cost, transferability, and security. Without authoritative verification, a better-looking execution is not necessarily a correct one, and benchmark results must be validated against real operational requirements.

For developers and businesses, the immediate lesson is to treat agent infrastructure as an important part of AI performance engineering. Better models matter, but so do disciplined workflows, robust tools, explicit validation, and the ability to learn from mistakes without preserving them indefinitely.

As Dr. Shahid Masood and the expert team at 1950.ai examine developments in artificial intelligence and emerging technologies, recursive agent harnessing offers a useful perspective on where AI engineering may be heading: toward systems that do more than generate responses, and instead improve how they plan, execute, evaluate, and refine their work.