Google Introduces 'Harness^2': A Recursive Agent Framework That Self-Improves at Test Time Without a Verifier

Google researchers have unveiled 'Harness^2' (Harness Squared), a research paper outlining a recursive agent harnessing framework that self-improves at test time without relying on external verifiers. Highlighted across the AI research community by DAIR.AI, the work establishes an actionable mechanism for scaling autonomous agent reliability in complex open-world tasks where deterministic ground-truth labels are unavailable.

Image source: DAIR.AI (@dair_ai) via X

Conventional agent refinement pipelines typically depend on predefined programmatic verifiers, ground-truth labels, or external reward models to guide reinforcement. However, in production environments and multi-domain open-world problems, programmatic verifiers rarely exist. Harness^2 circumvents this bottleneck by shifting self-improvement into test-time execution, comparatively testing harness edits for the active task and carrying distilled lessons forward via a secondary harness.

Test-Time Self-Improvement Without an External Verifier

The central premise of Harness^2 is that an agent can dynamically tailor its operational harness to the immediate task context through comparative internal testing.

• Candidate Edit Generation and Comparative Testing: Given an active task, the system proposes multiple candidate modifications to the agent harness. These candidates are evaluated against one another in comparative trials, isolating and preserving the best-performing combination of edits.
• Distilled Lessons and Secondary Harness Evolution: The empirical takeaways distilled from winning edits are subsequently integrated into a secondary harness maintained by the agent. The agent then carries this updated secondary harness forward into subsequent tasks.

By removing the hard dependency on an external verifier, this recursive pattern offers a concrete test-time compute scaling path for agentic systems facing open-ended goals.

Benchmark Results Across GLM-5, Gemini 3.5 Flash, and WorkBuddy-Bench

The researchers validated Harness^2 across three rigorous practical evaluation benchmarks: WorkBuddy-Bench, JobBench, and LAB.

• Consistent Foundation Model Gains: Across all tested harness configurations, Harness^2 delivered measurable performance improvements on both GLM-5 and Gemini 3.5 Flash.
• Parallel Test-Time Scaling: On WorkBuddy-Bench, running five candidate harness modifications in parallel added an additional 2.9 to 3.9 points.
• Outperforming Memory Baselines: In comparative coding evaluations, ReasoningBank—a conventional memory-based baseline—degraded performance by up to 6.6 points below the unmodified base harness. In contrast, Harness^2 maintained stable, positive performance margins without introducing regressions.

These findings highlight that recursively adapting and validating execution harnesses at test time provides a substantially more robust operational foundation than relying on passive memory-based baselines.

Practical Caveats: Inference Overhead and Rule Accumulation

While recursive harness evolution presents compelling advantages, deploying this architecture in production entails clear operational tradeoffs and engineering constraints.

• Self-Evaluation Bias Risks: Because candidate harness modifications are judged comparatively without an absolute ground-truth verifier, systems must manage the risks of candidate edit quality variance and comparative self-evaluation bias.
• Inference Latency and Token Overhead: Running five candidate modifications in parallel and testing them against each other demands substantial test-time compute and token budgets. Engineering teams operating under strict latency or cost SLAs will need to balance the number of parallel candidates against task value.
• The Need for Forgetting Mechanisms: As distilled harness rules accumulate across successive tasks, carrying legacy rules into unrelated domains can lead to rule pollution or counterproductive constraints. Long-running deployments will require explicit mechanisms for pruning or forgetting obsolete guidelines when transitioning across disparate workflows.

Sources

• DAIR.AI Official X (@dair_ai): Google Research 'Harness^2' Paper Discussion