Google Research Harness²: Automated Agent Self-Tuning and Experience Accumulation
Google Research open-sources Harness², enabling AI agents to modify prompts, skills, and memory, iteratively compare task runs, and accumulate experience for re
Image source: https://github.com/google-research/harness2
Harness², recently open-sourced by Google Research, introduces an automated architecture where autonomous agents actively inspect, adjust, and optimize their own scaffolding configurations—specifically prompts, skills, and memory—rather than relying solely on manual engineering by human developers.
A Three-Step Loop for Prompt, Skill, and Memory Optimization
According to researcher Xudong Han (@Xudong07452910), Harness² structures self-improvement into three sequential steps. First, the agent modifies its own prompt directives, skill definitions, and memory configurations. Second, it executes repeated runs on the same task, comparing outcomes across iterations to discover more effective working patterns. Third, the framework systematically preserves these improvement experiences, allowing future tasks to reference prior adaptations. This retention transforms localized trial-and-error into accumulated operational knowledge.
Benchmark Gains of 11.5 Points and the Practical Value of Automation
In reported experiments, Harness² achieved an improvement of up to 11.5 points on WorkBuddy-Bench within a single optimization round, proving that self-directed adjustments can drive measurable gains. In practice, builders spend significant time tuning harnesses—refining prompts, instructions, and tool interfaces. Automating this trial-and-error loop and retaining the resulting insights offers immense value, lowering manual overhead and accelerating workflow refinement.
Trial Costs and the Critical Challenge of Experience Transferability
Despite these strengths, practical caveats remain. Xudong Han notes that the cost of repeated trial-and-error across iterations represents a substantial resource expenditure. In addition, whether improvements on a single task can stably transfer to subsequent tasks requires further verification. Over-tuning to one task risks overfitting and performance degradation elsewhere. Adopting Harness² effectively requires balancing iteration costs against proven transferability.
Original source
Read the original analysis by Xudong Han (@Xudong07452910) on X: Xudong Han (@Xudong07452910) on X