arxiv-2605-09998 SRC-0086 · paper

Continual Harness: Online Adaptation for Self-Improving Foundation Agents

Seth Karten, Joel Zhang, Tersoo Upaa Jr, Ruirong Feng, Wenzhe Li, Chengshuai Shi, Chi Jin, Kiran Vodrahalli

Created: 2026-05-11 · Ingested: 2026-09-14

https://arxiv.org/abs/2605.09998(opens in a new tab)

Coding harnesses such as Claude Code and OpenHands wrap foundation models with tools, memory, and planning, but no equivalent exists for embodied agents' long-horizon partial-observability decision-making. We first report our Gemini Plays Pokemon (GPP) experiments. With iterative human-in-the-loop harness refinement, GPP became the first AI system to complete Pokemon Blue, Yellow Legacy on hard mode, and Crystal without a lost battle. In the hardest stages, the agent itself began iterating on its strategy through long-context memory, surfacing emergent self-improvement signals alongside human-in-the-loop refinement. Continual Harness removes the human fully from this loop: a reset-free self-improving harness for embodied agents that formalizes and automates what we observed. Starting from only a minimal environment interface, the agent alternates between acting and refining its own p…

In brief

An agent that rewrites its own prompt, sub-agents, skills and memory mid-episode can rebuild most of a hand-engineered scaffold from a bare screenshot-and-buttons interface, but only if the underlying model is strong enough; below a capability floor the same loop makes things worse.

Continual Harness adds a Refiner that reads the last trajectory window every F steps and issues CRUD edits to the 4 harness components without resetting the environment. Evaluated on Pokemon Red and Emerald with Gemini 3 Pro, Flash and Flash-Lite, against a minimalist harness and the PokeAgent expert harness, at least 3 seeds, metric is cumulative button presses to milestone. A second loop trains Gemma-4 students via DAgger rollouts of K = 256 steps scored by a process reward model and relabelled by Gemini-3.1-pro.

On Emerald with Pro, from-scratch refinement hits 100% of milestones at a $130 median versus 98% for $215, a ~40% cost cut. Flash is high variance (80% at $42 vs 77% at $30). Flash-Lite drops from 20% to 3-13%. Evolved navigation skills close from a near-half-cost penalty to single-digit deficit against a Dijkstra oracle.

Evidence is measured with matched harness conditions, per-component ablations, and a Qwen3.5 negative control, but milestone curves are seed medians on 2 games in 1 genre, the reset-free versus reset comparison is absent, and no convergence point is established.

Use it as evidence that self-scaffolding pays off only above a model-capability threshold.

Written from the abstract by claude-opus-5 on 2026-09-15, with every figure checked against it. Not a substitute for the paper.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

No claims cite this yet.