self-harness-loopLet the agent revise its own harness, regression-gated
Cluster failed traces into signatures, have the same fixed model propose a few minimal harness edits, and promote only edits that improve one split without degrading another.
Three stages, model weights untouched. Failed execution traces are grouped into verifier-grounded failure signatures. For each signature the same base model proposes K minimal, distinct edits to prompts, tools or runtime policy. Each edit is evaluated on two splits and promoted only if it improves one without degrading the other. The loop is the harness-level version of a ratchet: failures become permanent structure, and nothing is kept that a check did not pass. Its known weakness is that the promotion gate reads the held-out split, so that split stops being a clean test of generalisation — a truly external held-out set the loop never touches is the missing piece.
Requires: A verifier that can label a trace as failed, a benchmark that can be split, and the budget for many full re-evaluations per promoted edit.
Contexts: Coding agent, Autonomous agent
Does it work?
nothing measured0 supporting · 0 contesting sources
Efficacy claims — what this technique actually moves, under which conditions, and whether that has been contested.
No efficacy claim filed yet. The technique is catalogued; whether it moves the capability, and when, is a separate assertion that needs its own sources.
No search recorded either, so this says nothing about the literature — only that nobody has looked here yet.
Code
No repository linked yet. Contribute one.