agent-pipelines-generate-whole-domain-specific-ml-codebases-medical-imagingIn agent pipelines that generate whole domain-specific ML codebases (medical imaging) from scratch, execution/assembly validation and domain knowledge bases fix different failures: removing runtime and assembly testing mainly costs autonomy (more human interventions to debug), while removing the curated domain knowledge base mainly costs output quality (syntactically fine but domain-inappropriate pipelines) with little change in intervention count.
Ingested from a paper but not yet reviewed by a human. It is deliberately inert: it does not move any technique’s standing, does not count toward the backtest, and is excluded anywhere a claim would carry weight. Read the source before relying on it.
Capability: Generating and editing working code
Observed on
Multi-agent code generation of complete deep-learning pipelines in a specialised domain with objective end-task metrics; shown with Claude-4.5-Opus as backbone across six medical i.
Sources
- supportsTowards Fully Automated Medical Imaging Code Generation via Validation-based Context EngineeringOne ablation table on six datasets, single backbone model, Pass@3 best-of-three; component removals reported as average changes in human-intervention count and task metric. Human intervention count is a subjective measure by the authors; no independent replication.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.
Related claims
- On repository-level issue-fixing tasks (SWE-bench Lite/Verified/Pro), spending effort upstream — expanding the issue text into a structured requirement spec from retrieved repo context, then testing and refining that spec via generated code and tests — raises resolve rate over agents that consume the raw issue description, and raises the share of patches that even apply cleanly by a larger margin than it raises correctness.Generating and editing working code · unreviewed
- Tool-augmented multimodal models trained only on how to call vision tools accept tool outputs uncritically; adding supervised trajectories where tool evidence is checked against the image and re-queried, plus a reward for correction that reaches a correct answer, raises accuracy on perception-heavy benchmarks (counting, depth, spatial relations) in 3B and 7B Qwen2.5-VL, while slightly lowering it on knowledge-heavy ones.Fixing its own mistakes · unreviewed
- For long-horizon tool-calling agents (tasks needing five or more calls), embedding state-transition cues — preconditions, invariants, completion states — into the descriptions of the tools on the intended chain is what actually steers the agent's trajectory; runtime corrective text appended to tool results only patches residual drift, and the plausible user prompt alone (persona, deadlines, format constraints) does not establish the trajectory at all.Following instructions hidden in data · unreviewed
- For locally deployed open-weight models driving a stateful, dependency-ordered MCP tool server, cutting tool descriptions from full specifications (purpose, parameter semantics, constraints, failure conditions) down to one sentence each raises the fraction of calls the server rejects for every model tested, while its effect on task coverage is less consistent.Using the tools it is given · unreviewed
- In UI-to-code generation, where a rendered screenshot gives genuine external feedback, iterative self-critique still drifts below the initial draft over ten rounds, because a code edit changes the rendering non-locally and a fix for one region breaks regions that were already faithful; restricting each round to one typed, scoped repair target and carrying past targets forward as an avoid-list reduces that drift.Generating and editing working code · unreviewed
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: An ablation on similar from-scratch pipeline generation where removing the domain knowledge base raises intervention counts as much as removing execution testing, or where removing execution testing costs task quality but not autonomy. Drafted stance toward external-feedback-repair-works-only-with-real-grounding: supports -- Removing execution testing produced the largest jump in required human intervention, consistent with grounded external signals being what drives autonomous repair. Proposed technique, not catalogued: validation-gated context propagation between agents.