arxiv-2609-08248 SRC-0380 · paperAgentic ML Exploration (A-MLE) for Ads Ranking
Created: 2026-09-08 · Ingested: 2026-09-22
https://arxiv.org/abs/2609.08248(opens in a new tab)Modern industrial ads ranking stacks are increasingly bottlenecked not by model capacity or training compute, but by the throughput of human ML iteration - the cycles of research, implementation, training, debugging, evaluation, and launch required to surface a single statistically significant improvement. A typical ranking stack contains numerous differentiated models with heterogeneous data, architectures, and infrastructure constraints, and each cycle takes days to weeks of senior engineer attention per model. As a result, techniques that have proven effective on one model diffuse into othe
Referenced by
Claims in this catalog that draw on this source, and whether as support or counterpoint.
- supportsWith the agent loop, skills and prompts held fixed and only the model swapped, models split into two clusters on operating a multi-step asynchronous ML workflow: Claude Sonnet 3.5-4.0, Gemini 2.5 and GPT-5 completed tasks at 93-100%, while GPT-4 scored 13% and others could not run the loop at all, hallucinating workflow ids or failing to wait for asynchronous jobs; on open-ended exploration the effect of a stressful, competitive prompt reversed by family, with Sonnet 4.0 producing the largest improvement of any configuration under stress and GPT-5 turning conservative and giving back most of its basic-prompt gains.
- supportsIn a production ads-ranking stack, an agent that chains hypothesis generation, experiment execution, result analysis and a shared knowledge store, with human checkpoints at stage boundaries, delivered several times the completed ML iterations per engineer-week of the manual baseline and a 2.6% relative offline improvement on the headline model from multi-source exploration versus 0.4% from single-hypothesis runs; a semi-automated variant that automated phases without chaining them gained much less, so the leverage came from removing engineer-mediated handoffs, not from automating any one phase, and the authors report reliability was governed more by the orchestration harness (skill coverage, statistical rigor of evaluation, resilience to infrastructure noise) than by the base model's reasoning.