arxiv-2608-19347 SRC-0100 · paperHype Meets Reality: Large Language Models as Mutators in Search-based Automated Program Repair of Simulink-Stateflow Models
Created: 2026-08-19 · Ingested: 2026-09-15
https://arxiv.org/abs/2608.19347(opens in a new tab)Search-based Automated Program Repair (APR) techniques rely on carefully designed mutation operators to explore the space of candidate fixes. Recent advances in Large Language Models (LLMs) suggest that generative models could replace such operators by dynamically proposing repairs. In this paper, we investigate this hypothesis in the context of Cyber-Physical Systems (CPSs) modeled in Simulink/Stateflow. We extend the state-of-the-art FlowRepair approach by replacing a subset of its mutation operators with LLM-generated mutations, enabling more flexible and expressive patch generation. We eva
In brief
Swapping hand-crafted mutation operators for LLM-generated ones in search-based program repair makes repair worse, not better, when each candidate must be validated by expensive simulation.
FlowRepair, a search-based repair tool for Simulink/Stateflow CPS models, uses 15 domain-specific mutation operators. The authors replaced 10 of them (keeping 5 structural ones) with prompted LLM mutation of state and transition labels, and ran the same benchmark of 19 faulty Stateflow models across 4 case studies (pacemaker, fridge, door, elevator), 5 runs each, 1 hour wall-clock budget per run, temperature 0.8, with GPT-5.5, GPT-5.4-mini, and GPT-4.1-mini.
FlowRepair produced plausible patches for 18 of 19 models and valid patches for 16. The LLM variants produced plausible patches for 4, 6, and 5 models and valid patches for 4 models each. Runs generated 56-713 candidates. Failures clustered on faults needing exact value synthesis, missing expression insertion, structural edits, or coordinated multi-edit fixes; the one clear LLM win (fridge_2a) had its repair pattern in the few-shot prompt.
The budget is matched on wall-clock, not on candidates evaluated, so mutation quality is not isolated from inference latency. 5 repetitions, no significance testing, 1 model family, 1,724 plausible patches manually validated. Benchmark was public, so contamination is not excluded.
Treat generative mutation as a possible complement to domain operators, not a drop-in replacement.
Written from the abstract by claude-opus-5 on 2026-09-15, with every figure checked against it. Not a substitute for the paper.
Referenced by
Claims in this catalog that draw on this source, and whether as support or counterpoint.