llm-generated-hls-kernels-targeted-real-fpga-boards-two-stages CLM-0592For LLM-generated HLS kernels targeted at real FPGA boards, the two stages of failure respond to different interventions: an evolutionary search over simulation-validated kernel candidates mainly raises synthesizable and executable rates, while a C/C++-level functional verify-and-repair loop mainly raises functional correctness and barely moves synthesizability, so both are needed for end-to-end success.
Capability: Generating and editing working code
Observed on
Natural-language-specification-to-executable HLS generation on 78 HLS-Eval tasks, Vitis HLS 2022.2 and a Xilinx U200 board; ablation run on one 30B open model..
Sources
- Validated by: Against zero-shot pass@10 on 78 HLS-Eval tasks across five models: +16.9% synthesizable, +26.7% executable, +30.6% functional correctness on average; ablation on Qwen3-Coder-30B-A3B isolates each module (Eoh +17/+15/+9, Reflect -1/+2/+7, full +19/+22/+23).. Measured on one benchmark (78 filtered HLS-Eval tasks) and one FPGA platform; the component ablation uses a single backbone, so the attribution of gains to modules is not shown to hold across models. Zero-shot baseline is pass@10, roughly matching the search budget. No hardware-build cost-matched baseline.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For LLM-generated HLS kernels targeting real FPGA boards, simulation and synthesis success overstates deployability, and the two stages are fixed by different interventions: an evolutionary search over simulated candidates raises synthesizable and executable rates while barely raising functional correctness, whereas C/C++-level verify-and-repair raises functional correctness while leaving synthesizability flat.Generating and editing working code · reviewed by AI
- On simple HLS kernel generation tasks (85 designs from CHStone, MachSuite, Polybench, Rosetta, C2HLSC), giving a modest open-source model (gpt-oss-120b) a shell with a C++ compiler for self-verification and iterative refinement saturates the benchmark at pass@10 (100% through synthesis) and exceeds 90% at pass@1, so this class of task no longer separates models; repeated sampling helps the weaker model far more (+60 points synthesis pass rate at k=10 for gpt-oss-20b vs +11 for the 120b).Generating and editing working code · reviewed by AI
- In search-based automated program repair of Simulink/Stateflow models, replacing hand-crafted domain-specific mutation operators with an LLM that proposes label rewrites from static component context lowers repair coverage under a fixed wall-clock budget, because unconstrained generative edits widen the candidate space while each candidate costs an expensive simulation and the generator gets no behavioural feedback about why a patch failed.Generating and editing working code · reviewed by AI
- For LLM program-repair workflows built on self-correction, generated-knowledge or multi-agent scaffolding, the gain over plain prompting is unstable across benchmarks and can turn negative once extrinsic defect information (error messages, triggering tests, bug comments) is withheld, while simply putting the error message plus the failing test into the prompt yields more correct repairs than the scaffolds do.Fixing its own mistakes · reviewed by AI
- For LLM agents that must compose and then execute domain tool workflows, training separate adapters for the two roles splits the gains cleanly: supervised fine-tuning of the planner on reference tool sequences mainly improves workflow-order fidelity, while node-level RL on argument-match rewards mainly improves tool-argument grounding and final-answer quality, and combining both beats either on a 9B backbone.Planning a sequence of actions before acting · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: An ablation on comparable HLS tasks where the functional verify-and-repair module raises synthesizable/executable rates as much as the evolutionary search does, or where evolutionary search alone raises functional correctness as much as the repair loop. Drafted stance toward llm-generated-hls-kernels-targeting-real-fpga-boards-simulation-synthesis: supports -- The ablation reproduces the division of labour: evolutionary search lifts synthesizable/executable rates far more than correctness, while the verify-and-repair module lifts correctness while leaving synthesizability flat or slightly lower.