instruction-tuning-8b-open-model-text-to-sql-curated-set-under CLM-0406For instruction tuning an 8B open model on text-to-SQL, a curated set of under 900 examples selected to cover every table in the target schemas and score well on question-SQL alignment reaches higher execution accuracy than pipelines trained on tens of thousands to millions of samples, and dropping the schema-coverage term from the selection objective costs more accuracy than dropping any single quality term.
Capability: Writing a reusable locator: regex, SQL, XPath
Observed on
Qwen3-8B fine-tuned with SFT then GRPO, evaluated on BIRD-Dev and Spider-Dev execution accuracy; candidate pool itself synthesized from the same benchmark databases, so coverage is.
Sources
- Validated by: BIRD-Dev EX 69.1% and Spider-Dev 88.9% with self-consistency@8 from 796 and 863 samples, versus 67.0% for DB-Explore with 16,457 samples and 63.9-64.2% for OmniSQL trained on 2.5M; ablation removing the coverage term drops SFT accuracy from 65.4% to 58.4%.. Measured on two benchmarks with one backbone, against published numbers from other systems rather than same-pipeline controls; [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- When post-training a 7-8B open model for text-to-SQL with a fixed budget of synthetic examples drawn from a large external pool, selecting examples whose schema-independent SQL skeletons are farthest from those already in the seed set (greedy K-center) rather than sampling randomly from the same pool yields a small in-domain gain and a larger cross-domain one (BIRD dev 64.9 vs 64.4, Spider Test 85.3 vs 83.4, Science Benchmark 53.9 vs 49.8).Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- On natural-language-to-SQL synthesis over multi-table databases, a model's accuracy overstates its ability whenever the same database or the same query appears in both training and test: splitting by database rather than by example cut the best system's exact-match accuracy from 33.0% to 12.4%, and accuracy falls further as the target database has more foreign keys.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- For selecting among multiple generated SQL candidates, a listwise selector given retrieved structured selection criteria distilled from training question-SQL pairs, with candidates grouped by execution result and permuted only within groups, reaches higher execution accuracy than pointwise, pairwise and multi-selector baselines on the same candidate pools while using far fewer selector calls and tokens.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- On text-to-SQL questions that are underspecified with respect to the schema, retrieving clause-level fragments (conditions, joins, calculations, groupings, output specs) from a database's own past successful SQL queries and prepending them to existing few-shot pipelines raises execution accuracy across four different pipelines, but the gain saturates and can reverse once too many candidates are retrieved, because high recall with low precision lets the generator pick a plausible-but-wrong pattern.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- On text-to-SQL benchmarks, aggregate accuracy hides a steep difficulty gradient: pre-trained-language-model-based parsers resolved roughly 80% of Spider cases overall but only about 50% of its extra-hard cases, and as natural-language understanding improved the residual errors shifted toward database-side problems — large schemas, ambiguous columns, dirty cell values — rather than language parsing.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: Matched-budget comparison where a random or quality-only subset of the same size reaches the same execution accuracy as the coverage-optimized subset, or where the small curated set loses to large-corpus training on databases held out from the selection pool. Automatic check flagged: figures not in the source: 900.