text-to-sql-questions-underspecified-respect-schema-retrieving-clause-level-fragments-conditions CLM-0494On text-to-SQL questions that are underspecified with respect to the schema, retrieving clause-level fragments (conditions, joins, calculations, groupings, output specs) from a database's own past successful SQL queries and prepending them to existing few-shot pipelines raises execution accuracy across four different pipelines, but the gain saturates and can reverse once too many candidates are retrieved, because high recall with low precision lets the generator pick a plausible-but-wrong pattern.
Capability: Writing a reusable locator: regex, SQL, XPath
Observed on
Questions whose missing knowledge (join paths, value conventions, formulas) is recoverable from a per-database pool of clean, successful historical queries; retrieval restricted to.
Sources
- supportsBIRD-History: A Benchmark for History-Driven Text-to-SQL with Fine-Grained Knowledge AnnotationsValidated by: Execution accuracy on BIRD-History (1,393 tasks, 11 databases) against each pipeline's own retriever: DAIL-SQL 40.78 to 51.33, OpenSearch-SQL 59.15 to 60.95, N-rep 54.49 to 57.79, CodeS-7B 12.92 to 18.95; retrieval F1 49.77 vs 28.69-34.88 for baseline retrievers; shot-count sweep shows EX peaking at 4 shots (61.31) with precision falling from 57.95 to 35.53 as recall rises to 83.07.. Measured on one LLM-generated, human-validated benchmark built from BIRD dev queries as the history pool, with four pipelines and ablations on shots and similarity threshold. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- On text-to-SQL benchmarks, aggregate accuracy hides a steep difficulty gradient: pre-trained-language-model-based parsers resolved roughly 80% of Spider cases overall but only about 50% of its extra-hard cases, and as natural-language understanding improved the residual errors shifted toward database-side problems — large schemas, ambiguous columns, dirty cell values — rather than language parsing.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- For instruction tuning an 8B open model on text-to-SQL, a curated set of under 900 examples selected to cover every table in the target schemas and score well on question-SQL alignment reaches higher execution accuracy than pipelines trained on tens of thousands to millions of samples, and dropping the schema-coverage term from the selection objective costs more accuracy than dropping any single quality term.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- When post-training a 7-8B open model for text-to-SQL with a fixed budget of synthetic examples drawn from a large external pool, selecting examples whose schema-independent SQL skeletons are farthest from those already in the seed set (greedy K-center) rather than sampling randomly from the same pool yields a small in-domain gain and a larger cross-domain one (BIRD dev 64.9 vs 64.4, Spider Test 85.3 vs 83.4, Science Benchmark 53.9 vs 49.8).Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- For selecting among multiple generated SQL candidates, a listwise selector given retrieved structured selection criteria distilled from training question-SQL pairs, with candidates grouped by execution result and permuted only within groups, reaches higher execution accuracy than pointwise, pairwise and multi-selector baselines on the same candidate pools while using far fewer selector calls and tokens.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- On natural-language-to-SQL synthesis over multi-table databases, a model's accuracy overstates its ability whenever the same database or the same query appears in both training and test: splitting by database rather than by example cut the best system's exact-match accuracy from 33.0% to 12.4%, and accuracy falls further as the target database has more foreign keys.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: Showing that on a comparable history-grounded benchmark, clause-level log retrieval gives no execution-accuracy gain over the pipelines' own question-similarity retrievers, or that accuracy keeps rising monotonically with retrieved-fragment count so the precision penalty does not exist.