training-sequence-to-sequence-model-generate-regular-expressions-natural-language-rewarding CLM-0258mechanismsingle paperReviewed by AI
When training a sequence-to-sequence model to generate regular expressions from natural language, rewarding semantic equivalence to the reference (checked by DFA equivalence and test cases) yields more semantically correct regexes than maximum-likelihood training on the reference string, because one intent has many syntactically different equivalent regexes.
Capability: Writing a reusable locator: regex, SQL, XPath
Observed on
Seq2seq NL-to-regex generation on three public datasets; depends on having a decidable equivalence oracle for the target language..
Sources
- Validated by: Semantic accuracy against prior state-of-the-art syntax-based models on three public NL-to-regex datasets; the paper reports superiority but the excerpt gives no numbers.. Measured on three public NL-to-regex datasets against prior state of the art; the abstract gives no effect sizes, no ablation separating the DFA oracle from the test-case components, and the equivalence oracle is specific to regular languages.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For translating English descriptions into regular expressions, a generic attention-based sequence-to-sequence model trained only on description/regex pairs matches a domain-engineered semantic parser on a small corpus (814 examples) and beats it by a wide margin once the training corpus is scaled to 10,000 pairs, with no semantic-equivalence machinery built in.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- For synthesizing regexes from user intent, neither a natural-language description nor positive/negative examples alone is sufficient — descriptions are ambiguous and small example sets underdetermine the pattern — so a system that parses the description into a structural sketch and uses it to guide example-based search solves far more tasks than either modality alone (80% vs 43% NL-only and 26% examples-only on 300+ regexes).Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- For inducing regular expressions from a handful of labelled examples (entity spans in Czech clinical notes), prompting an LLM and then iteratively refining the pattern against match failures extracts entities more accurately than genetic-programming regex induction, with the margin largest when few examples are available.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- On text-to-SQL benchmarks, aggregate accuracy hides a steep difficulty gradient: pre-trained-language-model-based parsers resolved roughly 80% of Spider cases overall but only about 50% of its extra-hard cases, and as natural-language understanding improved the residual errors shifted toward database-side problems — large schemas, ambiguous columns, dirty cell values — rather than language parsing.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- When NL-to-PDDL problem generation is scored by operational acceptance (parses, conforms to domain, a planner finds a plan, VAL validates it, an LLM critic accepts), the score overstates faithfulness to the described task, and the gap widens for PDDL 2.1 temporal and numeric problems where operationally accepted instances can almost never reconstruct the curated reference encoding.Whether the measurement made the finding · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A controlled comparison on the same datasets where semantic-reward training matches or trails MLE training on semantic accuracy.