translating-english-descriptions-into-regular-expressions-generic-attention-based-sequence-to-sequence CLM-0259For translating English descriptions into regular expressions, a generic attention-based sequence-to-sequence model trained only on description/regex pairs matches a domain-engineered semantic parser on a small corpus (814 examples) and beats it by a wide margin once the training corpus is scaled to 10,000 pairs, with no semantic-equivalence machinery built in.
Capability: Writing a reusable locator: regex, SQL, XPath
Observed on
Short single-line regexes over a 15-rule grammar; accuracy scored by DFA equivalence; 2016-era LSTM models; gains depend on having thousands of paired examples..
Sources
- supportsNeural Generation of Regular Expressions from Natural Language with Minimal Domain KnowledgeValidated by: 10,000-pair corpus (NL-RX); a seq2seq model trained on it reached 58.2% DFA-equal accuracy on paraphrased test descriptions versus 38.6% for the prior semantic-parsing state of the art and 16.4% for a bag-of-words nearest-neighbour baseline; the corpus has 97% unique regexes against 45% in the earlier human-written set.. Measured on two datasets against two baselines, one of them the prior state of the art trained with the original authors' help. Single paper, one architecture, no seed variance reported. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- When training a sequence-to-sequence model to generate regular expressions from natural language, rewarding semantic equivalence to the reference (checked by DFA equivalence and test cases) yields more semantically correct regexes than maximum-likelihood training on the reference string, because one intent has many syntactically different equivalent regexes.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- For synthesizing regexes from user intent, neither a natural-language description nor positive/negative examples alone is sufficient — descriptions are ambiguous and small example sets underdetermine the pattern — so a system that parses the description into a structural sketch and uses it to guide example-based search solves far more tasks than either modality alone (80% vs 43% NL-only and 26% examples-only on 300+ regexes).Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- On text-to-SQL benchmarks, aggregate accuracy hides a steep difficulty gradient: pre-trained-language-model-based parsers resolved roughly 80% of Spider cases overall but only about 50% of its extra-hard cases, and as natural-language understanding improved the residual errors shifted toward database-side problems — large schemas, ambiguous columns, dirty cell values — rather than language parsing.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
- When extracting structured rules from multimodal enterprise guideline documents, wrapping the extraction in deterministic schema validation, an LLM semantic judge, and a duplicate/contradiction consistency filter over a shared rule store lowers the share of extracted rules with no matching source sentence from about 15.7% to 3.2% relative to a one-pass pipeline using the same parser and models.Stating false facts confidently · reviewed by AI
- For inducing regular expressions from a handful of labelled examples (entity spans in Czech clinical notes), prompting an LLM and then iteratively refining the pattern against match failures extracts entities more accurately than genetic-programming regex induction, with the margin largest when few examples are available.Writing a reusable locator: regex, SQL, XPath · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A domain-specific parser using regex semantic equivalence outperforming a comparably trained generic seq2seq model on a large, duplicate-free NL-to-regex corpus under DFA-equivalence scoring. Automatic check flagged: figures not in the source: 55%.