black-box-jailbreak-defense-test-time-abstracting-each-successful-attack CLM-0551mechanismsingle paperReviewed by AI
For black-box jailbreak defense at test time, abstracting each successful attack into a rule that names the structural wrapper (nested role-play, code obfuscation, flipped text) rather than the harmful topic lets one rule induced from a single prompt cover the rest of that attack family, so attack success falls to near zero after the first batch of ~20 streamed prompts without any parameter updates.
Capability: Improving its own scaffolding from failures
Observed on
Prompt-level black-box jailbreak families (DeepInception, CodeChameleon, ReNeLLM, FlipAttack) on 7-8B open models and a Gemini-class API model; single-turn, not agentic or multi-tu.
Sources
- Validated by: ASR-gpt (external GPT-4o-mini judge, threshold 7) and ASR-rej on AdvBench across four attack families and three models, against no-defense, safety prompt, Self-Reminder and AutoDefense baselines; MMLU/GSM8K utility stays within ~2 points; ablations removing triggering, enforcement or reflection each raise ASR.. Measured on AdvBench (520 prompts) across four attack families and three models with a cold-start online protocol, against four baselines; ASR drops to roughly 0.1-3.5% from 5-58% undefended. [truncated]
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Rewriting a harmful request in an emotional or narrative style (fearful, curious, compassionate) while keeping its intent raises jailbreak success by up to 57 percentage points across 16 instruction-tuned models, contextual rewrites beating fixed templates, and the vulnerability does not shrink with model scale; a preprocessing step that has a second model strip stylistic cues cuts the success rate substantially.Refusing harmful requests under obfuscation · reviewed by AI
- Prompt injection defenses that report near-zero attack success on short-context benchmarks lose most of that protection when the injected instruction sits inside a document of thousands to tens of thousands of tokens: fine-tuned separation defenses and detect-localize-remove pipelines still let a large share of injections through on paper review, resume screening, code review and email threads.Following instructions hidden in data · reviewed by AI
- Safety realignment after a poisoned fine-tune is brittle in three ways: existing defenses (SSRD, RESTA, soft-SFT, rollback) often destroy the downstream skill they were meant to keep, lose more than 30% of their effect when the defender's prompt template differs from the attacker's, and a realigned model can be re-jailbroken by switching the system prompt; distilling on-policy from two frozen teachers, the original model for refusal and the fine-tuned model for the task, routed per token, cut attack success to about 2% under the defender's template while holding task score near pre-attack level.Retaining skills and safety after fine-tuning · reviewed by AI
- When extracting structured rules from multimodal enterprise guideline documents, wrapping the extraction in deterministic schema validation, an LLM semantic judge, and a duplicate/contradiction consistency filter over a shared rule store lowers the share of extracted rules with no matching source sentence from about 15.7% to 3.2% relative to a one-pass pipeline using the same parser and models.Stating false facts confidently · reviewed by AI
- In 7-8B open-weight instruct models (Llama-3.1, Mistral-v0.3, Qwen2.5), the drop in a token's probability when a contiguous mid-layer MLP span is zero-ablated is near zero for function words, elevated for knowledge-bearing tokens, and abnormally spiked for hallucinated tokens, so this internal attribution signal — approximable by a small probe from one forward pass — can be used in beam scoring to raise factuality on open-ended and long-form generation, while leaving strict short-answer EM/F1 unchanged or worse.Stating false facts confidently · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A held-out jailbreak family whose wrapper differs in surface form but shares structure shows no reduction in ASR from an induced rule, or per-round ASR fails to drop after the first batch under a cold-start stream.