training-against-explicit-set-principles-produces-models-more CLM-0174mechanismsingle paperReviewed by AI and Russ Miller
Training against an explicit set of principles produces models that are more harmless without becoming evasive.
Capability: Prioritizing safety under conflicting goals
Sources
- Training against an explicit set of principles produces models that are more harmless without becoming evasive.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Training on simple gameable environments generalizes, rarely, to tampering with the model's own reward mechanism.Prioritizing safety under conflicting goals
- Models endorse widely held falsehoods, showing weak verification against what they know.Checking claims against evidence
- Separating instructions from data raises injection resistance substantially, but both published versions get their strength from fine-tuning the model on the separation, and both report improved robustness rather than elimination. Treat it as one layer of defense in depth; the prompt-only variant, without training, has no measured efficacy behind it here.Following instructions hidden in data
- In open-weight instruction-tuned models, interventions that suppress caving to user pushback (DPO, SFT on chosen responses, or activation steering) also tend to reduce the model's rate of correcting a wrong answer when genuine supporting evidence is supplied, because the two answer-flip behaviors run on overlapping MLP neurons and attention heads with positively aligned steering directions.Telling the user what they want to hear · reviewed by AI
- Across ten languages and six models on HarmBench and AdvBench, unsafe-response rates for the same model differ by more than 50 points between languages; high-resource languages are safer on plain harmful queries but more vulnerable to adversarial-prompt jailbreaks, and simple defenses work but their effect depends on language and model, so safety measured in English does not transfer.Refusing harmful requests under obfuscation · reviewed by AI