arxiv-2606-22676 SRC-0320 · paper

Skin-Deep: A Geometric Diagnostic for Alignment Fragility in Large Language Model Representations

Created: 2026-06-21 · Ingested: 2026-09-19

Alignment tuning is meant to make harmful-request refusal robust, yet this safety behavior can be erased by a small set of benign fine-tuning examples. This is a deployment risk for open-weight models because a checkpoint can pass refusal tests at release time and later lose refusal under low-cost downstream fine-tuning. Prior work has established these refusal failures, but existing studies do not show how to detect this fragility in the aligned model itself before an attack or fine-tuning intervention is run. We introduce Skin-Deep, a geometric diagnostic that detects alignment fragility dir

In brief

Refusal in aligned models sits on a small number of residual-stream directions, and which direction actually carries refusal differs by model, so variance dominance does not predict behavioral importance. Skin-Deep measures this geometry on the unmodified checkpoint and summarizes it as a Geometric Fragility Score (GFS), computed before any fine-tuning.

The method contrasts aligned and version-matched base covariances via contrastive PCA on 500 harmful requests (AdvBench, HarmBench, BeaverTails) and 500 length-matched benign instructions (Alpaca, OASST), across 21 instruction-tuned models at 3B-32B in 5 overlapping cohorts.

Split-sample cPCA directions separate held-out prompts (core models: held-out Cohen's d 2.71-3.23, probe accuracy 0.904-0.959), with a minimum held-out effect of 1.69 across a 14-model check. Ablating 1 direction at the peak layer lowers refusal in 4 of 6 models, but the effective direction varies: PCA-PC1 for Llama (0.14 to 0.02), Arditi for Qwen (0.86 to 0.68) and SOLAR (0.22 to 0.06); random directions stay near baseline. Benign LoRA on Alpaca subsets drives nearly all models to full harmful compliance at n=200; Gemma, the lowest-GFS initially safe model, ends at mean 0.68 (range 0.48-0.80).

Ablation specificity is well controlled (random and non-leading PCA controls, Wilson intervals, prompt-shuffle CKA null). The GFS-to-fragility link rests on 1 endpoint distinction in a small heterogeneous cohort with 3 seeds for some models only, not a calibrated forecast. English prompts, open-weight dense models, 1 harmless-data protocol; no multilingual or MoE test.

Treat GFS as a hypothesis-generating pre-screen, not a fragility predictor.

Written from the abstract by claude-opus-5 on 2026-10-24, with every figure checked against it. Not a substitute for the paper.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsRefusal learned by alignment tuning lives in a low-rank subspace of hidden-state activations that recurs across model families; ablating directions in that subspace lowers harmful-request refusal, and a scalar computed from the subspace's geometry before any fine-tuning (the Geometric Fragility Score) picks out which initially safe model will keep the most refusal after small benign LoRA fine-tuning.