arxiv-2608-21664 SRC-0430 · paperMeasuring Activation Control in Large Language Models
Marek Mateusz Kowalski, Joshua Fonseca Rivera, Uzay Macar, David Demitri Africa
Created: 2026-08-21 · Ingested: 2026-10-17
https://arxiv.org/abs/2608.21664(opens in a new tab)Safe deployment of increasingly capable models will likely come to rely on latent-space monitoring as a complement to behavioral evaluations, especially when evaluation-aware models exhibit scheming or deception. However, if models can also control their own activations, deception could extend into the latent space itself. With this in mind, we introduce the Activation Controllability Benchmark to quantify the extent to which models can modulate their residual stream via natural-language instruction. Across model families and capability levels, we find that most LLMs can control the direction
Referenced by
Claims in this catalog that draw on this source, and whether as support or counterpoint.
- supportsAcross 25 open-weight instruction-tuned models (4B-744B), the ability to modulate a named concept's residual-stream projection on plain instruction does not track parameter count or recency — the best scorer is an 8B 2024 model and some of the newest models score lowest — while within a single model line the score rises steadily across ordinary pre-training and post-training checkpoints, so activation controllability is acquired incidentally and must be measured per model rather than inferred from scale.
- supportsOpen-weight instruction-tuned models, told in plain language to not think about a concept while transcribing a sentence, suppress that concept's residual-stream projection enough to evade linear probes, the Jacobian lens, activation oracles and natural-language autoencoders, with no fine-tuning or prompt search — detection at a 1% false-positive budget falls to roughly 5-21% under suppression while engagement is caught near ceiling.