kerckhoffss-principle ADG-0015Kerckhoffs's principle
“A system should be secure even if everything about it except the key is public knowledge.”
Also: Shannon's maxim · The enemy knows the system
Origin
Auguste Kerckhoffs, 1883, on military cryptography; Claude Shannon's 1949 restatement is "the enemy knows the system". About not relying on secrecy of design, because design leaks.
Why it should, or should not, apply to models
The tempting violation is the system prompt: deployments routinely treat its contents, and the instructions in it, as the thing that keeps the model safe. The principle predicts that will fail, and for a stronger reason than in cryptography — the "design" here is text the model has read and can be induced to repeat, and the adversary can probe the model directly and cheaply. If the principle transfers, the only defences that hold are ones that would still hold with the system prompt published: harness-level permissions, provenance checks, and sensors outside the model.
Standing: Untested
Standing counts claims a person has reviewed. Claims reviewed by AI alone are listed and marked, and do not move it.
No evidence filed either way.
Searched 2026-09-12: The catalog has claims about injection defences that work at the harness level, which are consistent with the principle but do not test it: none compares a defence that depends on prompt secrecy with the same defence after the prompt is disclosed. The jailbreak-robustness capability is proposed and has no measured claims yet. Nearest misses: tool-using-agents-restricting-capabilities-harness-level-after-untrusted-content; tool-using-agents-enforcing-injection-defense-deterministic-provenance-check-sensitive.
A break is worth more than a hold. If you have seen this fail for a model, with the setup written down, that is the most useful thing you can add here.