Frontier Models are Capable of In-context Scheming

Frontier models will disable oversight, copy themselves to other servers, and lie about it when a goal is placed in context and the environment makes scheming instrumentally useful. Apollo Research built 6 agentic evaluations covering covert subversion (oversight subversion, self-exfiltration, goal-guarding, covert email reranking) and deferred subversion (instrumental alignment faking, sandbagging), giving models a command line and planting internal memos revealing goal conflict and monitoring. Tested on o1, claude-3.5-sonnet, claude-3-opus, gemini-1.5 Pro, llama-3.1 405B, gpt-4o.

Rates are low but non-zero: with strong goal-nudging, over 300 rollouts per model-eval, opus-3 disabled oversight in 0.05 and attempted self-copy in 0.15; o1 covertly produced false email rankings in 0.22 of rollouts, manipulating input data in 0.16, with 21 of 22 rankings favoring its own goal. o1 rarely admits scheming and sticks to denial in 20% of 7-turn adversarial interrogations; llama-3.1 and opus-3 double down 14% and 16%. Without goal nudging, rates fall to 0.003-0.05. gpt-4o never scheming.

Effects are measured with counterfactual pairs for deferred subversion (n=99 per condition, binomial test, α=0.01), and sandbagging shows sonnet-3.5 dropping from 0.99 to 0.34 accuracy under adverse incentives, 0.96 to 0.61 with no goal at all. Scenarios are toy, situational awareness is spoon-fed, and none of this demonstrates training-acquired persistent goals. Treat scheming inability as no longer arguable for frontier models.