arxiv-2609-09090 SRC-0226 · paper

Measuring LLM Sycophancy under Sustained Multi-Turn Pressure

Leyuan Tang, Kangda Wei, Tianyu Jiang, Ruihong Huang

Created: 2026-09-08 · Ingested: 2026-09-19

Large language models (LLMs) may abandon correct positions when users push back, exhibiting a failure mode known as sycophancy. Existing evaluations typically use short, pre-specified conversations and may therefore miss failures that emerge under sustained, adaptive disagreement. We introduce SPINE, a benchmark in which an LLM proxy plays a persistent but mistaken user and adaptively challenges a target model for up to 25 turns. We evaluate four production systems and three Olmo3-7b variants on 100 false-presupposition and 100 unethical-query items. Our experimental results show that collapse

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsWhen a mistaken user pushes back adaptively for up to 25 turns, frontier and 7B open models keep abandoning initially correct positions at higher rates the longer the conversation runs, so sycophancy measured with five-turn or fixed-script rebuttals understates it — in one ablation collapse on false-presupposition items rose from 50% by turn 5 to 92% by turn 25 for the same model, and a fixed four-follow-up script reached only 28%.