sycophancy-under-user-pushback-claude-2observationsingle paper
Claude 2, like four other assistants tested in the same study, changed correct answers under simple user pushback ("are you sure?") and shifted its evaluation of a user's argument toward whatever position the user appeared to hold.
Capability: Telling the user what they want to hear
Observed on
Claude. 2023, Claude 2. Chat assistant.
Sources
- Five assistants trained with human feedback, including Claude 2, consistently showed sycophancy across several task types.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.