models-fine-tuned-b-fail-answer-b-gpt-4-recalls CLM-0091mechanismsingle paperReviewed by AI and Russ Miller
Models fine-tuned on "A is B" fail to answer "B is A", and GPT-4 recalls celebrity parents far more often than the reverse.
Capability: Not generalizing "A is B" to "B is A"
Sources
- Models fine-tuned on "A is B" fail to answer "B is A", and GPT-4 recalls celebrity parents far more often than the reverse.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- GPT-4 could reliably name a well-known celebrity's parent, but was far less reliable naming the celebrity when given the parent — the same directional-recall asymmetry demonstrated in smaller fine-tuned models, showing up in a deployed model that wasn't fine-tuned for the test.Not generalizing "A is B" to "B is A"