arxiv-2305-18153 SRC-0307 · paperDo Large Language Models Know What They Don't Know?
Created: 2023-05-29 · Ingested: 2026-09-19
https://arxiv.org/abs/2305.18153(opens in a new tab)Large language models (LLMs) have a wealth of knowledge that allows them to excel in various Natural Language Processing (NLP) tasks. Current research focuses on enhancing their performance within their existing knowledge. Despite their vast knowledge, LLMs are still limited by the amount of information they can accommodate and comprehend. Therefore, the ability to understand their own limitations on the unknows, referred to as self-knowledge, is of paramount importance. This study aims to evaluate LLMs' self-knowledge by assessing their ability to identify unanswerable or unknowable questions
In brief
Language models can flag questions that have no answer, but less reliably than people, and the ability grows with scale, instruction tuning, and in-context examples.
SelfAware pairs 1,032 unanswerable questions drawn from Quora and HowStuffWorks across 5 categories (no scientific consensus, imagination, subjective, too many variables, philosophical) with 2,337 answerable questions from SQuAD, HotpotQA and TriviaQA chosen to be semantically nearest. Uncertainty in free-text responses is detected automatically by SimCSE similarity to a reference set of hedging sentences, threshold 0.75, scored as F1 with unanswerable as the positive class, across 20 models including GPT-3, InstructGPT, LLaMA, Alpaca, Vicuna and GPT-4.
GPT-4 scores 75.47% F1 against a human benchmark of 84.93%. F1 rises with parameter count under all 3 input forms; ICL lifts davinci by 27.96% over direct prompting; Vicuna-13B beats LLaMA-65B. On answerable questions accuracy climbs from 2.48% (text-ada-001) to 10.61% (text-davinci-001) to 42.64% (GPT-4).
Measured, with ablations on model size, tuning, and input form, so the three factors are separated. The human baseline rests on 2 volunteers and 100 samples, GPT-4 on 100 samples, and the uncertainty detector's reference sentences come only from GPT-3/InstructGPT outputs, so hedging phrased differently by other models may be missed. Nothing beyond direct, instruction and ICL prompting was tested.
Use it as a cheap abstention benchmark, with the caveat that the metric is a similarity heuristic.
Written from the abstract by claude-opus-5 on 2026-10-17, with every figure checked against it. Not a substitute for the paper.
Referenced by
Claims in this catalog that draw on this source, and whether as support or counterpoint.