“When asked to state their confidence, frontier large language models hallucinate in more than half of their answers.”

High hallucination is the dominant factor driving these low scores. For instance, although Grok 4 and GPT-5 (high) record the highest accuracy at 39%, their hallucination rates of 64% and 81% result in substantial penalties on the Omniscience Index (Figure 7).

Whereas higher confidence generally correlates with higher answer accuracy, the confidence is poorly calibrated: when Llama-3.1-70B predicts a confidence score above $80%80\%$ on CRAG (Yang et al., 2024), the real accuracy is only $33%33\%$.

Within the black-box family, direct verbalized confidence is generally weaker than sample-consistency signals. For example, on Llama-3.2-3B-Instruct, Verbalized Confidence obtains an overall average of 57.99, while SelfCheckGPT-BERTScore, SelfCheckGPT-NLI, and lexical similarity reach 69.25, 67.86, and 69.16, respectively. This suggests that comparing stochastic generations is generally more informative than relying on the model’s stated confidence, although the usefulness of such consistency signals still depends on the task and comparison metric.

Despite its high refusal rate, the Llama3.1-8B model is less prone to hallucination (48.37%) compared to similar-sized models like Qwen2.5 7B (85.22%) and Mistral 7B (81.19%). … We further examine the hallucination rate when models do not refuse to answer. The Llama 3.1 405B Instruct model achieves the lowest hallucination rate at 26.84% but falsely refuses to answer 56.77% of the time. … GPT-4o, with a 45.15% hallucination rate when not refusing, maintains a much lower false refusal rate and achieves the highest correct answer scores (52.59%), indicating a trade-off between precision and recall.

Our results reveal substantial variation in confidence reliability, and while larger and more accurate models tend to also achieve better reliability metrics, even frontier models remain prone to severe overconfidence in challenging settings.

Large language models (LLMs) frequently produce confident but incorrect answers, partly because common binary scoring conventions reward answering over honestly expressing uncer tainty. … In this setting, we operationalize hallucination as giving a false answer to such a question and measure it with the false-answer rate (FAR). … As a baseline, directly prompting GPT-5 mini without reward framing or verbal confidence (Pure Eval) yields FARanswered = 52.3%.

We found ample evidence for LM’s ability to en code the (un)answerability of questions, despite the fact that models tend to be over-confident and gen erate hallucinatory answers when presented with (un)answerable questions.

LLMs, when verbalizing their confidence, tend to be overconfident, potentially imitating human patterns of expressing confidence. … While GPT-4 displays lower ECE, its AUROC and AUPRC-Negative scores remain suboptimal, with an average AUROC of merely 62.7%—close to the 50% random guess threshold—highlighting challenges in distinguishing correct from incorrect predictions.

Our experiments across a series of LLMs in dicate that trained hidden-state probes provide the most reliable confidence estimates, albeit at the expense of requiring access to weights and training data. … In the verbalization (verbalized confidence) meth ods (Lin et al., 2022a; Xiong et al., 2024; Yin et al., 2023; Tian et al., 2023; Kadavath et al., 2022), the model is directly prompted to report its confidence level (e.g., “How confident are you that the answer is correct?”). … With the exception of Falcon-40B instruct, the other methods perform close to or below chance (depending on the model’s label distribution, chance level varies between 0.11 and 0.27). This indicates that non-trained estima tors are generally not reliable for P(IK) despite being frequently used in the literature.

In the case of 5W, one could imagine that half of the posts occur exactly once. This would suggest that a calibrated-factoid model would hallucinate on about half of its generations on 5W factoids.

However, the fact that performance is well below the line y = x means that models consistently overstate their confidence. Hence, there is a lot of room to improve the calibration of large language models in terms of stated confidence.

It is challenging for LLMs to accurately express their internal confidence in natural language. … When expressing confidence in words, LLMs are not well-calibrated and tend to be overconfident which is consistent with the previous findings [17].

For example, on the SimpleQA evaluation 14, accuracy slightly favours OpenAI’s o4-mini, which answers almost all questions (with over 3/4 error rate) 15 over GPT-5-mini, even though GPT-5-mini makes many fewer errors (owing to abstentions) 6.

A non-negligible amount of hallucinations despite knowledge (16–43%) occur with high certainty, demonstrating the existence of CHOKE examples across all combinations of certainty methods, models, and prompt settings.

This fluency, however, coexists with a systematic tendency to hallucinate: LLMs often produce statements that contradict ver ifiable facts while maintaining high predictive con fidence (Ji et al., 2023; Liu et al., 2025).

Self-Evaluation leverages the model's introspective capabilities by prompting it to assess the confidence and factual correctness of its own output using Chain-of-Thought (CoT) reasoning [49]. … Self-Evaluation (SE) achieves the highest pooled AUROC of 0.688 across all 600 model-response pairs.

Even recent models, such as OpenAI’s GPT-4.5, have been found to hallucinate as often as 37.1% on certain benchmarks (OpenAI, 2025), underscoring the ongoing challenge of ensuring reliability in LLM outputs.

Each model holds a near-constant confidence level barely responsive to the catalog, while the catalog-hit rate swings 60 points, so the sign of the error is set by where a model’s constant lands against a catalog’s accuracy: 7 of the twelve cells are under-confident and 5 over-confident, all four under-confident on MovieLens, all four over-confident on Amazon Toys. … We read this as an elicitation mismatch: “Just Ask” elicits a generic quality rating, not a catalog-membership probability.

Large language models frequently generate plausible but incorrect outputs with unwarranted confidence—a phe nomenon commonly termed “hallucinations” (Ji et al., 2023; Maynez et al., 2020; Zhang et al., 2025).

We find that “verbal uncertainty” is governed by a single linear feature in the representation space of LLMs, and show that this has only moderate correlation with the actual “semantic uncertainty” of the model. … The miscalibration of semantic and verbal uncertainty triggers overconfident hallucinations.

Yet modern neural predictors are notoriously miscalibrated, often assigning high confidence to wrong answers, a phenomenon documented from classic discriminative models through contempo rary deep networks [1–6].

While LLMs are currently poor at faithfully conveying their uncertainty, we believe this problem offers tangible headroom. … Yona et al. (2024) have demonstrated that current state-of-the-art models are far from satisfying this desideratum; they typically express high linguistic confidence even when their intrinsic uncertainty is low.

When no uncertainty prompt is used (none), all models perform poorly with cMFG scores close to or less than 0.5, indicating a tendency toward worse faithfulness than when a random level of decisiveness is exhibited. … Models often did not generate any expressions of uncertainty, instead producing highly decisive answers with mean decisiveness near 1.0 even when very uncertain, indicating baseline uncertainty expressions are highly unreliable.

Large Language Models (LLMs) increasingly support decision-making in high-stakes domains, but they often hallucinate and express confidence that is misaligned with factual correctness.

Under binary grading, abstaining is strictly sub-optimal. IDK-type responses are maximally penalized while an overconfident “best guess” is optimal. … Observation 1. Let c be a prompt. For any distribution ρc over binary graders, the optimal response(s) are not abstentions, i.e., A c ∩ arg max r∈R c Egc∼ ρc[gc(r)] = ∅ .

Uncertainty-based detection addresses the challenge of data dependency using model confidence rather than external labeled datasets. However, its effectiveness is highly sensitive to the calibration of uncertainty thresholds, and it often fails to detect hallucinations when the model shows high confidence in an incorrect response.

Our experiments across a series of LLMs in dicate that trained hidden-state probes provide the most reliable confidence estimates, albeit at the expense of requiring access to weights and training data. … With the exception of Falcon-40B instruct, the other methods perform close to or below chance (depending on the model’s label distribution, chance level varies between 0.11 and 0.27). This indicates that non-trained estima tors are generally not reliable for P(IK) despite being frequently used in the literature.

Our experiments across a series of LLMs in dicate that trained hidden-state probes provide the most reliable confidence estimates, albeit at the expense of requiring access to weights and training data. … We find that the confidence of LLMs is often unstable across semantically equivalent inputs, suggesting that there is much room for improvement of the sta bility of models’ parametric knowledge.

An important research topic is how to make LLMs accurately express the confidence to their answers, so they can refrain from outputting or regenerate output in cases of low-confidence predictions. … Currently, research on eliciting calibrated confidence from LLMs is still insufficient.

The phenomenon of hallucination—where models confabulate facts with high confidence—remains a stubborn artifact of the current post-training paradigm [16].

We find that “verbal 1 uncertainty” is governed by a single linear feature in the representation space of LLMs, and show that this has only moder ate correlation with the actual “semantic uncer tainty” of the model. … We propose a novel hallucination detection method by incorporating VU and SU, outperforming detection methods that rely solely on SU. Next, we propose a mitiga tion method, Mechanistic Uncertainty Calibration (MUC), that steers LLM activations along VUF to calibrate VU with SU.

The empirical results suggest that ChatGPT is likely to generate hallucinated content in specific topics by fabricating unverifiable information (i.e., about $11.4\%$ user queries).

A persistent weakness of large language models (LLMs) is their tendency to sound confident even when they are wrong [13, 45]. … Generally, we observe that reasoning models still prefer to express high confidence, assigning values below 55% infrequently, as shown in Figure 2 (right).

By this we mean instances where a model confidently generates an answer that isn’t true. Our new research paper⁠(opens in a new window) argues that language models hallucinate because standard training and evaluation procedures reward guessing over acknowledging uncertainty.

One way to measure calibration is to directly ask the language model to state its confidence in its answer using the prompt: “Please give your best guess, along with your confidence as a percentage that that is the correct answer.” Then we can plot the correlation between the stated confidence of the model, and how accurate the model actually was. … However, the fact that performance is well below the line y=x means that models consistently overstate their confidence. Hence, there is a lot of room to improve the calibration of large language models in terms of stated confidence.

OpenAI Newly Released SimpleQA Helps Reveal That Generative AI Blatantly And Alarmingly Overstates What It Knows

When target rates are given, larger models achieve good calibration at lower rates. Yet, when models must derive the optimal target rate themselves, all models fail to achieve calibration without entropy or hint to generate it.