A language model that always answers can look more capable than one that sometimes refuses. In practice, the opposite is often true. In domains where an incorrect answer is expensive — payments, medicine, law, operations — a fluent guess is not a minor failure. It is a failure of judgement.
Reliability, in that setting, is not only a matter of being right more often. It is also a matter of knowing when the available knowledge is too thin, too conflicted, or too context-dependent to support an answer.
The cost of sounding sure
Most evaluation still rewards coverage. Benchmarks ask a question and score the response. A model that says “I don’t know” is usually marked wrong, even when abstention is the honest outcome.
That incentive leaks into product design. Teams add retrieval, longer context, or more tools, then measure whether the system produces an answer. The missing metric is whether the system should have spoken at all.
This is especially visible in financial systems. A payment platform can tolerate latency more easily than it can tolerate a confident, incorrect explanation of a settlement failure. Users will wait. They will not forget being told the wrong thing with certainty.
Two reasons not to answer
Abstention is often treated as a single behaviour: the model is unsure. In practice there are at least two different situations.
The first is conflict. The model has internal support for more than one answer, and those answers are not compatible. This can happen when sources disagree, when a policy changed, or when a term means different things in different jurisdictions.
The second is insufficiency. The model does not have enough support for any answer that would be safe to use. The honest output is not a hedged paragraph. It is a refusal, a request for more information, or a hand-off to a process that can verify.
These two cases look similar from the outside. They are not the same problem to solve.
After the fact is late
Many current methods detect hallucination after a response has already been generated. They sample several times, ask the model to critique itself, or look at how much the output varies.
Those signals are useful. They are also late. In a production system, the expensive event is often the generation itself: a customer has already seen the draft, a downstream workflow has already started, or a human has already been primed by a fluent explanation.
A more useful question is whether the model can decide to abstain before it commits to an answer. That requires looking at the structure of what the model knows, not only at the style of what it says.
What this changes in product design
If abstention is part of reliability, then the interface has to treat “I don’t know” as a first-class outcome. That means:
- making refusal legible, rather than burying it in hedging language
- routing the user to a verification step instead of filling the gap with prose
- measuring false answers separately from unanswered questions
It also means being careful with post-training. If a model is rewarded for being helpful, it will learn to answer. Helpfulness is not the same as reliability. A system can become more useful in ordinary cases and more dangerous in the cases where it should have stopped.
I have been working on this problem in research, including work on deciding when a model should abstain because its internal knowledge is conflicting or insufficient. The technical details belong in papers. The product implication is simpler: a reliable system needs a way to remain silent.
Silence is not a lack of intelligence. In the systems I care about, it is often the correct action.