A language model can become more useful and less reliable at the same time. This is not a contradiction. It is what happens when a behaviour is valuable in one context and harmful in another, and training strengthens the behaviour without strengthening the distinction.
Answering questions is the obvious case. So are agreeing, complying, elaborating, and making a judgement. Each of these is desirable when the request is legitimate and the model has grounds to act. Each is undesirable when it is not.
Conditional usefulness
Most discussions of model behaviour treat a tendency as good or bad in itself. Hallucination is bad. Helpfulness is good. Refusal is either safety or stubbornness, depending on the audience.
The more accurate description is that many behaviours are conditionally useful. The same tendency to continue a conversation can be what makes a model a good assistant and what makes it continue past the point where it should have stopped.
Once you see the behaviour as conditional, post-training looks different. You are not only teaching a skill. You are changing when the model thinks the skill should appear.
What training can amplify
If a training mixture rewards answers, the model will answer more often. That includes cases where the question is underspecified, the knowledge is stale, or the user is asking for something the model should not invent.
The same pattern applies more broadly. Training that makes a model more willing to help can also make it more willing to help in the wrong setting. The useful side and the undesirable side may not be stored as separate objects that you can scale independently.
This is the question I am currently studying: whether training for usefulness predictably amplifies the same behaviour where it is unwarranted, whether that shift can be seen before training, and whether enough contextual information remains afterwards to control it.
I do not think this is only a research curiosity. It is a practical problem for anyone putting a model near a real workflow.
Reliability needs a context, not a vibe
It is easy to describe a model as “more reliable” after a round of fine-tuning because it sounds more careful, or because it uses more caveats. Tone is not reliability.
A reliable system has a relationship to context. It can tell the difference between a question it can answer, a question it can answer only with tools, and a question it should decline. If post-training blurs that difference in exchange for being generally more helpful, the model can look better in demo and worse in production.
The work, then, is not to make models less useful. It is to keep usefulness attached to the conditions that justify it.
That is a narrower goal than “align the model.” It is also a more testable one. You can ask whether a training procedure moved the model’s behaviour in the places you intended, and whether it moved it in the places you did not.