The problem
A model can be confident and still be wrong because its memory reflects an older world.
If an answer is truly grounded in evidence, it should change when the evidence changes. But language models often hold on to associations they learnt during training, even when newer evidence says otherwise. Confidence alone does not reveal that conflict.
Ask who the CEO of P&G is, and a model may recall Jon Moeller. But Shailesh Jejurikar became CEO on 1 January 2026. The model's memory can stay the same while the world changes.
That example is simple. The same failure becomes much more serious when outdated knowledge is used in finance, healthcare, law, or other high-stakes decisions.
What I do
Twin Worlds tests whether a model is actually following the evidence.
I create a paired version of the same problem by swapping entities while preserving the underlying relationships. If the answer is grounded in the evidence, it should move with that transformation.
If it does not, the model is likely relying on memorised associations rather than the evidence in front of it. In that case, it should abstain.
The test is this: if the world changes, an evidence-grounded answer should change with it.
Cite
Vy Nguyen, Ziqi Xu, Jeffrey Chan, Estrid He, Feng Xia, Renqiang Luo, Erik Cambria, Xiuzhen Zhang. 2026. Twin Worlds: Equivariance-Based Abstention for Evidence-Grounded Reasoning. Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing.