A lot of modern AI systems now work by giving a language model external information before asking it to answer. We use retrieval-augmented generation (RAG) to pull documents, call tools for live data, connect models to internal resources through MCP, or pass structured API responses directly into the context as JSON.
The assumption is usually simple: if the model has the authoritative information in front of it, it should use that information.
I think that assumption deserves more scrutiny.
Imagine a clinical assistant reading the record of the patient currently in front of a doctor. The chart contains that patient's medications, allergies, and latest pathology results. But the model has also seen thousands of similar cases during training. If it recommends a treatment because that treatment is strongly associated with the diagnosis, rather than because it correctly followed this patient's record, the answer can sound medically reasonable while being grounded in the wrong information.
Or imagine a financial assistant analysing a company. An API returns the latest revenue, cash position, and debt. One number needed for the analysis is missing, and the model silently fills the gap with a familiar industry assumption or a figure it remembers from somewhere else. The resulting analysis may be perfectly coherent, even though one of its foundations was never in the supplied data.
In all of these cases, the problem is not access to information.
The information is already there.
The harder question is:
Did the evidence actually determine the answer?
That question became the starting point for our paper, Twin Worlds: Equivariance-Based Abstention for Evidence-Grounded Reasoning.
Correctness is not the same as groundedness
There is a human analogy I find useful.
Imagine an open-book exam. I give a student a page of evidence and ask them to answer a question from it. But before reading the page carefully, they already have a strong hunch.
Maybe they remember something from class. Maybe the names remind them of a familiar example. Maybe they recognise a pattern and jump to the answer that usually goes with it.
If they get the answer right, I still do not know whether they actually used the page.
Language models face the same problem.
Consider P&G. Its leadership changed recently, with Shailesh Jejurikar succeeding Jon Moeller as CEO.
Now imagine a RAG system retrieves the following evidence:
Retrieved evidence
Procter & Gamble is a global consumer-goods company with brands spanning beauty, grooming, health care, fabric and home care, and family care.
During the year, P&G continued investing in product superiority, productivity, supply-chain capabilities, and execution across global markets.
The company also works with retailers, suppliers, technology providers, and other partners globally to support manufacturing, distribution, digital commerce, and consumer engagement.
Then we ask:
Who is the CEO of P&G?
The passages contain plenty of information about P&G, but they never identify its CEO.
A strong language model may still answer. One model may associate P&G with Jon Moeller. Another may know the newer answer, Shailesh Jejurikar. Another may arrive at a name through some other association.
If the answer happens to be correct, that still does not tell us whether it was grounded in the retrieved evidence, because the retrieved evidence never established the fact.
That is the distinction I care about:
A correct answer tells us that the output matches reality. It does not tell us what caused the model to produce it.
Why RAG and tools do not automatically solve this
External context is one of the most important improvements in modern AI systems.
If the model may not know something, retrieve it with RAG. If the information changes quickly, call an API. If the model needs private company data, expose it through a tool or MCP server. If the state is structured, pass JSON.
All of that helps.
But access is not the same as use.
A RAG pipeline can retrieve the right document while the model still leans on prior knowledge.
A clinical tool can return the correct patient record while the model follows a common treatment pattern that does not apply to this patient.
A finance API can return the latest numbers while the model introduces an unsupported assumption.
An MCP resource can expose authoritative internal state while the response is still shaped by a familiar association from pretraining.
Even JSON does not eliminate the problem. It makes the evidence explicit to us, but the model still has to decide how much that evidence should control generation.
That is where things get interesting.
Why this is hard to detect
We already have many ways to estimate uncertainty. We can ask the model for confidence, generate several answers, or check whether the context looks sufficient.
But none of these directly answers:
Did the reasoning actually depend on the evidence?
A model can be very confident in a memorised answer. It can reproduce the same shortcut repeatedly. It can also construct a convincing explanation around an association that was never supported by the supplied context.
The failure does not need to look like an explicit decision to ignore the evidence. A familiar company can activate a familiar executive. A diagnosis can activate a familiar treatment. A financial profile can activate a typical assumption. Those associations can then shape the rest of the reasoning.
The resulting answer may look coherent.
But coherence is not provenance.
So I became interested in a different kind of test: what happens if we weaken those familiar associations while preserving the reasoning problem?
Change the identities, keep the relationships
Suppose a passage says:
Alice Chen founded Northstar Labs in Sydney.
and I ask:
Who founded Northstar Labs?
The answer is Alice Chen.
Now imagine a structurally equivalent version:
Elian Voss founded Verathen Group in Nova.
and ask:
Who founded Verathen Group?
The names have changed, but the relationship has not.
If the model is really following the evidence, the answer should change correspondingly from Alice Chen to Elian Voss.
This is the intuition behind our method.
We replace familiar people, organisations, locations, and dates with synthetic entities of the same type while preserving the relationships that matter.
A person remains a person. An organisation remains an organisation. The logic of the evidence stays the same.
I like the principle because it is simple:
change the identities, preserve the structure.
If the reasoning survives, that is evidence that the model is following the context rather than relying mainly on entity-specific priors.
A model can be right for the wrong reason
This is the part I find most surprising.
In most evaluations, once the model gives the correct answer, we stop.
But suppose the answer came from a shortcut rather than the supplied evidence.
From an accuracy perspective, that is a success.
From a grounding perspective, we still have a problem.
Consider the clinical example again. Suppose the model recommends the right treatment, but only because that diagnosis strongly activates the treatment in its parameters. If this particular patient has an allergy that should change the decision, we need to know whether the model is actually reading the record.
Or take the financial example. A model may produce the right valuation because its unstated assumption happens to be close to reality. The same reasoning can fail badly on the next company.
A correct answer can therefore hide an unreliable dependency.
This is similar to testing whether someone understood a maths problem or simply remembered the worked example. Change the numbers and names while keeping the logic. If they can still solve it, we have stronger evidence that they understood what mattered.
Why equivariance matters
The technical idea underneath the method is called equivariance.
The intuition is straightforward.
If Alice becomes Elian, then the correct answer should also change from Alice to Elian.
So rather than asking for:
we want:
Transform the world, and the answer should transform with it.
That is useful because it tests whether the model's output tracks the structure of the evidence, not merely whether it repeats the same answer consistently.
When we transform the entities and map the answer back, the underlying result should agree.
If it does, we have evidence that the reasoning followed the supplied relationships.
If it does not, the original answer may have depended on something else.
What we found
We evaluated the approach across four knowledge-intensive benchmarks and three model backbones: GPT-5.1, LLaMA-4-Scout, and Mistral-Small.
The tasks included multi-hop question answering, evidence faithfulness, fact verification, and settings where the supplied evidence was deliberately insufficient.
I will leave the detailed tables in the paper. The main result is that checking whether answers remain equivariant under these structure-preserving transformations provides a useful signal for deciding when the model should answer and when it should abstain.
The human evaluation was also important. Annotators rated whether the transformed worlds preserved the original relationships, with an average score of 4.61 out of 5 and 99.1% of ratings at least 4. The method's answer-or-abstain decisions also agreed with the majority human judgement on evidence sufficiency 83.5% of the time.
That matters because the whole idea depends on the transformed example remaining the same reasoning problem underneath.
The one takeaway
If there is one idea I would take from this work, it is this:
Evidence in the context is not necessarily evidence behind the answer.
A model can receive the right RAG result and still rely on memory. It can call the right tool and still follow a shortcut. It can receive an authoritative patient record, MCP resource, or financial API response and still generate an answer shaped by prior associations or unsupported assumptions.
And it can even arrive at the correct answer while doing so.
For me, that is what makes evidence-grounded reasoning different from ordinary accuracy.
If I ask a model to answer from this evidence, then I care not only about whether the final answer is correct. I care whether the answer actually depends on the document, tool result, patient record, API response, or structured data I gave it.
The test we explore is simple:
change the identities, keep the relationships, and see whether the answer follows the evidence.
From the idea to the paper
This article only covers the intuition.
In the full paper, we formalise evidence grounding through equivariance, construct structure-preserving synthetic transformations, and use violations of that equivariance as an inference-time abstention signal.
We evaluate the approach across HotpotQA, MIRAGE, FaithEval, and FEVER using GPT-5.1, LLaMA-4-Scout, and Mistral-Small. We also run human evaluation, activation analysis, ablations, and error analysis to understand where the method works and where it still fails.
It was accepted to the EMNLP 2026 Main Conference.
If you are interested in the technical details, the full paper is available on arXiv.