Why Does Fine-Tuning Promote Hallucinations? Interference Among Semantic Representations, and the Solution Is Self-Distillation SFT

What does the new paper reveal?

An ArXiv paper published April 20, 2026 illuminates the mechanism by which supervised fine-tuning increases hallucinations in large language models. The finding is counterintuitive: hallucinations are caused neither by insufficient capacity nor by so-called behavior cloning, but by a specific phenomenon called interference among overlapping semantic representations.

Definition: hallucination in LLM context means the model fabricates factually incorrect information and presents it as true, with the same confidence as correct facts.

What is fine-tuning and why is it so widespread?

Definition: fine-tuning is the process of additional training of a pretrained model on a narrower, specific dataset, with the goal of having the model master a new task or domain. Every serious team that wants to adapt an LLM to their own needs uses it — from customer support bots to medical assistants.

The problem is that fine-tuning often degrades general knowledge of the model. After an LLM “learns” something new, it forgets part of what it knew, or — worse — starts mixing old and new knowledge into fabricated claims.

What is the mechanism behind the problem?

The authors argue the model does not lose knowledge due to insufficient capacity (it is not “full”), nor due to behavior cloning (imitating another model). The real cause is more subtle:

Overlapping semantic representations. The model stores related concepts in similar parts of its internal space. When fine-tuning gradients update weights for a new domain, they inadvertently modify neighboring representations — those tied to similar but not identical knowledge.

Metaphor: if in a library you move all books on medicine, you also shift some on biology because they are on the same shelf. It is not that the library is too small — it is that the fields overlap.

What solution do the authors propose?

The main innovation of the paper is a self-distillation method for SFT (Supervised Fine-Tuning). How does it work?

Definition: self-distillation means the model learns from both new data and its own previous output. During training, gradients optimize not just for new knowledge but also regularize output-distribution drift — the distribution of responses must not move too far from the original.

In practice: every training batch includes a “reminder” of what the model knew before, preserving old knowledge while learning new.

Fine-tuning as continual learning

The authors treat SFT as a problem in continual learning — a subfield of machine learning concerned with learning new tasks without forgetting old ones. This approach opens an entire arsenal of already well-researched techniques, including elastic weight consolidation, replay buffers, and parameter isolation.

Additional solution: selective freezing

As an alternative, the authors mention selective freezing — selectively freezing parameters in scenarios where new knowledge is not needed. If you want to teach the model a new legal domain without wanting it to forget how to write email, you freeze the part of the network that controls writing.

Who benefits from this?

Every team fine-tuning LLMs for sensitive domains:

• Customer support — a bot that must not fabricate return policies
• Medical assistants — a model that must not hallucinate diagnoses
• Legal tools — a system that must accurately cite regulations
• Financial advisors — a tool that must not fabricate market data

For all of them, self-distillation SFT and selective freezing are concrete techniques that can be applied immediately with minimal changes to existing training pipelines.

Conclusion

The paper provides a clear recipe: treat fine-tuning as continual learning, not as training from scratch. Hallucinations are not an inevitable consequence — they are a symptom of coarse weight updates that do not protect existing knowledge. For professional AI teams, this finding translates the problem from a “mysterious phenomenon” into a solvable engineering task.