arXiv:2607.14791: Transcoders Reveal How Deception Forms Within the Internal Circuits of the Qwen3-4B Language Model

How to “see” a lie inside a language model?

Darius Lim, Nathan Leow, and Xin Wei Chia use per-layer transcoders (PLT) — an interpretability tool that decomposes each layer’s internal activations of a model into understandable, individual features — on Qwen3-4B, a language model with 4 billion parameters. The goal is to track deceptive behavior at the level of “circuits,” the internal pathways through which information travels from input to output.

A vocabulary of deception

The team builds attribution graphs (graphs that show how individual features contribute to the model’s final output) and within them identifies a distinct “vocabulary” of deception-related features — a set of internal signals that consistently track deceptive statements. Feature steering, meaning manually steering the activation of those features, produces predictable shifts between deceptive and non-deceptive responses of the same model.

Why it’s interesting

The result shows that deception isn’t diffuse but arises from specific, isolated mechanisms within the network. This opens the door for transcoders as a tool for behavioral monitoring (tracking model behavior in real time) and early detection of safety risks before a deceptive response is even generated.