Anthropic Develops an 'Off Switch' for Dangerous Knowledge: GRAM Isolates Dual-Use Capabilities into Removable Modules

A New Approach to Controlling Dangerous Knowledge

Every sufficiently powerful language model is simultaneously a potential tool for misuse: the same text about pathogens that helps a researcher can also help an actor developing a biological weapon. The standard approach to this problem — filtering training data or applying post-hoc unlearning — has never fully satisfied the demands of safety researchers.

Anthropic, in collaboration with AE Studio, has published a research paper proposing a fundamentally different approach: rather than removing dangerous knowledge after training, it is isolated during training into dedicated, removable model components.

What Is GRAM and How Does It Work?

GRAM (Gradient-Routed Auxiliary Modules) adds a set of neural modules to each layer of the Transformer architecture — one module for each dual-use knowledge category. The key innovation lies in the learning process: when the model encounters dual-use content (for example, text about virus synthesis), the general model parameters are temporarily frozen. Only the relevant module — the “virology module,” for instance — updates its weights from that material.

The result is a clean separation: general knowledge and reasoning capabilities remain untouched in the base parameters, while dual-use knowledge lives exclusively within its dedicated module. Deleting a module removes the corresponding capabilities approximately as effectively as if the model had never been trained on that data — which is the central claim of the research paper.

Three Phases of Experimental Testing

The researchers took a methodical approach across three separate validation phases.

The first phase used synthetic data — small models trained on children’s stories. The goal was to prove the basic principle: do separate gradient pathways actually exist, and does module deletion work?

The second phase moved to realistic data: web text, code, and scientific papers, with four dual-use categories — virology, cybersecurity, nuclear physics, and specialized programming languages. Experiments confirmed that module deletion effectively removes capabilities in each category without degradation on general tasks.

The third phase tested scalability across seven model sizes, from 50M to 5B parameters. The key finding: GRAM’s effectiveness grows with model size — the gap between the version with the module and the version without it grows larger as models scale.

Why Does This Matter for AI Safety?

Existing methods — data filtering and unlearning — share a common flaw: all are susceptible to knowledge recovery attempts. With sufficiently persistent fine-tuning or adversarial attacks, capabilities can be partially restored.

GRAM is more resistant to these attempts, because dual-use knowledge is never embedded in the part of the model that subsequent training could reactivate. Recovery would require retraining the modules from scratch — an expensive operation that demands access to the original data and training infrastructure.

In practical terms, one training run with four dual-use categories yields 16 different model configurations (2⁴ = 16). An organization can distribute models with the virology module disabled to clients who do not need that capability, while research institutions with appropriate clearance receive the version with the module enabled — all from the same training run, without restarting.

Limitations and Research Status

It is important to emphasize what GRAM is not yet: Anthropic explicitly states that the method has not been applied to any production Claude model to date and has not been tested at frontier scales (above 5B parameters).

The researchers also note that measuring the concept of “dual-use knowledge” itself is not a trivial problem — the boundary between legitimate and dangerous knowledge is blurry and context-dependent. GRAM offers a technical solution for isolation and deletion, but defining what should be isolated remains a political and epistemic challenge.

Nevertheless, GRAM represents a conceptual shift that could influence in the long term how frontier models are developed and distributed — with built-in knowledge access controls that are more robust than existing post-hoc approaches.