In research with AE Studio, Anthropic introduces GRAM, an experimental technique to give language models removable compartments of dual-use knowledge. Can you imagine being able to remove just the part that knows how to exploit a vulnerability without destroying everything else that makes the model useful? That's what they're proposing here, with important caveats: it's early, technical, and still far from production.
What GRAM is and why it matters
GRAM stands for Gradient-Routed Auxiliary Modules. The central idea is simple in intuition: instead of letting all knowledge be learned and diffuse across the model's weights, you create dedicated modules for categories of dual-use knowledge (for example virology, cybersecurity). Each module is a set of neurons added to every layer of the Transformer.
During training, the model learns from general text as usual. But when an example belongs to a dual-use category, only the corresponding module can be updated; the general parts of the model are temporarily frozen. This way, the specific knowledge accumulates in that module and doesn't spread through the whole network. Practical result? The module can be deleted at deployment and the dangerous capability goes away with it, or it can be left active for trusted users.
