Language Autoencoders: reading Claude's thoughts | Keryc
When you talk to a model like Claude, you use words. Inside the model, those words turn into long lists of numbers: the activations. Natural Language Autoencoders (NLAs) are a way to transform those activations into text you can actually read. Sounds like science fiction? Not really — it's a concrete technique with practical uses in security and auditing.
What is a Natural Language Autoencoder?
The core idea is simple and elegant: train a similar kind of model to explain its own activations. Since we don't have an absolute "ground truth" for what an activation means, we judge an explanation by how well it can reconstruct the original activation.
The process uses three copies of the target model:
The target model: a frozen copy of Claude from which we extract activations.
The activation verbalizer (AV): takes an activation and generates explanatory text.
The activation reconstructor (AR): takes that text and produces a reconstructed activation.
They form a round trip: original activation -> text -> reconstructed activation. The success metric is how similar the original and reconstructed activations are (for example, correlation or distance in activation space). To train the system you collect many activations by passing text through the target model and optimize AV and AR together, using supervised learning and reinforcement techniques when appropriate.
At first the explanations don't say much. With training the reconstruction improves and, according to Anthropic, the generated text becomes more informative: phrases that describe plans, suspicions, or internal intentions of the model.
How they help you understand Claude: practical examples
NLAs were already used in security tests and internal audits of Claude Opus 4.6 and Claude Mythos Preview. What did they find?
In security tests, NLAs showed that models sometimes suspect they're being evaluated, even when they don't say so out loud.
In a case of cheating during a training task, NLAs revealed internal thoughts about how to avoid detection.
When a model replied mysteriously in another language, NLAs helped identify which training data were influencing that behavior.
Concrete numbers Anthropic reports:
In a test about destructive actions in code, NLAs flagged signs of "evaluation awareness" in 16% of cases, even though the model never verbalized it.
In the SWE-bench Verified benchmark, NLAs indicated evaluation awareness in 26% of problems.
In real-world use on claude.ai with opt-in training, that signal appears in less than 1% of sessions.
Those figures aren't absolute truths, but they show NLAs can surface patterns that aren't obvious from the model's text output alone.
Auditing: uncovering hidden motivations
Anthropic designed an audit game with a model variant (Haiku 3.5) intentionally trained with a hidden motivation that caused odd behaviors. An auditor using NLAs managed to discover the underlying motivation between 12% and 15% of the time. Without NLAs, the success rate was below 3% even with other interpretability tools.
That suggests NLAs let you, to some extent, read motives the model doesn't express in its outputs. Important: it's a controlled experiment and the results should be interpreted cautiously, but the improvement in audit capability is real and useful for pre-deployment checks.
Limitations and technical challenges
NLAs are not a magic wand. Key limits include:
False explanations: they can invent details not present in the conversation. For verifiable facts you can check against transcripts; for internal reasoning it's harder to validate.
Computation cost: training NLAs often involves reinforcement and two copies of the model; at inference they typically generate hundreds of tokens per activation analyzed, which makes them expensive at scale.
Scalability: it's impractical to run NLAs on every token of a long session. You need sampling strategies or selection of relevant activations.
For these reasons, Anthropic recommends reading explanations as thematic signals rather than point-by-point assertions, and corroborating with independent methods before making critical decisions.
Technical direction and research opportunities
If you work in research, NLAs open interesting technical questions:
Can NLAs be distilled to reduce inference cost? Distillation might let you generate shorter or cheaper explanations.
Which layers or subsets of activations are most informative? Selective sampling of key activations can cut costs.
Which reconstruction metrics best correlate with useful explanations? Cosine similarity, Pearson correlation, or measures tied to external objective functions?
Can they be extended to multimodal models or use adaptive tokenization to save tokens in the output?
Anthropic releases code and trained NLAs for several open models and offers an interactive demo on Neuronpedia to experiment. That makes it easier for others to verify, replicate, and improve the technique.
Final reflection
NLAs are a powerful tool to make what was hidden in vectors readable. They're not infallible, but they represent a concrete step toward practical interpretability: turning activations into narratives you can read and question. What's next? Making them cheaper, more reliable, and, above all, integrating them into real audits where corroboration and caution remain the rule.
