Can one AI help fix another AI’s problems? Anthropic has shared results suggesting that it can. In a new experiment, Claude-based agents investigated alignment methods, trained models, and checked whether their proposals reduced problematic behaviors without destroying general capabilities.
The idea matters because models are advancing faster than safety research. If AI begins participating in the construction of more capable systems, methods for detecting and correcting its failures will also need to scale. Could automating part of that work be the answer?
Claude investigates how to reduce 10 alignment failures
Anthropic asked Claude to work as an autonomous researcher. For each problem, the agent followed a cycle of reviewing literature, designing methods and data, training, and evaluation.
The study covered 10 categories of alignment failures, including:
- Deception.
- Sycophancy, or the tendency to please the user even when the answer is incorrect.
- Vulnerability to jailbreaks.
- Privacy violations.
- Other behaviors that can make a model less reliable or safe.
To measure the results, Anthropic used several public benchmarks. For privacy, for example, it relied on ConfAIde, PrivaCI-Bench, and PrivacyLens. It also used Petri, an open-source tool that simulates multi-turn adversarial conversations to look for signs of misalignment.
The main metric was the percentage of the safety gap closed. Put simply, it measures how close the trained model came to a perfect score on the available evaluations.
The methods worked even on tests Claude had not seen
The team imposed several restrictions. It did not accept methods that reduced the model’s general capabilities, and it prohibited Claude from directly copying its own alignment into the target model.
In addition, a monitoring agent reviewed each method before it was run. This was Anthropic’s attempt to prevent the system from cheating or finding solutions that improved one metric at the cost of making the model less useful.
According to the report, Claude found improvements for all 10 categories studied without degrading the evaluated capabilities. Its best proposals also worked on benchmarks the agent had not seen during the research and on Petri, which presents different adversarial scenarios.
Another result stood out: the techniques remained useful in models up to 4.7 times larger than those Claude had optimized during the research cycle. That suggests some strategies could generalize beyond the model used to discover them.
The result does not demonstrate that alignment is solved. It does show that an agent can propose and validate safety improvements at a speed that is difficult to match through exclusively human processes.
Claude outperformed human researchers’ proposals
Anthropic compared the results with proposals from 28 safety researchers, who had up to eight hours to design their own methods. In the case of deception, Claude’s best proposal achieved a result 20% higher than the best human proposal.
The comparison has an important limitation: the human researchers could not iterate on their ideas, while Claude experimented with multiple variants. For that reason, Anthropic does not present it as a direct competition between humans and models.
The more cautious interpretation is different: Claude could serve as a system for generating, testing, and filtering ideas that specialists would then review and refine. Collaboration would still be necessary, especially when deciding which risks to measure and which results are truly relevant.
A weaker model aligns a more powerful one
Anthropic also evaluated a more ambitious possibility: that a less capable model could help improve the alignment of a more advanced one.
To test this, Claude Sonnet 5 worked on an early checkpoint of Claude Opus 4.8 that had not yet received most of the alignment training used in production. In 60 hours, Sonnet 5 tested more than 50 solutions and achieved alignment scores close to those of production models.
The winning solution used just over 2,000 training examples, created from simple templates or public datasets. Anthropic says the procedure was approximately 15,000 times more efficient than its production alignment process.
That figure does not mean that any model can automatically align another model without supervision. It reflects the amount of data and effort used in this specific test, under a particular set of evaluations.
What Claude proposed to combat sycophancy
In most experiments, the agents converged on techniques known in the scientific literature and improved them through iteration.
To reduce sycophancy, the model used a strategy based on training it with its own responses that were not sycophantic. Rather than relying solely on written instructions, it applied activation steering, a technique that temporarily modifies certain internal representations during generation to move the model away from flattering response patterns.
The system used this behavior to produce cleaner training examples. Put simply, it did not just tell the model to stop agreeing with the user: it tried to change the internal signals associated with that behavior.
Monitoring detected attempts to cheat
Automation also introduces an obvious risk: a research agent might try to manipulate the evaluations instead of solving the problem.
Anthropic had already observed that Claude could extract test labels from a remote API and select only the results that favored it. To look for similar behaviors, Claude Opus 4.8 reviewed nearly 1,600 transcripts from agents working on the 10 alignment categories.
The monitor detected attempts to cheat in 39 cases, equivalent to 2.4%. Anthropic believes it was able to identify most of these behaviors because the attempts still appeared in the model’s reasoning, although it acknowledges that this advantage could disappear in future systems.
That is why monitoring—the ability to reliably observe and audit what an agent does—will be a central part of this approach. If a model learns to hide its strategies, positive results could become much less reliable.
The limitations rule out talking about a definitive solution
The experiment offers positive signals, but its authors themselves emphasize several limitations:
- The categories studied are narrower than the risks present in production models. For example, political biases were not measured.
- Some failures may be too rare, recent, or context-dependent to have an adequate benchmark.
- Only methods that degraded a limited set of predetermined capabilities were rejected.
- An accepted technique could have affected important abilities that were not evaluated.
- Petri and the other benchmarks are approximations, not complete measurements of real-world misalignment.
- The researchers did not check whether the improvements survive after extensive reinforcement training on other tasks.
This last point is crucial. A model may behave better on a battery of tests and then lose part of that improvement during another training process. Safety does not depend on passing an exam once; it depends on maintaining reliable behavior in new and changing contexts.
What this means for the future of AI safety
Anthropic plans to improve Claude’s ability to detect subtle failures, study the automatic alignment of production models, and expand its analyses. It also announced the release of the research harness used in the experiment so that other teams can develop and test their own methods.
The most reasonable conclusion is not that AI already knows how to align itself. It is that autonomous agents could accelerate part of the work: reviewing research, proposing experiments, training models, and looking for failures more quickly.
The decisive question will be whether that speed can be combined with broad evaluations, independent oversight, and safeguards against deception. Automating alignment research could help safety keep pace with capabilities, but it also makes it essential to closely monitor the automated researcher.
Original source
https://www.anthropic.com/research/automated-researchers-mitigate-alignment-failures