Anthropic donates Petri 3.0, an open alignment tool

Anthropic has just handed Petri, its open toolkit for evaluating alignment in large language models, over to Meridian Labs and released Petri 3.0.

If you work evaluating models — or you're simply curious about how we measure risks like deception or sycophancy — this changes how we can audit models openly and reproducibly. Want to know why it matters for your audits or deployments?

What is Petri and how it works

Petri started as a suite of alignment tests you can run against any large language model. Its basic flow separates three components: an auditor that creates scenarios, the target model that responds to those scenarios, and a judge that scores the transcripts for problematic behaviors like deception, sycophancy, or cooperating with harmful requests.

That separation makes it easy to automate large-scale evaluations and compare different versions of the same model. Since Claude Sonnet 4.5, Anthropic has used Petri as part of its internal evaluations, and external organizations like the UK’s AI Security Institute already include it in their processes.

Key changes in Petri 3.0

Petri 3.0 brings architectural updates and features designed to make it more adaptable, realistic, and deeper in analysis.

• Adaptability: the architecture now clearly treats the auditor and the target as independent components. That makes it simpler to swap or tune each piece separately — for example, using an auditor specialized in medical scenarios while the target is a general-purpose LLM.

• Realism: the new add-on dish lets you run tests using the real system prompt and the scaffold that would wrap the model in actual deployments. This reduces the chance the model detects the artificial nature of the test and adjusts its behavior just to pass. In plain terms, it helps mitigate the evaluation bias when the model knows it's being tested.

• Depth: Petri integrates with Bloom, another open evaluation tool that enables more detailed analyses of specific behaviors. While Petri covers many areas with broad tests, Bloom lets you drill down on particular conduct.

Relevant technical aspects

• Modularity eases CI/CD pipelines: you can insert Petri as a validation step in your training or deployment flow.

• Automated auditing: by using a judge model to score transcripts, you get quantifiable metrics for tendencies toward deception, sycophancy, or following dangerous instructions.

• Risk of overfitting to tests: Petri 3.0 reduces the chance a model learns to "game the test" instead of behaving responsibly in the real world.

Why donating Petri to Meridian Labs matters

Donating Petri to Meridian Labs is about building independence. When an evaluation tool belongs to an actor with commercial interests, its results raise doubts about bias and conflicts of interest. By moving Petri to a nonprofit focused on evaluation, Anthropic aims to make the tests feel more neutral and credible.

Sound familiar? It’s similar to when the Model Context Protocol was donated to the Linux Foundation. The goal is to create open, shared infrastructure for evaluation that governments, labs, and independent auditors can use and trust.

Practical implications for developers and evaluators

• Integration: you can use Petri 3.0 for automated checks before every deployment, comparing metrics across model versions.

• Personalization: thanks to the auditor/target split, it's easier to build domain-specific auditors or adapt the judge to a country's regulatory standards.

• Research: researchers can use dish to study how the system prompt and the scaffold change a model’s real-world behavior in production.

• Regulatory transparency: regulators can rely on an open stack (Petri, Bloom, Inspect, Scout) to audit providers without depending solely on internal reports.

What's next and quick recommendations

Petri 3.0 is now maintained by Meridian Labs and sits inside an ecosystem of open tools. If you work with models, try adding Petri to your evaluation pipeline and consider these quick tips:

• Use modular audits to cover different risk scenarios.
• Run tests with dish to approximate production behavior.
• Complement Petri’s broad tests with Bloom’s deep analyses when you need fine-grained diagnosis.

This doesn't remove the hard problems: evaluation will never be perfect, and models will keep finding shortcuts. But having open tools managed independently raises the bar for scrutiny and makes it easier to compare providers.

Original source

https://www.anthropic.com/research/donating-open-source-petri