Anthropic announced a partnership with Accenture to independently evaluate its most advanced artificial intelligence models. The initiative aims to have external specialists observe how these systems are trained, protected, and deployed from within the company.
Independent Evaluators Inside Anthropic
The program will be led by Faculty, Accenture’s artificial intelligence division. Its work will include evaluating models, conducting safety tests known as red-teaming, analyzing their alignment, and checking whether their protective measures work as they should.
What’s the difference compared with a traditional audit? Embedded evaluators will have an access level comparable to that of an employee. This will allow them to follow decisions made during training, speak directly with teams, and observe how models evolve before reaching the public.
From that position, they will be able to identify blind spots, check whether Anthropic is meeting its safety commitments, and report incidents. They could also give the public a clearer view of the benefits and risks of advanced AI.
Independent evaluation does not remove Anthropic’s responsibility for its models. Its goal is to make that responsibility more verifiable.
A One-Billion-Dollar Investment
Anthropic and Accenture expect to invest at least $1 billion each over the next five years to develop capabilities in this area.
The investment comes at a time when frontier models—systems with increasingly broad and advanced capabilities—require controls that go beyond internal testing. It’s not enough to ask whether a model works. It’s also necessary to understand how it could fail and whether its safeguards can withstand unexpected situations.
Accenture will also bring experience observing how companies and governments use AI across very different sectors. This practical knowledge could help evaluate models not only in laboratories, but also in scenarios closer to real-world use.
A System That Is Still Taking Shape
Anthropic acknowledges that embedded evaluation does not yet have common rules. There is still no standard defining what information evaluators should receive, how they should communicate their findings, or who should fund their work over the long term.
The company believes that, over time, resources should come from shared funds or governments. For now, it will directly fund Accenture’s work and is in talks with METR and other nonprofit organizations to test different elements of the model using their own resources.
This raises an important question: can an evaluator remain independent if it receives funding from the same company it is analyzing? Anthropic says it is exploring different mechanisms while an ecosystem with multiple organizations and shared standards takes shape.
A Non-Exclusive Partnership
The agreement is not exclusive. Anthropic says it will work with other evaluators, whose names it expects to announce soon. Accenture, for its part, will also collaborate with other artificial intelligence developers.
The idea is to prevent a single organization from controlling the evaluation of the most powerful models. A system with multiple independent teams could provide more useful comparisons, detect problems that an internal review misses, and increase public trust.
The proposal is still at an early stage, but it marks a significant shift: evaluators would not only review a finished product, but also accompany its creation from within. In an industry moving quickly, that oversight could help detect risks before they reach millions of people.
Anthropic will continue training and launching frontier models while developing this program. The company also promises to share how its approach evolves as it brings in new evaluators and the industry establishes better practices.
Original Source
https://www.anthropic.com/news/accenture-embedded-evaluation