An AI agent can complete a task correctly and still be unreliable. Why? It may respond correctly today and fail tomorrow when given exactly the same request, even when it uses zero temperature and a fixed seed.
In critical workflows, such as reconciling a financial transaction or reviewing a contract, that variability is not a statistical detail. It is a production problem. IBM Research proposes measuring it more precisely through new metrics and a system capable of detecting an agent’s unstable decisions.
Average accuracy can hide failures
Most benchmarks report an average figure. For example, a GPT-4.1-based ReAct agent achieved a Mean@5 of 77.4% in AppWorld after running each task five times.
The result seems solid. But there is another question that is closer to the user’s real experience: how many tasks did it get right in all five runs?
The answer was a Pass⁵ of just 53.0%. This means that nearly a quarter of the tasks were solved some of the time, but not consistently.
An agent can be capable and unpredictable at the same time.
The difference between these two metrics is known as the consistency gap. In this case, it was 24.4 percentage points:
- Mean@5: 77.4% average success.
- Pass⁵: 53.0% of tasks passed in all five runs.
- Consistency gap: 24.4 percentage points.
Mean@k, Pass^k, and Pass@k do not mean the same thing
These metrics answer different questions:
- Mean@k: average success rate after running a task
ktimes. - Pass^k: percentage of tasks the agent solves correctly in all
kruns. - Pass@k: percentage of tasks in which at least one of the
kruns succeeds.
Pass@k is useful when you can try several times and verify the result. Pass^k is more demanding: every attempt must work. That is why the expected relationship is Pass^k ≤ Mean@k ≤ Pass@k.
Why an agent can change its decision
Every action an agent takes depends on a probability distribution over the next tokens. If one option has a clearly higher probability than the others, the decision is stable.
But some decisions are much closer. Two tokens may end up nearly tied, and any small variation in the environment can change which one appears first. These variations include parallel processing, floating-point arithmetic on the GPU, or minimal changes in the provider’s infrastructure.
This can happen even when the agent uses temperature=0.0. Zero temperature prevents conventional random sampling, but it does not eliminate differences in the internal probabilities or necessarily resolve close ties.
The problem grows along a trajectory. An agent often makes dozens of decisions: which API to use, which argument to send, whether to search for more information, or whether it makes sense to retry. A small chance of change at each step can eventually produce a completely different trajectory.
That is why increasing the model’s size does not always solve inconsistency. A more powerful model may raise the average success rate, but consistency is a different dimension of capability.
Consistency Analyzer: detecting unstable steps
IBM Research added a new component to ALTK-Evolve, a system that turns an agent’s previous trajectories into reusable guidelines for future runs.
The new tool is called Consistency Analyzer. Its function is to locate the decision points where the agent came close to choosing another action, even when it made the correct decision in the original trajectory.
The process works as follows:
- A recorded agent trajectory is selected.
- Each decision step is reviewed using the context that already existed.
- For each step, several controlled completions are requested—five by default.
- The amount of variation in the responses is measured.
- A score is generated to identify the steps with the greatest risk of changing.
The analysis does not require access to the logits or changes to the model. It also does not need to rerun the entire task or interact again with the tools and environment. It is performed offline with one additional call per decision step.
This matters for real-world systems. In production, repeating an entire task can be expensive, slow, or simply impossible. By contrast, analyzing an already recorded trace offers a more practical way to study reliability.
From detection to reusable guidelines
Once the analyzer finds an unstable decision, ALTK-Evolve turns it into a consistency guideline.
These guidelines are incorporated into the system’s normal storage and retrieval flow. The idea is not to memorize a specific answer, but to capture an error pattern that may appear in other tasks.
In one AppWorld task, for example, the agent had to count activities marked in a SimpleNote note. Based on one trajectory, the system generated recommendations such as:
- When counting checkbox-like markers, use a regular expression anchored to the beginning of each line instead of a simple substring count.
- Verify search results by checking that multiple matches exist and confirming which note is the correct one.
Why does this level of detail matter? Because a note may repeat the checkbox symbol in a legend or title. A superficial count may look correct in one run and fail in the next.
The tool focuses precisely on these points of instability. It does not only study what the agent did wrong, but also what it barely did right and might get wrong later.
The gap is cut nearly in half
The team evaluated the method on 168 tasks from AppWorld test_normal. It used a ReAct agent with GPT-4.1, generated guidelines from a single initial trajectory per task, and then tested each case in five new runs.
The results were as follows:
- Pass⁵: rose from 53.0% to 69.0%.
- Mean@5: rose from 77.4% to 81.0%.
- Consistency gap: fell from 24.4 to 12.0 percentage points.
In other words, nearly one-third of the tasks that were previously inconsistent became correctly solvable in all five runs.
The improvement was stronger at medium and high difficulty levels:
- Medium tasks: 22.9 percentage-point increase in Pass⁵.
- Difficult tasks: 14.3 percentage-point increase.
- Easy tasks: 12.2 percentage-point increase.
In addition, average accuracy did not decline. That was a central requirement of the experiment: improve reliability without shifting failures to other scenarios.
The guidelines also transfer knowledge
The system did not only improve the same tasks from which it obtained the guidelines. When tested on similar tasks within the same scenario, Pass⁵ increased by 13 percentage points.
This suggests that the guidelines were not memorizing a specific trajectory. They were capturing broader patterns, such as the need to validate a search or choose a more precise counting method.
The result was also observed with a weaker model, gpt-oss-120b. On the same tasks, Pass⁵ rose from 10.1% to 16.1%, a 6-point improvement. On similar tasks, the increase reached 8.7 percentage points.
This behavior is relevant for companies that cannot always use the most expensive model. A well-constructed guideline can stabilize behavior without requiring an organization to change its entire architecture or replace the model.
What changes when evaluating AI agents
The most direct recommendation is to report Pass^k alongside Mean@k. An average success rate can make a lucky agent appear as reliable as one that responds correctly again and again.
It is also worth paying attention to difficulty. The gap tends to grow on complex tasks, precisely where an average metric can be most misleading.
Before looking for a larger model, it is worth diagnosing the source of the variability. Perhaps the agent does not need more knowledge, but specific instructions to stabilize decisions that are too close to one another.
The question is no longer just whether the agent can complete a task. It also matters whether you can trust it to complete that task again.
The open-source ALTK-Evolve project includes the Consistency Analyzer and the consistency-guideline generation used in these experiments. For systems that execute actions on data, APIs, or business processes, this approach offers a concrete way to move from measuring average capability to measuring operational reliability.
Original source
https://huggingface.co/blog/ibm-research/altk-evolve-consistency