--- title: "Why AI Agents Fail in Production" section: "Insights" canonical_url: "https://leverge.ai/insights/why-ai-agents-fail-in-production" topic: "why AI agents fail in production" published: "2026-07-02" updated: "2026-08-01" publisher: "Ailoitte Technologies Private Limited" --- # Why AI Agents Fail in Production Agent projects rarely fail for exotic reasons. Six failure modes account for almost every rescue engagement we take: no evaluation set, retrieval that was never measured separately, unbounded permissions, unhandled partial failure, unit economics that only worked at pilot volume, and escalation designed for the metric rather than for the human receiving it. Each has a specific, unglamorous preventive practice, and none of them is about prompt quality. ## Key takeaways - Without an evaluation set built from real historical cases, no change to an agent can be shown to be an improvement. - Retrieval quality must be measured independently of answer quality, or you cannot tell which half of the pipeline failed. - Permissions granted during the prototype are almost never narrowed afterwards, which is how a demo becomes a security incident. - Partial failure in a multi-step process has to be designed for, because "it depends" is not an answer when step four of six fails. - Routing every step through a frontier model is fine at pilot volume and indefensible at production volume. - Escalation quality decides whether the team using the agent advocates for it or works around it. Over the last two years we have been brought into a number of agent projects that were already in trouble — built by capable teams, working in demonstrations, failing on real traffic. The failures cluster. Six patterns account for nearly all of them, and none is about the model being insufficiently clever. ## 1. There is no evaluation set This is the root cause behind most of the others. Without a fixed set of cases with known correct outcomes, nothing about the agent is measurable. Prompt changes are unfalsifiable. A model version upgrade is an act of faith. Nobody can answer "how accurate is it" except anecdotally. The consequence is not just that quality is unknown — it is that quality drifts downward. Small changes each look fine on the two examples someone checked, and six months later the system is measurably worse than at launch with no record of which change did it. **What prevents it:** build the set from your real history before writing agent logic. Support transcripts, resolved tickets, completed cases — these are labelled data. Sample to match your real distribution, include the cases a human got wrong, adjudicate the correct outcome with someone who knows the process. It takes about a week and it is the highest return week in the project. ## 2. Retrieval was never measured separately When an agent gives a wrong answer, the retrieval step is at fault far more often than the generation step. The model usually summarised faithfully; it was handed the wrong passages. Teams that measure only end-to-end answer quality cannot see this. They spend weeks on prompts while recall at k sits at 60%, and no amount of prompt work recovers information that was never retrieved. **What prevents it:** score retrieval independently — recall at k, precision at k, grounding rate. Four numbers that make a quality change diagnosable instead of mysterious. ## 3. Permissions were never narrowed During the prototype, the agent gets a broad credential because it is faster. The permissions are not revisited. What ships is an autonomous system with wide write access, no spend ceiling, no approval step on irreversible actions, and no audit trail that would let you reconstruct what it did. This is the failure mode that ends programmes rather than delaying them. One incident is usually enough to get everything paused. **What prevents it:** per-tool credentials at minimum access, reads separated from writes, spend and volume ceilings enforced in code, approval gates on anything irreversible, and an immutable log of every action with the context that produced it. Prompt instructions are the weakest layer of this stack and should never be the only one. ## 4. Partial failure was not designed for A six-step process fails at step four. Two steps have committed, four have not. There is no compensating action, no idempotency key, and no human queue — so the case is silently abandoned in an inconsistent state and discovered during reconciliation weeks later. **What prevents it:** answer the question before launch. What happens when step four fails? Idempotency so retries cannot double-apply. Compensating actions for committed steps. Partial failure as a visible state with an owner, not a gap. ## 5. Unit economics only worked at pilot volume Every step routed through a frontier model, including intent classification, field extraction and output formatting. A hundred cases a day costs nothing. A hundred thousand is indefensible, and by then the architecture assumes one model everywhere. **What prevents it:** a provider abstraction from day one, and per-step benchmarking against the evaluation set. Route each step to the cheapest model that passes for that step. On high-volume workloads this routinely cuts inference spend four to eight times with no measurable quality change. ## 6. Escalation was designed for the metric, not the human The agent gives up and hands over a raw transcript with a note saying it could not help. The human now reads a conversation before starting work, so the automation added latency rather than removing effort. The team stops trusting it and starts checking everything it does, which is worse than having no agent. **What prevents it:** design the handover as a prepared case. The issue in two lines, the retrieved context, what was attempted, what is recommended, and why it stopped. Done properly the human resolves it faster than if they had picked it up cold — and the team becomes an advocate rather than an obstacle. ## The pattern behind the pattern None of these six is a modelling problem. They are all engineering discipline problems, and they are all boring to fix. That is the actual lesson from two years of rescue work. The prototype is the interesting 20% of an agent build. The remaining 80% — retrieval quality, tool contracts, evaluation, permissions, failure handling, cost routing — determines whether the thing is still running in six months. It does not demo well, and it is the whole job. ## Frequently asked questions ### Why did our agent work in testing but fail in production? Almost always because testing used a small set of hand-picked cases that did not reflect the real input distribution. Production traffic has a long tail of exceptions nobody documented, ambiguous inputs, and users asking for things outside the intended scope on day one. The fix is not more testing of the same kind — it is an evaluation set sampled to match your actual distribution, including the awkward cases. ### Is prompt engineering the answer to agent reliability? It is a small part of it. On the builds we have shipped, prompting accounts for roughly a tenth of the engineering effort, behind retrieval quality, tool contracts, evaluation and guardrails. Teams that expect the reverse distribution are the ones who get surprised, because prompt iteration produces small, unstable gains while the structural fixes produce large and durable ones. ### How do we know if our agent is actually degrading? Run the evaluation suite on a schedule and sample live traffic continuously, then alert on score drift and refusal-rate change rather than only on exceptions. The failures that matter most in agent systems return a perfectly well-formed wrong answer, which throws no error and shows up in no dashboard built around errors. --- Source: https://leverge.ai/insights/why-ai-agents-fail-in-production — Leverge