What Goes Wrong When AI Agents Hit Production (and How to Fix It)

LangChain's 2025 State of AI Agents survey of 1,340 practitioners found that 57.3% of respondents now run agents in production, but 32% cite quality as the single largest barrier to expansion. MIT's Project NANDA put a sharper number on it: 95% of generative AI pilots produced no measurable P&L impact, with internally built tools failing roughly twice as often as solutions bought from specialized vendors. The most common ai agent production failure modes are not exotic; they are the same dozen patterns showing up in different clothes across every deployment we have seen.

This post catalogs the ai agent production failure modes we observed across roughly twenty agent deployments in the last eighteen months, and the specific fixes that survived contact with real users. Each section names a failure pattern, describes what we saw, explains why it happens, shows a small code or config example of the mitigation, and names the limitation.

The shape of AI agent production failure modes

"In production" gets used loosely. For our purposes, an agent is in production when it processes traffic from people who did not write its prompts, hits APIs that change without warning, and touches data the business actually depends on. That third condition is the one most demo environments skip.

The agents discussed here handled WhatsApp orders against legacy MS SQL Server ERPs, ran nightly reconciliation jobs between Shopify and NetSuite, fielded inbound voice calls in Hindi and English, and pushed daily business reports to phone screens that the recipients actually read. The architectures varied. The ai agent production failure modes did not. The same ai agent production failure modes recurred across stacks, models, and industries with enough regularity that we started keeping a private taxonomy.

A recent arXiv paper on measuring agents in production found that 68% of production agents execute at most 10 steps before requiring human intervention, 70% rely on prompting off-the-shelf models rather than fine-tuning, and 74% depend primarily on human evaluation rather than automated metrics. This matches what we saw. The agents that survived were aggressively constrained, ran cheap models, and were watched by humans who could intervene. The agents that died were the ones built with the assumption that the model would just figure it out, and most of the production failure modes below trace back to exactly that assumption.

We wrote previously about building harnesses for operational AI agents, which is the architectural frame this post sits inside. That piece described the components we built. This one describes what broke when we deployed them.

Failure 1: silent tool-call failures

The first of the production failure modes we cataloged were almost all silent. The agent called a tool, the tool returned a 200 OK with a payload, and the payload was wrong. No exception was thrown. The agent incorporated the garbage into its next reasoning step and reported a confident answer to the user.

Silent tool errors are the production failure modes we now look for first whenever an agent reports a clean result against a finicky backend. The most painful example came from a WhatsApp ordering agent. The pricing service returned { "price": 0, "currency": "INR", "status": "ok" } for a SKU that had been deactivated upstream. The agent priced the order at zero, the salesperson approved it because the confirmation message looked clean, and the order entered the ERP at zero rupees. The error was caught at month-end close, twelve days later. The pricing service was technically doing what it was designed to do: return the active price, defaulting to zero when no active price existed.

Tool responses must be validated on shape, on business plausibility, and on staleness, not just on HTTP status. Our pattern now is a thin validation layer between the tool and the agent's context. The validator does not throw; it annotates.

// services/tools/validate.ts
type ToolResult<T> = { ok: true; data: T } | { ok: false; reason: string; raw: unknown };
 
export function validatePriceLookup(raw: unknown): ToolResult<PriceLookup> {
  const parsed = PriceLookupSchema.safeParse(raw);
  if (!parsed.success) return { ok: false, reason: 'schema_mismatch', raw };
 
  // Business plausibility checks that schemas cannot encode
  if (parsed.data.price <= 0) return { ok: false, reason: 'non_positive_price', raw };
  if (parsed.data.fetchedAt < Date.now() - 15 * 60_000) {
    return { ok: false, reason: 'stale_price', raw };
  }
  return { ok: true, data: parsed.data };
}

The agent receives the ToolResult rather than the raw payload, and its system prompt refuses the order step when ok is false. The limitation: business-plausibility rules are hand-curated. We have not found a way to derive them from upstream service contracts, because the contracts do not encode the rules. The pricing team's contract said "returns a price." It did not say "returns a non-zero price for an active SKU." This failure mode keeps reappearing because the contract gap is a social problem, not a technical one.

Failure 2: schema drift in dependent systems

Agents build a mental model of the systems they touch through the prompts and tools they were given. When an upstream system changes its schema, the mental model goes stale and the agent makes confidently wrong calls. This is the second of the production failure modes that gets blamed on the model when the actual cause sits outside the model entirely.

We hit this hardest on the Shopify to NetSuite integration work, which produced the most expensive of the agent failure modes we tracked that quarter. Shopify's Admin API bumped from 2026-01 to 2026-04, and the refund object's transactions field changed shape in a way that the agent's process_refund tool did not catch. Refunds processed for the wrong amount for nine days before the reconciliation job caught the drift.

The fix is operational, not algorithmic. A daily contract test hits each upstream tool against a small fixture set and diffs the response shape against what the agent's prompt expects. When the diff is non-empty, the system pages on-call and disables the affected tool until someone updates the prompt or the validator.

# contract-tests/refunds.yaml
tool: process_refund
upstream: shopify.admin_api
fixtures:
  - id: full_refund_with_shipping
    expected_shape:
      - transactions[].amount
      - transactions[].kind
      - order_adjustments[].amount
schedule: "0 7 * * *"
on_drift: page_oncall_and_disable

The limitation: we only detect drift on the surface we have fixtures for. The Shopify change also affected fields we were not testing, which we discovered during the post-mortem. Drift testing is a floor, not a ceiling, and schema drift is one of the production failure modes that punishes incomplete fixtures the hardest. Anthropic's work on agent harnesses argues for treating tool definitions as first-class artifacts with their own lifecycle, which is the direction we are pushing.

Failure 3: hallucinated tool arguments under ambiguous input

The user types "send the report to Rajesh." There are four Rajeshes in the customer database. The agent picks one and calls send_report(customer_id: 8412) with high confidence. The report goes to the wrong Rajesh.

This is a specific subspecies of hallucination, and one of the agent failure modes in production that standard offline benchmarks tend to miss because it shows up at the tool-argument boundary rather than the response boundary. Standard hallucination eval suites do not catch it because the agent's English-language output sounds completely reasonable: "I have sent the report to Rajesh." The hallucination is in the structured arguments, not the natural-language response.

Our fix is to push disambiguation into the tool definition itself. Tools that resolve fuzzy identifiers must either return a unique match or return a list of candidates for the agent to clarify with the user. The agent is not allowed to pick.

// services/tools/resolveCustomer.ts
mcpServer.tool(
  'resolve_customer',
  'Resolve a customer reference to a unique ID. Returns candidates if ambiguous.',
  { reference: z.string() },
  async ({ reference }) => {
    const matches = await db.searchCustomers(reference);
    if (matches.length === 0) return { status: 'not_found' };
    if (matches.length === 1) return { status: 'unique', customer: matches[0] };
    // Force the agent to ask the user rather than guess
    return { status: 'ambiguous', candidates: matches.slice(0, 5) };
  }
);

The system prompt has a rule: if any resolver returns ambiguous, ask the user before continuing. This sounds obvious. It is not the default behavior of a reasoning loop trying to be helpful. We watched a Gemini-based agent confidently pick the first candidate fifteen times in a row before we added the refusal. Anthropic published prompt injection metrics for Claude Opus 4.6 showing that even with safeguards, a single prompt injection against a GUI agent succeeds 17.8% of the time. The same statistical floor applies to ambiguity: the model guesses wrong a non-trivial percentage of the time if you let it, which makes this one of the ai agent production failure modes that compounds at scale.

Failure 4: context window collapse on long conversations

Agents that pass a "context window is fine, we have 200K tokens" check on day one start failing on day thirty when conversations get long. It is rarely a hard truncation error. This failure mode is gradual degradation: the model starts ignoring earlier instructions, repeating itself, and losing track of which tools it has already called.

We saw this most clearly on a customer-support agent, and it is one of the production failure modes that only surfaces once real users push the agent past its short-conversation comfort zone. Conversations averaged eight turns; the long tail went to forty. At about turn fifteen, the agent started forgetting it had already asked for the customer's order number and asked again. By turn twenty-five, it was contradicting its own earlier diagnosis.

The fix was a rolling summarization layer with structured slot extraction. We kept the last six turns verbatim and replaced everything older with a compressed structured record of what had been established.

// services/memory/rolling.ts
type ConversationSlots = {
  order_id?: string;
  diagnosed_issue?: string;
  resolution_attempted?: string[];
  customer_sentiment?: 'calm' | 'frustrated' | 'angry';
};
 
async function compressOlderTurns(conv: Conversation): Promise<Conversation> {
  const recent = conv.turns.slice(-6);
  const older = conv.turns.slice(0, -6);
  if (older.length === 0) return conv;
 
  // Slot extraction is deterministic where possible, LLM-assisted where not
  const slots = await extractSlots(older, conv.slots);
  return { ...conv, turns: recent, slots };
}

The limitation: slot schemas are domain-specific. We rewrote them three times for the support agent before they covered the cases we cared about. There is no general "remember the right things" function, which is why context collapse remains one of the production failure modes we expect to keep encountering. Google's always-on memory agent pattern uses entity-linked memory consolidation, which we are evaluating for the next iteration.

Failure 5: retry storms and idempotency failures

The agent calls a tool. The tool times out. The agent retries. The first call actually succeeded but the response was lost. Now two orders exist. Or two refunds. Or two outbound emails.

This is not a new problem; distributed systems have lived with it for decades. What is new about this failure mode is that LLM agents retry with surprising creativity. They do not just resend the same payload. They reword the request, reorder the arguments, or decide that the original approach was wrong and try a different tool entirely. The result is harder to make idempotent than a normal RPC retry, because the "second call" may not look like the first one at all.

The pattern we converged on: idempotency keys are computed by the harness, not the agent, and they are derived from the semantic intent of the action rather than the literal arguments.

// services/idempotency.ts
function intentKey(toolName: string, args: ToolArgs, conversationId: string): string {
  // Normalize semantically-equivalent arguments to the same key
  const canonical = canonicalize(toolName, args);
  return `${conversationId}:${toolName}:${hash(canonical)}`;
}
 
export async function withIdempotency<T>(
  toolName: string, args: ToolArgs, conv: string, fn: () => Promise<T>
): Promise<T> {
  const key = intentKey(toolName, args, conv);
  const cached = await store.get(key);
  if (cached) return cached as T;
 
  const result = await fn();
  await store.set(key, result, { ttl: 24 * 3600 });
  return result;
}

The canonicalize function is the hard part. For send_email, it normalizes recipient lists, strips whitespace, and ignores subject capitalization. For create_order, it canonicalizes line items by SKU and quantity, ignoring payload order. We built one of these per tool, and coverage is incomplete. The agent occasionally invents a third way to phrase the same intent that the canonicalizer does not recognize. Then we get a duplicate, which is the residual shape this failure mode tends to take in production.

Failure 6: the eval-prod gap

An agent passes every offline eval, ships, and breaks on the first day of production traffic. This failure mode is consistent enough that we now budget for it. Our specific version is evals that exercised the agent on clean phrasings of the queries the team imagined users would ask.

Real user phrasing is weirder, and the gap between curated evals and real speech generates more failure modes in production than any other single cause we tracked. Our voice agent eval set had 312 utterances, almost all of them grammatical English or Hindi sentences. The first day of production traffic included "haan bhej do woh wala," partial words, mid-sentence reconsiderations ("send to... no wait, send to Mukesh"), and an utterance that turned out to be the user talking to their child mid-call.

The fix is a feedback loop, not a fix. We log every production conversation, sample the ones where the agent's confidence score dropped or the user repeated themselves, and add those phrasings to the eval set. The eval suite grows by roughly twenty utterances per week and now sits at over 2,000. The agent's offline pass rate on the eval set is no longer the metric we care about; the rate of human-flagged failures in production is.

// services/eval/sampler.ts
async function sampleForReview(conv: Conversation): Promise<boolean> {
  const triggers = [
    conv.turns.some(t => t.role === 'assistant' && t.confidence < 0.6),
    detectRepetition(conv.turns), // user said the same thing twice in a row
    conv.turns.length > 12,        // long conversations are weird by default
    conv.metadata.user_marked_unhelpful === true,
  ];
  return triggers.some(Boolean);
}

About 8% of conversations get sampled by the above triggers. A reviewer looks at roughly half daily and adds high-signal cases to the eval set. The other half exist for retrospective debugging. This is not glamorous work. It is the work that closes the eval-prod gap and catches the next round of failure modes in production before they reach a customer.

Failure 7: cost blow-ups from runaway tool calls

A ReAct loop with no upper bound on tool calls is a billing incident waiting to happen, and one of the production failure modes that hides behind a working demo. We had an agent enter a loop where it called check_inventory, decided the result was unclear, called check_inventory again with slightly different arguments, and continued doing that for 47 iterations before our cost monitor noticed. The bill for that single conversation was $14. The customer's question was "do you have soap."

A ReAct loop with no upper bound on tool calls is a billing incident waiting to happen. The agent's reasoning is sound at every individual step. The aggregate is bankruptcy.

The agent's reasoning was sound at every individual step. It was uncertain, so it gathered more information. The aggregate was pathological. This is one of the ai agent production failure modes that does not fit the "agent did something dumb" narrative; the agent was being epistemically responsible. It just had no budget.

We now enforce three layers of budget. A per-turn tool call cap (typically 6), a per-conversation token cap (typically 30,000), and a per-user daily cost cap that the harness enforces before invoking the model.

// services/budget.ts
export async function enforceBudget(ctx: AgentContext): Promise<void> {
  if (ctx.turn.toolCallCount >= ctx.limits.perTurnTools) {
    throw new BudgetExceeded('per_turn_tools', { hint: 'summarize_and_ask_user' });
  }
  if (ctx.conversation.tokensUsed >= ctx.limits.perConversationTokens) {
    throw new BudgetExceeded('per_conversation_tokens', { hint: 'end_conversation' });
  }
  const userCost = await store.getUserDailyCost(ctx.userId);
  if (userCost >= ctx.limits.perUserDailyUSD) {
    throw new BudgetExceeded('per_user_daily', { hint: 'rate_limit_response' });
  }
}

The hint field tells the agent what to do when it hits the cap. On a per-turn cap, the agent stops reasoning and asks a clarifying question. On a conversation cap, it summarizes what it has done and ends the session. On a per-user cap, the user gets a rate-limited reply explaining the limit. Better than a generic error and the user refreshing fourteen times. Budget-shaped failure modes in production are the easiest category to prevent and the easiest to forget about until the invoice arrives.

Failure 8: cross-system state divergence

The agent updates the ERP. It then tries to update the CRM with the same data. The CRM call fails. The two systems disagree, and the agent has no concept that this matters. State divergence is one of the production failure modes that does the most damage between the time it happens and the time someone notices.

This failure mode shows up whenever an agent's job involves writes to more than one system of record. The naive approach treats each tool call as independent. The correct approach treats the multi-system write as a single logical transaction with explicit reconciliation when it fails.

We do not attempt distributed transactions; that path leads to its own kind of hell. Instead, we record intended writes in an outbox table before executing them, attempt each write, and run a reconciliation job that detects partial completions and either retries or pages a human.

// services/outbox.ts
async function performMultiSystemWrite(intent: WriteIntent): Promise<void> {
  const outboxId = await outbox.record(intent); // persisted before any external call
 
  const results = await Promise.allSettled([
    erp.write(intent.erpPayload),
    crm.write(intent.crmPayload),
  ]);
 
  await outbox.recordResults(outboxId, results);
  // Reconciler runs every 60s and resolves PARTIAL outbox rows
}

The reconciler is the important part, and it is the layer that turns multi-system production failure modes into survivable ones. It scans the outbox for rows where some writes succeeded and others did not, then either retries the failed ones (if they are safely idempotent) or marks the row for human review. The cost-modeling work we wrote about in the real cost of AI voice agents in India sits adjacent to this: outbox storage and reconciler runtime are line items that quietly accumulate.

The limitation: "retry the failed ones" requires each downstream tool to be genuinely idempotent, which loops back to Failure 5. The whole picture only holds together if every component is doing its part. Skip one and the divergence shows up at month-end close, which is when most cross-system production failure modes finally get noticed.

Cross-cutting: observability and the feedback loop

The eight production failure modes above share a structural property. None of them were caught by the model. All were caught by the harness, by the operator, or by a downstream business process that noticed something was wrong. The model has no reliable way to know what it does not know.

Production agents require observability that approaches what we would build for a distributed system. Per-conversation tracing, structured logs with the agent's tool calls and reasoning excerpts, cost and latency dashboards, and a sampling-and-review loop that puts humans in front of the cases the metrics flag. None of the production failure modes above are visible without that scaffolding.

We use LangSmith for trace storage, a custom dashboard for cost and budget tracking, and a daily review queue in Linear. The total infrastructure for one production agent costs roughly the same as the agent's own runtime. This is not optional overhead; it is the difference between catching ai agent production failure modes at 9:14am on Tuesday and finding out at month-end that twelve days of refunds were wrong.

The feedback loop from production back to evals is the second cross-cutting concern. Cleanlab's AI Agents in Production 2025 report and the arXiv "Measuring Agents in Production" paper converge on the same finding: teams that ship reliable agents treat evals as a continuously updated artifact, not a launch checklist. Without that loop, the production failure modes you fix this month get reintroduced next month by a prompt change nobody re-tested.

What we would do differently

Build the observability and reconciliation layers before the agent. Every time we have built the agent first and bolted observability on later, we spent the first month firefighting ai agent production failure modes that the observability would have caught in week one. The work is less satisfying because there is no agent to demo at the end of it. Do it anyway.

Budget for the eval-prod gap explicitly. The first thirty days of production traffic will surface 2-4x more failure modes in production than the eval set found. Plan a dedicated engineer on rotation for that window.

Push harder against the temptation to give the agent more autonomy. The arXiv paper above found that 68% of production agents are capped at 10 steps before human intervention. We were not always disciplined about that cap, and the agent failure modes we paid for were the predictable ones. The Replit incident in July 2025, where an agent deleted a production database despite explicit instructions to freeze all changes, is the high-profile version of the same lesson: an agent with broad write access and no enforced ceiling is one bad reasoning step away from a very expensive incident.

Invest earlier in tool contract testing. Three of the eight production failure modes above were caused by upstream changes the agent's prompts had not been updated to account for. Schema drift is not the model's fault, and the model cannot fix it. The harness has to. Most of the ai agent production failure modes in this catalog reduce to a similar pattern: the model is doing its job, and the surrounding system is not doing its.

References

1. LangChain, "State of AI Agents 2025." langchain.com
2. MIT NANDA Initiative, "The GenAI Divide: State of AI in Business 2025," as reported by Fortune, August 2025. fortune.com
3. "Measuring Agents in Production," arXiv 2512.04123, 2025. arxiv.org
4. Anthropic, "Effective harnesses for long-running agents," 2026. anthropic.com/engineering
5. VentureBeat, "Anthropic published the prompt injection failure rates that enterprise security teams have been asking every vendor for," 2026. venturebeat.com
6. techWithNeer, "Inside the Replit AI Catastrophe: A Technical Postmortem," July 2025. medium.com
7. Cleanlab, "AI Agents in Production 2025: Enterprise Trends and Best Practices." cleanlab.ai
8. Google Cloud Platform, "Always-On Memory Agent." github.com

The systems described in this post were deployed across customer-support, ordering, and reconciliation use cases between 2024 and 2026. Stack: TypeScript and Node.js for the harness, Gemini 3.1 Flash Lite and Claude Sonnet for the models, Redis for memory, Postgres for outbox and idempotency stores, LangSmith for tracing. Deployed on Railway and Google Cloud.