The AI Agent Reliability Paradox
By Vikas Malpani. I run two companies where AI already does most of the production work. I have shipped agents that dazzled in a demo and then quietly wrecked a workflow in production, and I have spent more nights than I want to admit turning those same agents back into boring, predictable software. This is what I learned about why that works.
Every enterprise is running the same experiment right now, and getting the same result. Cisco put a number on it: 85 percent of enterprises are piloting AI agents, and only about 5 percent have shipped one into production. That is not a capability gap. The models are strong enough. It is a reliability gap, and reliability is a design decision, not a model release.
The industry keeps waiting for the next model to close it. It will not. The fix is architectural, and it runs against the instinct that made agents exciting in the first place.
Table of Contents
- The 5 percent wall nobody wants to name
- The framework: the autonomy-reliability tradeoff
- The constraint dividend
- The scaffold spectrum: five rungs of decision latitude
- Deterministic scaffolding: the reliable parts are the parts you wrote
- The reliability budget: how to size autonomy with math
- The agent decision map: when an agent is actually justified
- The contrarian take: the rebuild era is not about smarter models
- What to do Monday morning
- FAQ
The 5 percent wall nobody wants to name
The agentic debate is over. The whole market agreed that agents are the future, that they will do real work, that autonomy is the point. And then almost everyone hit the same wall.
Cisco’s number is the cleanest version of it. 85 percent piloting, roughly 5 percent in production. Amazon’s director of AGI autonomy put the diagnosis on stage in front of a room full of builders: the blocker is reliability, not capability. Agents ace the internal evaluation and then collapse in the wild. He broke reliability into four parts that are worth memorizing, because a demo can fake all four and production tests every one of them: consistency, resilience, predictability, and safety.
Here is the part that stings. The reason your agent works in the demo and dies in production is usually not that it got dumber. It is that you gave it too much freedom, not too little intelligence.
A demo is a single happy path. You picked the input. You knew the answer. You ran it until it looked good. Production is ten thousand inputs you did not pick, half of them malformed, some of them adversarial, all of them arriving while the agent is three decisions deep into a plan you never saw it make. Every one of those decisions is a place the model can be wrong, and wrong decisions do not add. They multiply.
I learned this the expensive way. One of my early agents was a “smart” reconciliation loop. In the demo it read a document, decided what to check, checked it, decided what to fix, and fixed it. Beautiful. In production it made a plausible wrong decision at step two of a seven step chain, and because every later step trusted the earlier ones, it confidently produced a clean looking result that was completely wrong. Nobody caught it for a week. The demo had one decision point I was watching. Production had seven I was not.
The builders who are quietly getting agents into production are not using better models than you. They are letting their agents make fewer decisions. That is the whole trick, and it has a shape.
The framework: the autonomy-reliability tradeoff
Autonomy and reliability trade off against each other. Not sometimes. Structurally. The more decisions you hand to the model, the less reliable the system gets, and the relationship is not linear, it is compounding. This is the single most important curve in agent design, and almost nobody draws it.
Read the curve left to right. On the far left, the model makes almost no decisions. You wrote the steps, the model just fills in the blanks at each one. Reliability is near the top. On the far right, the model owns the whole loop. It decides what to do, in what order, when it is done, and how to recover from its own mistakes. Reliability falls off a cliff.
Most teams start a project on the right side of this curve, because that is where the excitement is. “Give the model a goal and let it figure out the rest” is a thrilling demo. Then they spend six months walking leftward, adding constraint after constraint, until the thing is reliable enough to ship. By the time it works, it is barely recognizable as the autonomous agent they pitched. That leftward walk is not failure. It is the actual work. The mistake is starting on the right and treating every constraint you add as a retreat instead of the point.
The law under the curve is simple. Reliability is bought with autonomy. You do not get both. Every decision you delegate to the model is a coin you spend against your reliability budget, and every decision you take back is reliability you buy. The question in agent design is never “how autonomous can I make this.” It is “how little autonomy can I get away with and still do the job.”
This is a different axis from the one I wrote about in deploying AI agents without losing control. That piece is about authority, how much an agent is allowed to do, the blast radius of its actions, what it can touch and undo. This piece is about latitude, how many decisions it is allowed to make. You can have an agent with tiny authority that still makes far too many decisions, and it will be unreliable even though it can barely break anything. Authority governs how bad the failure is. Latitude governs how often it happens. You have to tune both, and they are not the same dial.
The constraint dividend
Here is the counterintuitive payoff that makes the whole approach work. When you take a decision away from the model and encode it in your own logic, you do not just remove one failure point. You compound reliability back across the entire chain.
Say your workflow has seven steps and each one the model handles independently is 90 percent reliable. Chained end to end, that is 0.9 to the seventh power, about 48 percent. A coin flip with worse odds. Now suppose four of those steps are actually decidable in advance. The document type determines which checks run. You do not need a model to decide that, you need an if statement. Encode those four as deterministic logic that is effectively 100 percent reliable, and you are left with three genuine model decisions at 90 percent each. That is 0.9 cubed, about 73 percent. You did not improve the model at all. You removed four coin flips, and your end-to-end reliability jumped 25 points.
I call this the constraint dividend. Every decision you convert from “the model figures it out” to “the code already knows” pays you back more than its own weight, because reliability multiplies. Removing one weak link from a chain does not subtract a little, it lifts the whole product.
This is why “just use a smarter model” is such a weak lever by comparison. Pushing per-step reliability from 90 to 95 percent on all seven steps gets you from 48 to 70 percent. Removing four unnecessary decisions gets you from 48 to 73 percent, and you can do it this afternoon without waiting for anyone’s next release. The constraint dividend is available to you today. The model upgrade is a hope.
The reflex most builders have is exactly backwards. When an agent is flaky, they reach for more intelligence: a bigger model, a longer prompt, a reasoning mode, another tool. Sometimes that helps at the margin. But the highest-return move is almost always subtraction. Find the decisions the model is making that it did not need to make, and take them back. A flaky agent is usually not too dumb. It is too free.
The scaffold spectrum: five rungs of decision latitude
Autonomy is not a switch, it is a dial, and it helps to name the notches. I think about agent architecture as a spectrum of decision latitude, from a fully scripted pipeline where the model decides nothing about control flow, to open-ended autonomy where the model decides everything. Five rungs cover almost every real system.
The full detail matters, so here is the same spectrum with the tradeoff spelled out for each rung.
| Rung | Who decides the control flow | Failure exposure | Use it when |
|---|---|---|---|
| 1. Pipeline | You do. The model only classifies, extracts, or generates at fixed points. | Lowest. Failures are local and visible. | The path is knowable in advance. Most business workflows. |
| 2. Router | The model chooses among branches you defined and wrote. | Low. One decision, bounded set of outcomes. | Inputs vary but the responses are a known, finite menu. |
| 3. Bounded loop | The model iterates, capped by a hard step limit and a check between steps. | Medium. Contained by the cap and the checks. | The number of steps is unknown but progress is verifiable each pass. |
| 4. Planning agent | The model writes its own plan, then executes and revises it. | High. The plan itself can be wrong, silently. | The path cannot be hardcoded but each result is checkable, like a coding agent with tests. |
| 5. Open autonomy | The model owns goal, plan, execution, stopping, and recovery. | Highest. Every property compounds against you. | Rarely in production. Research, exploration, low-stakes and reversible only. |
The design move is to pick the lowest rung that can actually do the job, and to justify every climb. Most teams do the opposite. They reach for rung four or five because it is the most impressive, then try to bolt reliability on afterward. You cannot bolt reliability onto open autonomy. Reliability is what you have left after you have removed the decisions the model did not need to make.
Notice that a lot of what gets sold as an “agent” is really a rung one or rung two system wearing a costume. That is not a criticism. It is the goal. The systems doing real work in production are mostly pipelines and routers with a model at the decision points, not autonomous loops. When I wrote about the unit of work in spec-driven development, this was the quiet lesson underneath it: the spec is you taking decisions back from the model before it ever runs.
Deterministic scaffolding: the reliable parts are the parts you wrote
There is a rule I now apply to every agent I build. The reliable parts of your system are the parts you wrote. The model is the least reliable component in the whole thing, by a wide margin, and you should treat it that way.
Anthropic makes this point in their guide to building effective agents, and it is the most useful piece of engineering advice in the whole agent space. They draw a line between workflows, where the model and tools are orchestrated through code paths you defined, and agents, where the model directs its own process. Their guidance is blunt. Find the simplest solution possible, and only add complexity when it demonstrably improves outcomes. Sometimes the right answer is not to build an agentic system at all. Reserve real autonomy for the cases where you cannot hardcode the path but you can still verify progress.
Deterministic scaffolding is what I call the code you write to hold the model in place. Input validation before the model ever sees the data. A fixed sequence of steps instead of a plan the model invents. Typed outputs that fail loudly when the model returns something malformed, instead of a free text blob you parse and pray. A check between every step that confirms the world is in the state the next step expects. Retries with a hard cap so a confused model cannot spin forever. The model provides judgment at specific, guarded points. Everything around it is boring, testable software.
This is the opposite of the instinct that a more capable model means you can remove the scaffolding. The scaffolding is not training wheels you take off once the model is good enough. It is the load-bearing structure. A stronger model lets you widen what happens at each guarded point, it does not let you remove the guards. I have watched teams “upgrade” a working pipeline into a flaky planning agent because the new model was smart enough to plan, and every time, they traded a reliable system for an impressive one.
The tell that you are relying on the model instead of your scaffolding is that you cannot answer a simple question: when this fails, where does it fail, and how will I know? In a pipeline you can point at the step. In an open loop you cannot, because the failure is a decision the model made three turns ago that looked fine at the time. This is the same silent-failure problem I keep circling in why AI agents fail in production. That piece catalogs the ways agents break. This one is about the design choice upstream that decides how many of those ways are even open to you.
The reliability budget: how to size autonomy with math
You can make the autonomy decision with arithmetic instead of vibes, and you should. The math of chained reliability is unforgiving, and once you internalize it, over-autonomy stops looking clever.
The rule is that reliabilities multiply. If a workflow has n steps and each succeeds with probability p, the whole thing succeeds with probability p to the n. Not the average of the steps, the product. This is where the famous numbers come from. At 95 percent per step, ten steps land at 59 percent. At 90 percent, ten steps is 35 percent. At 85 percent, which is a genuinely strong score on a hard reasoning task, ten steps collapses to 20 percent. Run twenty steps at 95 percent each and you succeed about 36 percent of the time. The 2026 literature is full of this now, and one paper even maps runaway agent retry loops onto integral windup, the classic control-systems failure where a system chasing its own error spirals instead of settling.
Turn the math around and it becomes a budget. Decide the end-to-end reliability you need. If you need 95 percent success across a 10 step workflow, each step has to hit 99.5 percent. No current model does that reliably on open-ended reasoning. So you have three levers, and only three. Raise per-step reliability, which mostly means waiting for a better model. Reduce the number of model decisions, which is the constraint dividend. Or add verification that catches and repairs errors between steps so failures do not propagate. The second and third are yours to pull today. The first is not.
This is also why long-horizon tasks are so much harder than they look in a demo. Benchmarks in 2026 show leading models scoring 80 to 90 percent on single-turn tasks and then dropping to roughly 18 to 24 percent on sustained multi-step workflows that cross applications. Researchers found agent success starts declining after about 35 minutes of human-time-equivalent work, and that doubling a task’s duration roughly quadruples its failure rate rather than doubling it. Duration is autonomy in disguise. A longer task is more decisions, and more decisions is more product terms below one.
I treat the reliability budget as a hard gate before I add any autonomy. Count the decisions the model will make in the worst-case path. Multiply your honest per-decision reliability that many times. If the result is below what the job needs, you do not get to ship it as-is, and no prompt tweak changes the arithmetic. You either take decisions back or you add verification. This is the same discipline I argued for in the AI eval budget, where reliability is a line item you pay for on purpose. The budget here is the input to that one. Evals tell you your real per-step p. This math tells you how many steps you can afford to spend it on.
Duration is autonomy in disguise
Here is a trap that catches careful teams. You count the decisions in your agent, the number looks reasonable, and you ship. Then it falls apart on the long inputs, and you cannot see why, because the decision count did not change. What changed is duration, and duration is autonomy wearing a disguise.
A task that takes the model longer is a task with more decisions inside it, even when the top-level plan looks the same. A document that is three pages instead of one is more places to extract wrong. A conversation that runs forty turns instead of five is thirty-five more forks. When researchers found agent success rates dropping sharply after roughly 35 minutes of human-equivalent work, and doubling a task’s duration roughly quadrupling its failure rate, that quadratic shape is the compounding math showing up as a clock. Time is not neutral. Every extra minute is extra latitude you handed the model without writing it down.
I got burned by this on a summarization agent that was flawless on the test set and erratic in production. The test documents were short. Real ones were not. Same code, same prompt, same decision count on paper, and yet the long documents pushed the model through far more implicit judgment calls, and the errors stacked. The fix was not a smarter model. It was chunking the long inputs into bounded pieces, verifying each one, and refusing to let the agent hold a hundred pages of context and decide, in one undifferentiated pass, what mattered.
The practical rule is to treat duration as a first-class part of your autonomy count. When you estimate the decisions in a task, scale by how long the worst-case input runs, not the demo input. If a task can balloon from thirty seconds to thirty minutes, its autonomy can balloon with it, and your reliability budget has to survive the long version, not the pretty one. The way you cap duration is the same way you cap everything else on this curve. You break the long autonomous stretch into bounded steps you can check, and you take the open-ended judgment back into code wherever the length was doing the deciding for you.
The agent decision map: when an agent is actually justified
None of this means autonomy is bad. It means autonomy is expensive, and you should spend it only where it buys something a pipeline cannot. There are real tasks where you genuinely cannot hardcode the path, and for those, a constrained agent is the right tool. The trick is telling them apart before you build, not after you have burned a quarter.
Two questions settle it. Can you hardcode the path in advance? And can you verify progress at each step? Those two axes give you a map.
Top left, you can script the path and check the output. That is a pipeline, and it is where most business automation actually belongs. Bottom left, you can script the path but cannot fully verify the result, so you keep a human on the sign-off. Top right is the one place a real agent earns its cost: you cannot know the path in advance, but you can verify progress at each step, so a bounded loop can safely explore. A coding agent with a test suite is the classic example, because the tests are ground truth the model cannot argue with. Bottom right is the danger zone, no fixed path and no way to verify, and deploying an autonomous agent there is how you end up with a confident, unaccountable mess.
The workflow-versus-agent distinction underneath the map is worth making explicit, because the two have different failure modes and you choose them on purpose.
| Property | Workflow (scripted path) | Agent (model-directed path) |
|---|---|---|
| Who owns control flow | You, in code you can read and test. | The model, decided at run time. |
| Predictability | High. Same input, same route. | Low. Same input can take a new route. |
| Cost and latency | Bounded and knowable. | Variable, and it can balloon. |
| Failure mode | Local, at a named step. | Diffuse, a decision made turns ago. |
| Best for | Knowable paths, which is most work. | Unknowable paths you can verify. |
The map is not telling you agents are wrong. It is telling you that “should this be an agent” is a question with a real answer, and the answer is usually no. When the answer is yes, it is a bounded agent in the top right quadrant, not an open-ended one, and it lives there because verification, not intelligence, is what makes it safe.
The contrarian take: the rebuild era is not about smarter models
The common story about 2026 is that agents did not work yet because the models were not ready, and once the next generation lands, autonomy will finally deliver. Enterprises are quietly rebuilding their agent stacks, and the assumption is they are waiting for capability to catch up.
They are not. The teams crossing the production line are not the ones with the best models. They are the ones who stopped asking their models to make so many decisions. The rebuild era is not a capability story, it is a subtraction story. It is founders taking latitude back from the model, one decision at a time, and calling the result an agent because that word raised the round.
This flips the usual advice. The industry tells you to make agents smarter. The move that actually ships is to make them dumber and more constrained, and to spend your intelligence budget on a few well-guarded decisions instead of spraying it across a loop. The most reliable agent in your product is the one that is barely an agent at all, a pipeline with a model bolted in at the two places judgment is genuinely required.
Now the honest counterweight, because the argument fails if I pretend constraint is free. Over-constraining is a real failure mode with a real cost. If you turn every task into a rigid pipeline, you lose exactly the tasks where open-ended reasoning is the whole value, the ones where you cannot hardcode the path and a bounded agent would have found a route you never anticipated. Anthropic’s rule cuts both ways. Do not build an agent when a workflow will do, and do not cripple a task that genuinely needs to explore. Some of the highest-value work sits in that top right quadrant, and a founder too scared of autonomy will cede it to someone braver. The skill is not minimizing autonomy. It is spending it precisely, buying reliability everywhere it is cheap so you can afford autonomy in the one place it pays.
There is a trust dimension here too, and it is why this matters more in 2026 than it did a year ago. Cisco’s framing is that the barrier is a shift from information risk to action risk. A wrong answer you read is annoying. A wrong action an agent takes is a liability. I unpacked that in the AI trust gap. The reason the trust gap is so hard to close is that trust is downstream of predictability, and predictability is exactly what you spend when you add autonomy. You cannot ask a customer to trust a system whose behavior you cannot predict, and you made it unpredictable on purpose the moment you let the model own the loop.
What to do Monday morning
Concrete steps you can run this week on any agent you are building or already shipped.
1. Count the decisions. Take your agent’s worst-case path and write down every point where the model, not your code, decides what happens next. Not the model calls, the decisions. Choosing a branch, deciding it is done, deciding what to fix. That count is your autonomy number, and most people are shocked how high it is.
2. Run the reliability budget. Put an honest per-decision reliability on those steps, something like 0.9 for a real reasoning decision, and multiply it that many times. If the product is below what the job needs, you have a math problem, not a prompt problem. Write the number down where the team can see it.
3. Find the fake decisions. Go through the decision list and mark every one that is actually decidable in advance. Document type, routing, whether a field is present, which tool applies. These are if statements the model has been doing for you at a 10 percent error rate. Convert them to code. Bank the constraint dividend.
4. Drop a rung. Look at where your system sits on the scaffold spectrum and ask what breaks if you move it one rung toward pipeline. Turn the planning agent into a bounded loop with a step cap. Turn the bounded loop into a router over branches you write. Ship the lower rung if it still does the job. It almost always does.
5. Add a check between steps. For every model decision you keep, add a cheap verification before the next step trusts it. A type check, a sanity range, a second look, a test. Verification is how you stop one wrong decision from poisoning the whole chain, and it is usually cheaper than the smarter model you were about to reach for.
6. Answer the failure question. For your agent, write one sentence: when this fails, it fails at ____ and I will know because ____. If you cannot fill both blanks, you are relying on the model instead of your scaffolding, and you have found your next rung to drop.
Do these six and you will not have a more impressive agent. You will have one that survives Monday’s inputs, and Tuesday’s, and the ones nobody thought to demo. That is the trade. If you want the wider context on where agents fit in a solo or small team’s stack, I laid it out in the AI-native founder playbook and mapped the opportunities in the AI opportunity map.
FAQ
Why do AI agents work in a demo but fail in production?
A demo is a single happy path you chose, with an input you picked and an answer you already knew. Production is thousands of inputs you did not pick, arriving while the agent is several self-made decisions deep. Each decision is a chance to be wrong, and chained decisions multiply their error rates rather than averaging them. The gap is not that the model got dumber. It is that production exercises every decision you handed the model, and the demo only exercised one.
What is the autonomy-reliability tradeoff?
It is the structural rule that as you give an AI agent more decisions to make, its end-to-end reliability falls, and it falls in a compounding way rather than a linear one. Reliability is bought with autonomy. Every decision you delegate to the model multiplies the failure surface, and every decision you encode in your own logic multiplies reliability back. You cannot maximize both, so agent design is really the practice of spending autonomy only where it buys something a scripted pipeline cannot.
How do I make an AI agent more reliable?
Take decisions away from the model. Convert every step that is decidable in advance into deterministic code, cap any loops with a hard step limit, add a verification check between steps so one error cannot propagate, and use typed outputs that fail loudly. Reaching for a bigger model is usually the weakest lever, because removing unnecessary decisions raises reliability more than a few points of per-step accuracy would, and you can do it today.
What is the difference between a workflow and an agent?
In a workflow, you own the control flow in code you can read and test, and the model only does specific tasks at fixed points. In an agent, the model owns the control flow and decides its own path at run time. Workflows are predictable, bounded in cost, and fail at named steps. Agents are flexible but unpredictable, variable in cost, and fail in diffuse ways. Most production work should be a workflow. Reserve agents for paths you cannot hardcode but can still verify.
When should I actually use an autonomous agent?
When two things are true: you cannot hardcode the path in advance, and you can verify progress at each step. That combination, the top right of the agent decision map, is where a bounded agent earns its cost, because verification keeps the exploration honest. A coding agent with a test suite is the model example. If you cannot verify progress, an autonomous agent is a liability regardless of how capable the model is.
Does the compounding error math mean long tasks are impossible for agents?
Not impossible, but far harder than a demo suggests, and you have to design for it. If each step is 95 percent reliable, a ten step chain is about 59 percent and a twenty step chain about 36 percent. Long tasks are just more decisions, which is why research finds agent success dropping sharply past roughly 35 minutes of equivalent work. You beat it by shortening the chain, adding verification between steps, and refusing to let duration hide how much autonomy you granted.
Is constraining an agent just giving up on AI’s potential?
No. Constraint is how you make the potential shippable. The point is not to minimize autonomy everywhere, it is to spend it precisely. You buy reliability cheaply on the decidable parts so you can afford real autonomy in the one place a task genuinely needs to explore. Over-constraining is a real mistake that costs you the tasks where open-ended reasoning is the whole value. The skill is knowing which is which, not being reflexively afraid of autonomy.
Will better models make the reliability problem go away?
Only partly, and not the way people hope. Better models raise per-step reliability, which helps, but the compounding math still bites and long-horizon tasks still degrade. Enterprises rebuilding their agent stacks in 2026 are not mainly waiting for capability, they are removing decisions and adding scaffolding. Treat model improvements as a way to widen what you attempt at each guarded point, not as permission to remove the guards. Reliability stays a design choice no matter how good the model gets. For the related question of building on models that keep changing, see how a business survives model churn.
If your agents keep dying between the demo and the ship date, the failure taxonomy in why AI agents fail in production, the readiness view in the production gap, and the skill question in what to learn in the AI era are the natural next reads.