A few weeks ago I was talking to a team at a large financial company that has spent the last year building AI into one of its core operational workflows.
They had made real progress. They’d broken the process into individual checks, used LLMs to automate parts of the review, and were already seeing work move faster. Now they wanted to take the next step and let the system make some decisions without a person reviewing every case.
That’s where they hit a problem.
Some of their checks were easy to make deterministic. If you need to verify a fact against a database, you can look it up and compare the result. But other checks required reading an unstructured document and deciding whether it satisfied a policy. An LLM could do that well, but not with the consistency they needed to let it make the final decision.
So they were going back through the entire workflow, check by check, asking where they could replace an LLM with deterministic logic and where they actually needed the judgment of an agent.
I’ve had versions of that conversation across compliance, underwriting, due diligence, and research workflows over the past three years.
The prototype is rarely the hard part. Problems appear when teams run the same workflow hundreds of times and need consistent results, traceable decisions, and predictable costs.
By mid-2026, roughly 79% of large enterprises were piloting AI agents. Only 8.6% to 14% had an agent operating at true production scale, handling real task volume with monitoring and incident response behind it.
That range comes from our review of seven major surveys covering more than 125,000 respondents, including research from Deloitte, Gartner, and Sinch. Each survey defines production differently, but the gap between pilots and durable deployments is consistent.
Meanwhile, model performance improved sharply between 2024 and 2026. On WebArena, which tests agents on multi-step web tasks, the top score rose from 14.4% to 71.6%.
You’d expect those improvements to translate into more agents making it into production. They haven’t.
We examined two independent sources of production evidence. Clyro collected 591 documented agent failures, while Presenc AI analyzed public evaluations and deployment instrumentation from more than 60 enterprise agent customers. In Clyro’s dataset, 88% of failures involved infrastructure around the model. Hallucinations and other model-quality failures accounted for around 10%.
That finding matches our experience pretty closely.
Read the full Grep AI whitepaper on Why Agents Fail in Production here.
One way to understand the problem is through some simple math. Say an agent has to complete a 20-step workflow and is 90% accurate at each step. A 90% success rate sounds pretty good in isolation. Across all 20 steps, though, the probability of completing the entire workflow successfully falls to 12.2%.
At 50 steps, the probability falls to 0.5%.
Production workflows also contain dependent errors. If an agent makes a bad tool call at step three, that result enters the context for step four. The agent makes its next decision using bad information, then carries that decision into the following step. It can execute a coherent plan built on a mistake it made ten minutes earlier.
When someone shows me an agent performing an impressive task, I want to know what happens on the hundredth run, then the thousandth. I want to know how it handles an API timeout halfway through a task, contradictory sources, or a context window crowded enough to obscure an instruction from the beginning of the run.
We started Grep AI in compliance, which gave us an unforgiving place to learn how to build agents. If an agent reviews a business for money-laundering risk, “usually right” provides little value. Analysts need to know what it found, where it found it, how it used the information, and what happens when it lacks enough evidence to decide.
Our first products used GPT-4-era models. We abandoned a free-running agentic loop in favor of a pre-orchestrated agent, which we called an agent on rails. We wrote about that architecture in Agents aren’t all you need. The rails improved consistency because we defined the sequence of steps in advance and isolated the model calls.
That architecture also limited what the agent could do. Each workflow required hand-written logic, which restricted us to a small number of compliance use cases.
We added configurable steps so customers could change the order and behavior of a workflow. That made it easier to create new agents while preserving the controls we needed in production.
As models improved, the rails began to constrain the quality of the work. An investigative agent assessing negative news could open the articles it had received, extract evidence, and classify the result. It could not decide to search for another source, follow a connection between two companies, or investigate a person mentioned deep inside an article. Each step remained an independent model call because we had designed the system to contain compounding errors.
That led us to move from agents on rails to an agent in a sandbox, initially built on the Claude Code harness. We described the architecture in Claude in a box. The agent could plan its own work, use tools, write code, and pursue evidence across multiple sources. It opened up deep research, due diligence, legal analysis, and other high-stakes workflows that our previous architecture could not support.
The new agent was far more capable. It was also too expensive to run.
Earlier this year, we reviewed our Anthropic bills and found that we were spending more on tokens than the agent generated in revenue. We had to reduce the cost of each run for the business to work.
That pushed us to build our own harness around a few operating requirements.
First, the harness needed to reuse proven workflows. General-purpose coding agents plan each task from scratch because each input may require a different approach. Enterprise agents often perform the same workflow thousands of times. Once an agent discovers a reliable sequence, it should reuse that work instead of paying to rediscover the process on every run.
The harness also needed to learn operational facts. If an agent discovered that one data provider lacked business-registration coverage in the Bahamas while another provider covered it, future runs should use the right provider from the start. Some steps may no longer require a model once the system has learned a deterministic procedure.
We also added verification during execution. A check at the end of a 20-step run arrives too late if step three contaminated everything that followed. The system needs to inspect intermediate work, catch unsupported claims, and correct errors before later steps inherit them.
Finally, model selection became the most important cost and reliability control. Complex reasoning may require a frontier model. Summarizing a search result, extracting a date, or formatting an output may require a smaller model or a piece of code. The harness can choose between them at each step rather than sending the entire workflow to the most expensive model available.
The new harness is in testing, and we plan to roll it out to our largest enterprise customers. Early tests show cost reductions of 80% to 90%, with accuracy holding steady or improving. We still need more production volume before treating that range as a durable benchmark.
A quick note on Grep: We built Grep to solve these production problems. Grep turns repeatable, high-stakes workflows into agents with verification, citations, audit trails, and cost controls built into each run.
If you have a workflow stuck between prototype and production, you can try Grep or talk to us.
Technical architecture covers only part of the work. Customers ask different questions once an agent moves from helping someone make a decision to making the decision itself.
They want to know how we tested it, why it reached a conclusion, and which sources it used. They ask how it handles conflicting evidence and whether a model update changed its behavior. They want to run the agent against cases where they already know the correct answer.
Customers have pushed us to build those controls.
One customer automating due diligence for an underwriting workflow wanted to verify every material claim the agent made. We exposed citations alongside each claim, showed the underlying tool outputs, added confidence information, and made the execution trace easier to inspect.
Another customer found that asking the agent to do more work improved accuracy and auditability. It also produced reports several times longer than their analysts wanted to read. Report length looked like a product preference until analysts stopped reviewing the work. We had increased accuracy while reducing the practical value of the output.
Production teams have to balance accuracy, consistency, latency, cost, auditability, and the amount of human attention each run requires. More investigation may improve coverage while increasing cost and review time. Tighter controls may improve consistency while preventing the agent from following useful leads.
We expose the agent’s reasoning, tool calls, sources, and generated files so customers can inspect its work. We combine that visibility with versioned agent configurations, in-product evaluations, and cost controls.
Future model improvements will help. They will not repair an unreliable API integration or define the right permissions for a workflow. Teams will still need to detect policy drift, roll back changes, and decide where a person should review the work.
The financial services team I spoke with was asking the right questions. They were deciding where an agent’s judgment added value, where software could produce a certain answer, and where a person still needed to review the result.
Putting agents into production requires teams to define the workflow, constrain access, verify intermediate work, monitor changes, and make the economics hold at real volume. More capable models expand what teams can build. The reliability of the finished system depends on the choices they make around them.
You can read the full Grep research report on Why agents fail.
About the author:
I’m AJ Asver, co-founder and CEO of Grep. I’ve spent the past three years building and deploying AI agents for high-stakes work inside large companies. I write The Operator’s Guide to AI to share what we learn when those systems meet real workflows, customers, costs, and production constraints.