62% of SMBs piloted an AI agent in 2025. Only 19% made it to production. Here are the seven failure modes we see over and over — and the playbook that gets a pilot across the finish line.
Pricing note, 5 September 2026. Our published prices changed on this date: Operators moved from $4,950 build / $1,997 per month to $7,500 / $2,950, and Agent in a Day moved from $497 one-time to $2,500 build / $497 per month. The arithmetic below was run against the prices in force when it was written and has been left as it was rather than quietly restated. Current prices are on the pricing page.
The McKinsey 2025 mid-year report tracked something brutal: 62% of SMBs piloted an AI agent in 2024-25. Only 19% made it to production. The other 43% spent budget, lost time, and walked away convinced "AI doesn't work for our business."
It works. The pilot didn't.
Below are the seven failure modes I see in nearly every dead pilot, in rough order of frequency, plus the playbook for avoiding each one.
Symptom: the agent goes live, nobody checks the call recordings, nobody updates the prompt when something breaks, and within 30-60 days the pilot is "broken" and gets shut off.
Frequency: showed up in 38 out of 50 dead pilots I've reviewed.
Why it happens: AI pilots are usually championed by a founder or department head who doesn't have time to babysit the pilot daily. The vendor assumes the customer is monitoring. The customer assumes the vendor is monitoring. Nobody is.
The fix: before kickoff, the customer names a single person — by name, not by role — who owns the metric for 90 days. They don't have to do the engineering. They have to read the dashboard every Tuesday and Friday and decide whether the pilot is on track. If nobody on the customer side can commit 30 minutes twice a week, the pilot will fail. Cancel before you start.
Symptom: 60 days in, the founder asks "is this working?" and gets a vague answer like "people seem to like it." No upgrade decision can be made because there's nothing to upgrade against.
Frequency: 31 of 50.
Why it happens: AI pilots get sold as "improve customer support" or "save time on calls" — squishy outcomes that nobody knows how to measure. By the time someone thinks to define the metric, the comparison baseline data is gone.
The fix: before the pilot starts, define ONE primary metric and the exact way it'll be measured. Examples that work:
Then collect baseline data for two weeks BEFORE going live. Without baseline data, the pilot can't be evaluated even if it works.
Symptom: the pilot scope includes the gnarliest 5% of tickets — the emotional escalations, the multi-system disputes, the edge cases. The agent struggles. Confidence collapses. The whole pilot is judged on the 5%, not the 95% that worked fine.
Frequency: 27 of 50.
Why it happens: the customer's internal advocate has been told "AI can handle everything" by some vendor demo. So they want to prove it on the hardest cases. They lose the political fight when those hard cases get botched.
The fix: scope the pilot to the easy 70%. The categories where the agent will obviously crush it. Stack the deck. Win those cases. Then in phase two, expand into the harder categories with the political capital you earned in phase one.
Symptom: the agent hits a question it can't answer, says something awkward like "I don't know" without offering an escalation, the customer hangs up, leaves a 1-star review.
Frequency: 24 of 50.
Why it happens: the build team assumed the agent would handle 100% of cases. It won't. Ever.
The fix: every agent ships with an explicit "I'm going to grab a human for you" branch. The handoff includes the full conversation transcript so the human doesn't have to ask the customer to repeat themselves. Test the escalation path on day 1 — most installs never do, and discover at 2am two weeks in that the escalation phone number was wrong.
Symptom: the agent answers a question with last month's pricing. Or quotes a discontinued product. Or directs the customer to a phone number that was changed in February.
Frequency: 22 of 50.
Why it happens: nobody owns the KB refresh process. The KB was scraped on day 1 from the website + a Google Drive of policy docs. Then the website got updated, the policy docs got moved, and the agent's RAG index never caught up.
The fix: KB refresh must be on a cron — weekly at minimum, daily for fast-moving content. AND a human spot-checks the agent's answers weekly against the latest source-of-truth docs. Better yet: connect the KB to a structured source (Notion, a CMS, a database) where changes flow through automatically rather than living in a fragile collection of PDFs.
Symptom: the agent gets built once, deployed, evaluated 60 days later. Nobody touched the prompt or the routing in between. The verdict is rendered on the day-1 version.
Frequency: 20 of 50.
Why it happens: most vendors price the pilot as a fixed scope and don't include iteration. The customer thinks "the build is done." It isn't.
The fix: the pilot retainer must explicitly include 4-8 hours/month of prompt iteration, eval re-run, and KB refresh. The agent at month 2 should not be the agent at week 1 — and if it is, you're underinvesting in maintenance. Bake the iteration cadence into the contract.
Symptom: the customer-side team that the agent partially replaces (or augments) is hostile from day 1. They flag every error. They tell customers "the AI made a mistake, here let me fix it." Within 60 days the political pressure builds enough that the executive sponsor backs down.
Frequency: 14 of 50 — and this one is the hardest to fix.
Why it happens: the pilot was positioned as "we're going to replace your job with AI." Of course the team is hostile. Or — equally common — the pilot was positioned as "this is going to augment you" but in practice nobody on the affected team was consulted, trained, or given a role in the rollout. They feel run over.
The fix: the affected team has to be in the room from week 1. Not as observers — as co-owners. The right framing: "the agent handles tier-1, you handle tier-2 and become a higher-paid specialist in 6 months." When the team sees themselves moving UP the value ladder, not getting cut, the political problem disappears. Concretely: include 1-2 customer-side team members on the eval-grading panel from day 1.
Stacking the seven fixes into a single playbook:
Week 0 (before kickoff)
Weeks 1-2 (build + soft launch)
Weeks 3-8 (scale + iterate)
Week 9 (the decision)
This playbook is boring on purpose. Boring is what gets pilots into production. The exciting pilots — the ones with the unrealistic scope and the heroic single owner — those are the ones in the 81% that don't make it.
Every pilot we ship gets the playbook above baked into the engagement. We name the metric owner together on the kickoff call. We collect baseline. We scope the easy 70%. We meet twice a week. The cost of doing all this is exactly why the Operators retainer is $1,997/mo rather than a few hundred — but the cost of NOT doing it is a five-figure build sitting unused in 90 days.
The 30-page State of AI Operations 2026 report includes the failure-mode taxonomy above plus the 15-point readiness scorecard we use on every kickoff. Download it here.
Or if you're staring at a pilot decision right now: book a call and I'll tell you whether to ship it or skip it, no pitch required.