The uncomfortable pattern in enterprise AI is that pilots succeed and deployments do not. The demo impresses, the steering committee approves, and then the thing quietly never ships. Having been brought in to rescue a number of these, the causes are consistent enough to list.
The pilot was designed to impress, not to de-risk
A pilot should attack the riskiest assumption in the project. Instead, most pilots are built to look good in a steering-committee slide, which means they are run on the cleanest available data, on the happy path, with the hard cases quietly excluded.
That produces a demo that proves nothing. The question a pilot must answer is not "can this work" — you already know it can. It is "what will stop this from working here."
Design the pilot around your worst data, not your best. If it holds up on the messy quarter, the clean quarters take care of themselves.
Nobody wrote down what "working" means
"Improve ticket handling" cannot be passed or failed, so the project never concludes — it just loses momentum.
A specification looks like: correctly categorize and route 90% of inbound tickets, measured against 500 human-labelled tickets held out from development, with fewer than 2% routed to the wrong team.
That sentence lets you build an evaluation set, gives every iteration a scoreboard, and converts an opinion argument into a measurement.
Data access was assumed
This is the single most common timeline killer. The architecture presumes clean API access; the reality is a nightly CSV, a system owner who is not convinced, a data-sharing review, or a field that three departments populate differently.
Verify access in week one by actually extracting real data. Not a sample someone emails you — the live path the production pipeline will use.
There was no plan for the remainder
An AI system that handles 85% of cases is excellent. But if the remaining 15% has nowhere to go, the process blocks and the whole thing is worse than the manual baseline.
Every deployment needs a designed exception path: a confidence threshold, a queue, staffing for that queue, and a feedback loop so today's exceptions improve next quarter's model.
The pilot ran outside the actual workflow
A system that requires people to open a different tool to get value will not be adopted. If the work happens in Salesforce, the output belongs in Salesforce. Integration is not a phase-two nicety — the absence of it is why adoption stalls at 8%.
Nobody owned it afterwards
The consultants left. The prompt drifted. An upstream schema changed. Quality degraded slowly, nobody was accountable, and eventually the team went back to doing it manually.
AI systems need an owner the way databases do. Name them before launch, not after the first incident.
What a pilot that ships looks like
- One process, end to end. Narrow scope, but the full path from real input to real action — not a slice that skips integration.
- Real data, including the ugly parts. The pilot corpus should be drawn randomly from production, not curated.
- A numeric success bar, agreed in writing, before development starts.
- A verified data path, proven in the first week.
- A defined exception route, staffed.
- Production integration, even if crude, inside the pilot.
- A named owner and a handover that includes the eval set and runbook.
That pilot is less impressive in a demo and far more likely to still be running a year later.
Frequently asked questions
How long should an AI pilot take?
Six to ten weeks for a narrow, well-scoped process. Anything under a month usually skips evaluation or integration — the two things that determine whether it ships. Anything past a quarter without production contact tends to drift into a research project.
What percentage of AI pilots reach production?
Published industry estimates have ranged widely, but every serious survey puts the majority of enterprise AI pilots as never reaching production. The rate is much higher for teams that scope pilots around a single end-to-end process with a numeric success bar than for exploratory "AI innovation" programmes.
Should we pilot with our own team or a consultancy?
Either works; what matters is that your team ends up owning it. The failure mode with external delivery is a system nobody internal can maintain. Insist that your engineers are in the codebase from week one and that handover includes the evaluation set, not just the code.
What is the right first use case?
High volume, clear rules, tolerant of a small error rate, with an existing manual baseline you can measure against. Document classification, ticket routing, and invoice extraction are common first wins for exactly these reasons. Avoid anything where a single wrong output is catastrophic, and avoid anything with no measurable baseline.
How do we know if the pilot succeeded?
Compare against the numeric bar set beforehand, on data the system has never seen, measured by someone who did not build it. If any of those three conditions is missing, the result is not trustworthy.