AI (artificial intelligence) projects usually stall after the first model because nobody owns the work around it: the decision, the data feed, the integration and the monitoring. Data Department, an applied machine learning consultancy in Istanbul, helps operations and IT (information technology) leaders plan these four responsibilities before go-live, so that a working model becomes part of a weekly decision.
What does a stalled AI project look like?
A common pattern in the industry runs like this. The model is delivered in March. Accuracy holds up in testing, the dashboard looks good and the steering committee signs off. By June nobody uses it.
Nothing was wrong with the model. The planner who championed it moved to another role. Nobody owned the weekly data refresh, so the inputs went quietly out of date. The recommendations sat on a separate screen, not in the ERP (enterprise resource planning) system where orders are placed. After a few odd suggestions, people returned to their spreadsheets. Nobody noticed, because nobody measured usage.
None of this appears in a model review. All of it decides whether the project pays.
This is common. In 2024 Gartner predicted that at least 30% of generative AI projects would be abandoned after the proof of concept by the end of 2025, citing poor data quality, inadequate risk controls, escalating costs or unclear business value (Gartner, 2024). Only one of those four reasons is about the model itself.
Is the bottleneck the model?
Rarely. New models appear every few weeks, and most business problems already have a method that is good enough. The work that takes time is elsewhere: getting access to the right data, agreeing what a good answer is, and deciding where the output goes.
A related point is easy to miss. Being able to build a model does not mean it is worth building. A failure-prediction model for every machine in a plant is often technically possible. The useful question is which decision changes if you know about a failure a few days earlier. If the spare part is not in stock, the maintenance team cannot move the schedule, or the cost of downtime does not cover the cost of building, monitoring and changing the process, the model will not pay for itself, however accurate it is.
Why do working models stop being used?
In our experience the causes fall into six groups. Each has an early sign that can be checked in the first weeks after go-live.
| Cause | Early sign | What to put in place before go-live |
|---|---|---|
| No decision owner | Output is "for information" | One manager who acts on it every week |
| Data feed without an owner | Inputs stop updating and nobody is told | A named owner, an update schedule and an alert |
| Output outside the workflow | A separate dashboard or file | Results written into ERP, planning or CRM (customer relationship management) |
| No trust after early errors | Users quietly override everything | A short parallel run and a clear explanation of each output |
| No usage measurement | Nobody knows who uses it | A usage log reviewed monthly |
| Drift goes unnoticed | Error rises slowly over months | Monitoring of error and input data, with a review owner |
Which four names should you have before go-live?
Before a model goes live, ask for four names rather than four metrics:
- [ ] Decision owner: who acts on the output every week?
- [ ] Data owner: who fixes the data feed when it breaks?
- [ ] Monitoring owner: who watches drift and usage, and how often?
- [ ] Override owner: who decides when a person overrides the model, and how is that recorded?
If any of the four is "the project team", the model will probably stop when the project team leaves. The owners should be people who will still be in the business a year later.
How do you measure adoption, not only accuracy?
Model accuracy is an intermediate number. Adoption needs its own measures, agreed at the start:
- Usage: the share of decisions in scope that were made with the model's output.
- Overrides: how often people changed the recommendation, and whether their changes were better.
- Business result: the metric the finance or operations team already tracks, such as holding cost, waiting time or margin.
The third measure is the one that justifies the project, and it only moves if the first two are healthy. In a project for an automotive distribution group, inventory holding cost fell by about 10% because the probabilistic forecasts were built into stocking decisions, not only because the new method gave about 30% better forecast accuracy than the legacy method (case study). We explain the difference in why forecast accuracy is not enough.
What does a project that stays in use look like?
It is built around one decision from the beginning, as described in how to choose your first AI project. Its pilot has an end date, a business metric agreed on day one and a written plan for what happens next: hosting, monitoring, integration and training for the team that will own it.
It also leaves skills behind. For a technology group, we mentored four squads through the full model lifecycle and helped build a 30+ person ML (machine learning) and AI department that built 20+ models across 7 industries (case study). Models stay in use when the people running them are in the business, not in a consultancy.
If you have a model that works but is not used, the free 30-minute AI opportunity review is a reasonable place to check which of the six causes applies.
Frequently asked questions
Why do AI projects fail after the proof of concept?
Most fail at adoption rather than modelling. The usual causes are no decision owner, a data feed nobody maintains, output that sits outside the systems people use, and no measurement of usage.
Who should own an AI model in production?
The manager who owns the decision, supported by IT for hosting and data. A model owned only by a data team tends to stop when that team moves to the next project.
How long after go-live should we review adoption?
Check usage and overrides weekly for the first month, then monthly. Review the business metric at the end of each planning cycle, for example each quarter.
Should we switch to a newer model when one is released?
Only if it performs better on your own test set of real cases. Switching models rarely fixes adoption problems, because those sit in ownership, data and integration.