Data Department built a day-1 churn prediction model for a mobile game studio. The model estimates which new players are likely to stop playing after their first day. It reached an AUC (area under the curve) of 0.82. That figure is the only result we report.
The decision
Mobile games lose many players very early. In general, the first day is when a new player decides whether a game is worth coming back to. A studio that can spot likely leavers early has a chance to act while they are still in the game, for example by adjusting onboarding, difficulty or the offers a new player sees.
The question behind this work was simple: can early player behaviour tell us, with useful reliability, who will not return after day one?
What we built
We built a classification model that scores each new player with a risk of churning after day one. Churn here means a player who stops playing. The output is a ranked list: players with the highest risk at the top.
How such models typically work. The following describes the general shape of day-1 churn models. It is not a description of this studio's data or features.
| Step | What usually happens |
|---|---|
| Define churn | Agree exactly what "did not return" means, and over which window |
| Gather early signals | Use what a new player does in the first session or day |
| Train and validate | Fit a model on past players, test it on players it has not seen |
| Measure ranking quality | Report AUC, which shows how well the model separates leavers from stayers |
| Decide the action | Agree what the studio would do differently for high-risk players |
What does an AUC of 0.82 mean?
AUC stands for area under the curve. It measures how well a model ranks people by risk, on a scale from 0.5 to 1.0.
- 0.5 means the model is no better than a coin flip.
- 1.0 means the model ranks every leaver above every stayer, which almost never happens with real behaviour.
- 0.82 means that if you take one random player who churned and one who stayed, the model gives the churner the higher risk score about 82% of the time.
An AUC of 0.82 shows the early signals carry real information about who will leave. It does not, on its own, say how much revenue or retention the studio would gain. That depends on what the studio does with the scores, which was outside the scope of this work.
Results
- AUC of 0.82 for the day-1 churn model.
We report no retention, revenue or engagement uplift for this project, because none was measured. A model that ranks players well is a starting point, not a business result.
Lessons
These lessons come from how we approach churn work in general.
A good score is not yet a decision. Ranking players by risk only matters if someone acts on the ranking. In general, the next step after a model like this is to test an action on high-risk players against a control group.
Report the honest metric. AUC describes ranking quality. We do not convert it into a revenue claim we did not measure. More on this in why forecast accuracy is not enough and on our churn prediction and customer analytics page.
Frequently asked questions
What did the mobile game churn model achieve?
The day-1 churn model reached an AUC (area under the curve) of 0.82. No other result was measured.
What does an AUC of 0.82 mean in plain terms?
Take one player who left and one who stayed. About 82% of the time, the model gives the player who left the higher risk score.
Did the model increase player retention?
We do not claim that. This work measured how well the model ranks players, not the effect of acting on its scores.