Using data to improve your game’s retention and revenue
The innovations which have elevated the games industry over the last year haven’t just been about the free to play model. In fact, arguably the most important change has been the rise of games as a service and in particular the use of Data.
I’m lucky that for much of my career I was the games guy in Telecom companies so I was exposed to the love of data very early; but for many this topic is still new. In this post I plan to give you a quick starters guide to practical ways to use data to reduce player Churn and improve both revenues from Ads and in app purchases. More than that though I want to show how important it is to use data to create a dialogue with your players and allow you to respond to their changes as they become more engaged with your game.
First thing to understand is that players’ reactions and choices aren’t static; they fluctuate over time. There are patterns but these are specific to each game. If we want to understand how to motivate players effectively, we need to understand where in that journey they are and how different groups of people react at those lifecycle stages. I break these down into four sections, Discovery, Learning, Engaging*. Understanding what (and why) players actually do at these stages is key to determining our best response. In this session I will also go into more detail on the Engaging stage itself and look at the specific of Super-Engagement and Re-Engagement as well. A video of the webinar recorded on Tuesday, May 3rd 2016 can be found at the end of this blog.
Terms of Reference
Before we dive into the player lifecycle, we need to make sure we have a common set of terminology and an understanding of the kinds of data we can access and what they mean.
Type of Data
What is is?
How we can use it?
Knowing who our players are
Events recorded within the game
Knowing what players did
Performance data of our systems
Knowing how stable the game is
A group of players with similar set-up
Creating a common base-line
A way to monitor stages of play
To locate where we can improve
Personal feedback from players
Understanding Rationale of Behaviour
How Do We Capture Events?
In order to get the behavioral data we need, we capture the events or activity the player actually did in our game. Unlike qualitative research, we aren’t looking for opinions or vague memories of players; we need to know what specific actions/choices they actually made. To do that, we create a series of events which are typically player initiated trigger points which tell the game to capture some values.
For example, we might wish to capture that a player was shot in a FPS game. We might give that a name, say PlayerShot, and then store a range of variables with that event. This could include the Date and Time that the hit was resolved, an anonymized PlayerID, the X,Y,Z coordinates of their Avatar, the damage taken, and the anonymized Player ID of who fired the shot as well as the Session ID for that game.
Importantly we don’t have to capture everything. There are some kinds of data which are static reference information. For example, the specific position in a specific map. As long as the version of the map used at that time is known, then X,Y,Z coordinates alone can be used to create a heatmap later. We can also infer a lot of data from other events as long as there is some connecting information. For example, we don’t need to capture the Level that the player is using for that game in every event or even a list of all the players in that session. We can capture that information with a specific Start Session events and use the associated Session ID to allow us to identify everything that happened in that specific game session.
Notice we want the players to remain anonymous. We don’t want or need to spy on our players, but we do need to understand how the game plays across all players.
What Events Should We Capture?
Because we are trying to understand the Lifecycle of our players it’s important to think of the game events in terms of the timeline in which I might encounter them. I find it useful to map out the player experience from the flow of the first time user and separately from the flow of the repeat user. Personally, I’m not necessarily looking for every button press, I’m looking for moments which include meaningful choices. There is an approach used by the Food industry called HACCP. I wrote about this for Develop Online, so I won’t go into detail here, but in essence the point is that we are looking for the “Hazards” such as whether they churn (i.e. leave the game) but also trigger points for more positive action such as paying for an IAP or watching a Video Ad.
It’s also important to recognised that the data we collect will be incomplete, for example if the battery dies or the player switches to a phone call – we will probably not get the last upload. This is less of a problem with a server-based game but it’s never 100% and the compromise is that the game can’t be played offline; impacting our chance for them to create habits of play.
Some typical events could include the following (apologies for the use of pseudocode variable names):
From creating events in this way we can infer a huge amount of information. For example, if we want to know the percentage of players who complete a level we can count the number of GameMenuLaunch events with the number of LevelComplete. But we can also get smarter with our analysis. We can look at how many people completed a specific ObjectiveID in a specific LevelIDSelected and compare that to the number of LevelComplete in the subsequent level to find out if skipping objectives in earlier levels have a particular impact on performance later.
It’s not just about comparing events in terms of number, but by looking at the timestamps of specific activities in our game we can identify other deeper issues. For example, if we start to notice that the amount of time spent on the menu page starts increasing, this might be warning sign that they are about to Churn or that changes to your menu page are confusing players.
What Don’t We Know?
Data capture like this can be very powerful, however, the way you capture information will be inherently biased towards your understanding of the way the game is meant to be played. This means that we have to constantly review our metrics. The biggest issue is that we can’t capture what players will do or might have done. That might seem obvious, but we will be using insights from analytics to inform our design decisions. If it’s true that only 2% of players spend in a free to play game, then that means we don’t know what would have triggered the other 98% to spend. There might not be anything that would have convinced them, but the point I’m making is that the data we capture cannot ever be complete and that means we haves to consider statistical significance and be very wary of assuming causation rather than correlation. Looking at the lifecycle of a player helps us mitigate this, because we can break down each decision stage and look for ways to increase the likelihood of converting players at each stage. It’s all about asking the right questions at the right time.
There are some great techniques out there to help us look at data as part of the player lifecycle. In particular one of the most useful is called Funnel Analysis. This method allows us to look at a set of players and identify how many of them go through to the next stage of the game. The most often quoted version is the ARM funnel.
With this we look at the number of players we acquire (often as total downloads, although the number of GameMenuLaunch is arguably more useful) through organic installs, direct advertising or virality. Then we compare that with the number of those players who also play the game on subsequent days. When we compare players on the second day (which I call D2 – others call D1) with players on the 7th day (i.e. D7) or the 30th day(d30) we can get a pretty good idea of what our retention looks like. Then comparing that with the % of players who pay, we get a good idea of our success in terms of monetization.
However, this doesn’t give me enough detail. So when running a game as a service, it can be useful to expand this funnel to reflect the player’s lifecycle – I call this the Service Funnel. This helps because it allows us to think about the role of the Engaged player as helping feed not only virality (which obviously has diminished over the last few years) but also the willingness of others to spend more in our game. This is why Freeloader players actually add hidden value in the game – as well as their income from, for example watching rewarded Video Ads.
This way we can better map the flow of the gameplay with the way that we engage and retain users; and more importantly look at the factors which create a Repeat purchase in the game.
Can We Make Better Games?
All this may seem quite commercial and dry, indeed there will be some designers who will think that this kind of approach kills creativity. Actually, the reverse should be true. This is about using evidence to create the most engaging game. We want our player to continue playing over time and if they feel good about spending in the game then they will want to spend money more than once (as long as we continue to give them value in doing so).
The breakdown of our game into stages of engagement matters and allows us to use gameplay, Ads and In-App Purchases to increase enjoyment of play.
However, data has to inform the designer, not become a barrier – qualitative research can help. Asking players and observing them using your game can be invaluable as long as you understand the findings are usually only useful to help you understand motives. Focus groups and online surveys help provide some insight but remember that players are essentially incapable of accurately telling you about what they did, let alone what they will do; but they can tell you how they feel about your game and why they made certain choices. However, this needs to be done in a formal setting – asking your friends down the pub isn’t usually that useful. Direct feedback can be very useful but again we have to be careful about how we use it as it will usually describe the game that the specific player would have made, not what the wider audience need. However, it’s always important to listen and understand.
What Are We Looking For?
Every game will be different, the following is what I’m generally looking for in terms of the performance of the game and the behaviour of players at each of the player lifestages for each. The key question is about how we get players into that stage and what we need to achieve to transition them to the next stage.
Discovery is the very initial transition stage from players becoming aware of the game to their action to install it on their devices. At this stage, it’s key for us to understand if we converting Players to Download; but also to actually play the game!
Attribution – we need to know not just which sources give us users; but which bring value for retention and monetisation as well. Where you can track these as separate custom segments and compare behaviour.
Always compare players by their elapsed time playing. Often bringing Custom segments with the same start date works really well; although it can also be useful to compare players based in the number of days of play (even if not concurrent)
Whilst many ad networks use a combination or CPI or eCPM for calculating the cost of advertising campaigns, we need to look at the effectiveness of every campaign in terms of its % Conversion To Download
Downloads aren’t the end of the line – do you know what % Ever Launch the Game or how long it takes them from Launch of the App To playing the game?
Learning is the first time user experience for the player, but only ends when playing this game becomes routine. That’s vital, as we are looking to truly engage the player and help them become fans. The core question we must ask is if players understand how to play and how the game fits into their lifestyle!
What is the retention on the second day of play (I call this Day 2 Retention)? If we can’t get them back the next day, the game isn’t going to be interesting enough to them to pay for content. Even if this is a premium game abandoning the game so early is a bad sign.
What is the Frequency & Duration of Sessions? This is going to vary with each game and there is no one level which is ‘good’ or ‘bad’. However, what we are going to be interested in is how this rate changes over time.
Playing the Tutorial is rarely fun. But do we know what % Initiate Level 2? Or later levels. The rate and pace of unlocking new content can indicate the genuine interest in the game.
During the Learning Stage I’m not all that keen on converting users to paying – largely the early payers are often one-time payers. However, it is essential that we set an expectation of value for the goods. One way to check that is to understand the % Who Watch Video Rewarded Ads
It’s often useful to keep track of the Last Position Played in a game session. This will often indicate natural breaks and places where you should have a ‘Call Back Action’ i.e. a reason to set an appointment to return the game at a later date.
At the Engaging Stage we really have a true fan. This is the point where we really need to focus on retention and expectation of value. As a designer, I value “Repeat Players and Payers”. I see one-time payers as a red flag that something is wrong with my monetization. Key to revenue generation is to create an expectation of value in the player. Show them the value of the items (often assisted by access to premium elements through rewarded video Ads). However, the key question we need to ask is: are they ready to pay?
The key metric is still retention. The impact of long term commitment on willingness to pay and how much is linked to how long they keep playing. This may sound cynical but it really isn’t. It’s a reminder that we need to make a game so good, so enjoyable that player will want to keep playing. What is your Day 7 Retention?
Every game is different so there are no hard and fast rules about % Conversion To Pay. However, it’s important that spending money in a game should feel great; and not victimise non-payers. Imaging playing Scissor-Paper-Stone. If I buy the Lizard-Spock upgrade this gives me more choices and all other players an exciting twist. But we can all still play together.
Just as during the learning phase, the frequency of Watching Video Ads can give great insight to the attractiveness of your IAP and if it rapidly changes, this can help identify a willingness to make the initial conversion to buy (or indeed a risk of churn). It’s likely that your biggest spenders will also be heavy ad users too.
Pay attention to the average level of success and failure of your players during the Engagement phase. Frustration can be a great driver to commitment, but it can quickly turn against you.
Pay attention to any obvious deviation from Average Success/Failure Ratio.
Another neat trick is to map the first point in the game where a player decided to watch a Rewarded Ad or paid for IAP. It can be really useful to understand why – not just where. We need the experience to be fabulous if we want them to do it again!
One of the things that separates a game with good IAP from a premium game is the ability to people who really fall in love with the game to want to invest more into the experience. Again, this should not be a cynical exercise but we should be able to work out through analytics not just which Players are ready to get serious, but more importantly what is it they really value.
Again, my personal choice is to look at Day 15 Retention; this is a very serious level of commitment regardless of whether they are spending or not.
Keep an eye on the % of Repeat IAP Purchase as well as the frequency & changes in Types of Purchase. This is important as especially of you can identify better value goods which help extend retention even further. This about why the goods are likely to give players a reason to return. It’s important that players feel they really got value for what they spent money on.
Also look at the level of social communication. Social capital is an essential element to the perceived value of items in your game, just as much as their gameplay value.
Players will stumble and fall out of the game, but we should look for ways to stimulate their enjoyment and ensure that they continue to get the most from the experience. If we can pre-empt a problem or frustration point, perhaps even offer new content this can help players keep doing what they love about our game. But sometimes they may just be ready to churn.
Players are normally stimulated by an update, but look at the different cohorts of players to see if some fail to re-engage. Any change in behaviour after updates can provide important insight.
Watch out for the Rate of Change in frequency of sessions, purchase/Video Ad behaviour, Social Communication, even the Average Success/Failure Ratio. If this slows, can you act to stimulate interest?
Players leave. It’s inevitable. Our job is to keep their interest as long as possible and give them a stimulating, entertaining playing experience as long as possible. Recognizing that the time has come is as important as recognising when we can re-engage players
What was the pattern in terms of rate of change of session length, time between sessions, response levels to updates prior to the last play session
Are those players still willing to recommend the game to others
How would those players rate the game now they no longer play? This is key for me. Because the more players who leave and remain happy, the more better chance they will return for the next game.