At one stage in your career you’ve begun to care about data. You decide that you want to know what your players are doing, so you start tracking gameplay events. You track everything. With a flurry of code your app is sending tracking events for every card combination, move, spell effect and battle stat!

Pat yourself on the back, you have created terabytes of garbage.

“Data is not information, information is not knowledge, knowledge is not understanding, understanding is not wisdom.” Clifford Stoll, Physicist

Data collected without reason, is just data. To be able to make informed decisions you need context and only by understanding your player’s intentions will you start to be informed. There are a lot of similarities between mobile games and therefore there are some very standard metrics you might look at. We’ve created a simple tracking plan that you can use to get started.

Mobile Free to Play Tracking Plan

A tracking plan is the first step to getting useful information. Use it as a guide to help your whole team design and create events. Feel free to make a copy and customize it as you see fit.  The code section on the right assumes you’re using FB Analytics and Unity but this can be easily adapted or simply left out. You can open up the template by clicking the image below 👇

The plan leans on three ideas and applies them to Mobile Gaming.

1. GQM (Goal,Question,Metric) Model – Wikipedia
2. Pirate Marketing – 500Hats – Dave McClure
3. MVT (Minimum Viable Tracking) – Mobile Growth Stack

The GQM model lays out the right order to think about your events. You always need a goal in mind then to ask a question that can be represented as a metric. Pirate marketing gives you a way of easily thinking of 5 important high level goals that matter to marketing and growing all free to play mobile games.  MVT provides a framework for focussing on creating the right events in the right order.

Together these methodologies can help you to plan the lowest number of valuable events. When an app or game is young, it’s more important to understand what’s driving your users to convert to payers.  Conversion, or lack of conversion, kills more apps than low retention. If you can’t make money from your game, you have not got a viable free to play product.

You can use the plan as a pure planning tool or you could use it to help track the implementation of the events in an app.  The plan comes with the basic events most games would need to make simple marketing decisions.  As every game is different, creating goals and questions that are specific for your game will be up to you.

Designing Questions

Using the GQM model you first set a goal for the game. Goals can be very specific or very general, but all apps face similar problems when they are starting out, they lose users and users don’t spend any money. Dave McClure’s Pirate Marketing provide 5 large goals that, if perfected, will drive high growth for your app. These are commonly referred to as AARRR. Say it with a swagger and cutlass in hand! ☠️

A – Activation – getting people into your game.

A – Activity – getting people to use your game.

R – Retention – getting people to come back to your game.

R – Referral – getting people to talk about your game.

R – Revenue – getting people to spend money in your game.

Each category provides a clear goal for your app. You want to encourage more actions in each category and you want to know why some actions are better than others. Setting goals for your core gameplay should focus on what is driving players to play my core loop more? 

After choosing a clear goal fro a category such as Activity, a common question might be “where do players spend most of their time?”.  Now, pause a second. WHY have you have settled at this question? Is it that you feel you already know the answer, “People spend most of their time fighting!” (feeding a bias) or is it “Fighting leads to higher conversions, people should fight more!” (improve a KPI). Try to always think of questions in a quantative mannor, where you’re not setting any bias but let the data speak. Perhaps “What actions do people spend most of their time doing?” can reveal some useful information. You could then decide on a set of metrics that work well in your game.

What’s driving your users to convert to payers?

Dave McClure’s Pirate Marketing talk provides 5 good mental anchor categories for most apps and free to play games to focus on. The nature of mobile gaming, in a market which is highly competitive and effectively commoditized, the Retention category becomes a strong focus for teams to improve. You may want to defer the Referral category until you have built out and got an app that has a stable and monetizing user base.  During soft launch you want to also focus on the FTUE / top of the funnel actions because this will be where most of your users will churn, there is a separate tab on the plan specifically to map out your FTUE. Remember conversion is king and even if you have a high churn rate of say 80% in day 1, if you have a high conversion rate of >5% then you still have a viable free to play product.

To further aid all tracking I suggest you always create an XP system and provide it as a dimension in core tracking events. 

XP is one of the best and simplest ways of segmenting your users.  XP is an ever increasing metric based only on actions in game. You should always try to include XP as a system in your game, even if you don’t surface it to the player, as it’s so useful for tracking.  Using XP levels as a bucket to separate users who have performed a similar number of actions allows you to detect points in game time where your designs start to fail.

The goal of all tracking is never to tell you what to build, it can only tell you how badly you built it as no feature is 100% effective.

Designing a Tracking Event

Let’s take the classic Base, Battle and Build game, Clash of Clans.  As a game designer you may want to know the answer to “How often do people win their battles?”

Firstly, we must establish that battling is the critical element in this question.  For now we only need to know just enough about the battle from the point of view of the defender and attacker. In Clash of Clans people search many enemies before committing to an attack so the first metric should be a count “enemiesScanned”. Next once an enemy is picked the battle is commences, to answer the question we don’t need to know anything about the battle itself such as which troops, how much gold the base contained or what Town Hall level they are, we simply need to know the win or loss rate.  Even though this extra information is interesting, stay focussed on your initial question for now.

There are 4 outcomes from an attack in Clash, either 1⭐, 2⭐ or 3⭐, or a loss 0⭐.  Therefore we need to record these outcomes as “attackOutcome” we may want to go further than this and also send over the “totalDamage” as an integer 0-100. This might give us some useful information in qualifying what type of win it was.

Here’s how it might look on the tracking document:

KPIs and custom KPIs

A developer should now be able to create an event on any platform and you should be able to quickly check how many battles result in a win (1-3 ⭐) or loss (0⭐). To add some more information to the result the enemyScan/Battle ratio could be important as more scans might have a big effect on BattleWin rate. If you feel satisfied that this has answered your initial question, you can now repeat the process to add more depth. Create a new line and add a new question and try to use as many of the same variables as you have already created.  You may want to know if battle win rate varies if the player is a Payer/Non-Payer or changes with XP Level?  Don’t go wild here or you will suffer from analysis paralysis and then the outcome won’t be useful. It’s always easier to add than to take away, start with the minimum.

To make life simple, games use common KPIs to measure themselves against other games.  Many KPIs come as standard and are useful benchmarks, DAU, ARPU, ARPDAU etc. Standard KPIs are a great way of comparing games with each other. However, if your game does not fit within normal ranges, you don’t always need to worry.  Your game might excel at some KPIs but not others and you should capitalize on that.  If all your KPIs are low, you will have a hard time building them back up and you might want to Kill your Project.

Related Read

GDC Europe 2014 : “The Art of Killing Games”

In Industry & Market Trends

Adam Telfer

Creating a KPI target will allow you to aim for something with your tracking. Coming up with the target requires some research (Adjust Mobile Measurement Benchmark Report) for your games genre as each can vary wildly.  We’ve also seen many benchmark metrics drop over the years as competition on the app store gets more fierce.

You can and should set your own custom KPIs.  These are measures that are specific and matter greatly to your game.  When creating a KPI try to think of it as a number that is critical to the well being of my game or users.  You can then start to measure and improve it by adding features that help or communicate these functions to your players.  Uncommon but generic Events/KPIs that I find helpful are:

• % Tutorial Complete – how many people get to tutorial complete event?
• Completed Views per DAU – (video ad monetisation)
• Min,Max,Median Rounds to complete – (level, world, track, mission)
• 1st gem purchase – what did a user buy after their first IAP?
• Playtime Tracking – count of total number of game minutes inside your app?
• Sources/Sinks – Which actions are providing most currency and most spend in app?

Every game is different but usually when you playtest or watch other people play your game you will notice points where people “get it” or moments of joy where every tester got really excited and wanted to play more or play further.  When coming up with a KPI talk with your team and be quite limiting initially.  Work hard to establish which custom KPI will help with the project in the next 2 weeks, not one that you think you’ll need in the distant future.

A Good Dashboard

Displaying your data in a simple format is the next step. There are a number of different dashboards you can use or create and many services that provide them.  It’s a choice of preference which analytics provider or dashboard you use as they don’t need to be from the same company.  The best companies try to own their own data so that they have flexibility in how it can be presented.  This is no small task and so only larger studios with strong revenue should consider building it themselves. Free services do provide good dashboards but are less flexible and you will loose or share direct control over your data.

A Good Dashboard

A Bad Dashboard

There are many way to represent data but I like to try to keep the following in mind.

1. Every piece of data needs a comparison period if possible.  Usually 7 Days or 30 Days will work best
2. Use colour to represent clear changes in figures when compared to that comparison figure
3. Display real numbers where possible to get a sense of the data.
4. Every chart needs a scale or an axis
5. Flexibility to flip charts to display data by major shared dimension, such as XP or Days since install.
6. Have separate dashboards that answer specific questions or topics in your app. Not one dashboard to rule them all.

Dashboards don’t need to be pretty, they need to be informative and they need to be good at revealing trends.  No matter what your data is telling you, the trend in your data is more important.  If you are increasing your 1D retention by 0.5% per week for the last 4 weeks, that’s much more valuable than a 1D retention being 32%.  A dashboard should clearly reveal the trend and allow you to filter or view these trends against time or app version.  Versioning is an important aspect of software development as it allows your to clearly know the code that people are viewing.  Any dashboard software that makes it easy for you to filter your views by version help you to see trends as your software develops.

Be aware of your Dashboards audience.  Some dashboards and KPIs are meant for marketing teams (top level, money orientated) some for your design team (progression, churn, feature usage) and some for your developers (errors, crashing, load times) if you’re creating a dashboard be sure to check that it’s the relevant information for the audience.

Summary

Everyone puts analytics in wrong the first time.  We all believe we know how our players are going to play, but once real users use the system things change.  

If you’ve not sat down and used the GQM approach before then give it a try.  There’s nothing worse that joining a project and realising that all of the data is garbage and that picking through and joining tables will be a tiring a painful process.  Starting with the smallest but most useful data points will allow you to stay razor focussed as you develop your game.  Using a tracking plan will also pay dividends in the long run as you can go back through and remember why you tried to track something in the first place.

Tracking needs to work for the project and to do that it needs to be easy and accessible for all.  I’m a great believer in sharing your data openly and freely with everyone on the team as it allows everyone to see how their work affects the product.  A great set of dashboard is the best way to do this.  The whole process of using data effectively is not a simple task in game development, but the best companies use it every day, and the greatest companies create a competitive advantage through it.  Start small, focus on events that matter and continue to make informed design decisions.

All games feature three dimensions that determine player success: skill, stats and luck are required to win and progress in a game. However many prototypes and many games that I play each day seem to struggle with this mixture.

The main reason this happens is the player suffering from a lack of transparency  about how they can improve. If your players can’t answer the question: “Why did you lose this round?” — then you have issues in transparency.

You need to seek ways to isolate and clarify the impact of Skill, Stats and Luck to make your game engaging.

The Importance of Stats, Skill and Luck

To start to fix these issues of transparency, it is important to define what the balance of stats, skill and luck your game is. Each is important for the success of your game.

Stats are essentially everything that the player invests in outside of the core gameplay. Most games have RPG stat systems that players are seeking to upgrade. For example, upgrading building defense stats, upgrading unit abilities, or upgrading the speed of a car. These systems are most likely what paces and slows down the progression of the player. A strong stat system is necessary if you want to ensure a level 100 player feels dramatically more powerful than a level 1 player. These systems are most likely what your players are investing real dollars to purchase quickly and to give them an advantage in gameplay. Without stats, long term gameplay is extremely difficult — there’s nothing players can invest in to get better at your game.

Luck is any systems in your game that are purely random. For example, the gems falling down during Candy Crush or Hearthstone deciding which card will be drawn next from the deck. These systems are important to create drama and entropy in gameplay. Luck ensures that the gameplay never is boring since a positive (or negative) action can happen at any time. Luck also widens the breadth of skill needed to master the game. Being able to craft the best deck is easy, but being able to handle the near infinite possible outcomes of card order is an extremely difficult skill to master.

Skill is any player choices or player interactions that have an impact on the outcome of your game. Skill is the player’s ability to time jumps in Mario, their ability to effectively place units in Boom Beach, or their ability to choose between cards to play in Hearthstone. This variable is by far the most important to a player’s retention to your game. Players that feel the game has no skill will not stick to a game. Fun and engagement come from learning and feeling smart about one’s decisions (Theory of Fun, Raph Koster). Without skill, there is nothing to learn, there is no game.

Clash of Clans focuses on Stats and Skill. Stats comes from Units & Buildings. Skill comes from the player being able to choose location of attack.

For example, Clash of Clans clearly focuses on Stats and Skill. There are very few sources of luck in the game. Players seek to upgrade their buildings with currency to progress in the game. Stats is clearly the strongest focus for Clash of Clans to ensure that progressing and upgrades are felt and long-lasting. Skill is secondary: players optimize placement of buildings and placement of units to defend and attack. Skill is controlled and balanced very closely by Supercell.

Candy Crush focuses on Luck and Skill. There are no upgrades in Candy Crush, just boosts. These boosts can’t be upgraded, and give a very clear (transparent!) benefit to the player. Clearly Luck is their largest focus, ensuring that each move a player chooses they never will be able to predict fully what will happen. This creates casino-like “near misses” that are only cured through replaying a level or paying for boosts.

Different games on the top grossing chart focus on different balances of Stats, Skill and Luck. A rule of thumb is to select two of stats, skill and luck as your game’s focus.

The more factors that are included in your game, the more likely you will have issues with transparency. When the balance of Luck, Skill and Stats is unclear to the player, engagement suffers.

Be as Transparent as Possible

Fire Emblem Radiant Dawn. A Tactical RPG game. Control Movement of Units.

Let’s take for example a typical tactical RPG battle.
Each battle has a significant amount of variables that have an impact on its outcome:

• Each unit has stats for attack, defense, speed, attack range, etc. (Stats)
• A player can choose what unit types they want to bring into the battle. (Skill)
• Each turn the player chooses where their units will move and when they will attack (Skill)
• Each attack has a critical chance to do extra damage (Luck)

After a battle is concluded, the player lost. As a result, a player must ask themselves — what did I do wrong? How do I improve? What do I need to do in order to win this battle?

Fire Emblem. Stats, Skill and Luck go into the result of each battle.

It could be Stats: they need to upgrade their units more.
It could be Skill: they made poor decisions in the battle.
It could be Luck: they were just unlucky with critical hits.

Ensure your game is giving the player a clear answer.

When there are many variables determining your players’ success, it is best to isolate each variable as efficiently as possible.

When there are many variables determining your players’ success, it is best to isolate each variable as efficiently as possible. To isolate, you don’t always have to give feedback immediately when the action has occurred. Feedback on each variable can come before, during and after the gameplay.

Isolate Stats, Skill and Luck

Immediately from looking at the bar, I know that this is an “Extreme” race. To win, I need to upgrade.

CSR Racing is an excellent example of how games can isolate each variable. Before a race, CSR Racing isolates Skill from Stats. Using a small progress bar, they compare the player’s stat level against the opponent. If the race is going to be extremely difficult, the player is warned before they begin. This helps the game in two ways:

1. Drives desire for Stats. Stats are given a clear value in the game. Players clearly see that if they improve their stats, they improve their chances of winning.
2. Avoids unnecessary losses on levels that are too difficult. Many times players will blame their skill above all other factors. If they continually play a level that clearly isn’t balanced for their level, they will eventually feel the game is unnaturally difficult and leave.

CSR isolates Stats before the race with this small UI change.

Throughout the race the player gets clear feedback about their Skill.

CSR also seeks to isolate how much skill impacts the outcome of a race. During the race, the player gets very clear feedback about their timing skill. “Bad Shift”, “Good Shift” and “Perfect Shift” come up throughout the race giving immediate, clear feedback about how well the player is doing. At the end of the race, the player clearly can say “I had all perfect shifts! I did everything I could!”. Vice versa, a player can say “I had some bad shifts in there, maybe next time I can improve my time…”.

Through these two small mechanisms, CSR has effectively isolated skill from stats. Players clearly know the reason for winning and losing, and can decide the best course of action to improve their chances.

A Tool for Transparency

To start to fix issues of transparency it is imperative that you play test your game often. When discussing the game with playtesters be sure to ask after each core gameplay round :

“Why did you lose this battle?”
“What could you have done differently to win this [battle/race/round]?”
“How did you figure this out?”

If they can’t answer these questions, it is time to improve your transparency.

To improve, list the inputs (Variables) and outputs (Feedback) of your game:

• Which variables impact the result? ex. Unit Stats, Unit Placement, etc.
• How do we give feedback about each of these variables?
• Which feedback systems are actually working? (players are acknowledging them in playtests)
• Can we give strengthen feedback for this variable before, during or after the round?

Strengthening feedback is dependant on whether the variable is stats, skill or luck driven.

Pokemon is a good example of giving Skill-based feedback. Ensuring “Super-effective” comes up directly after the action make sure that players know that it was their skill that improved their outcome.

Variables that come from skill usually require more direct feedback immediately after the action is taken. For example, Pokemon gives the feedback “Super-Effective” immediately after a player has made a good choice about what attack to use against the opponent.

In Mario games, whenever mario is killed by a baddie, the game pauses and highlights exactly where mario was hit. This small and subtle trick ensures that players clearly see the reason for their failure and helps with learning.

Slots, Dice, Cards are all clearly luck based. Use them to ensure players know what is random and what is not.

Variables that are luck-based can be made more transparent by using expected visuals of where luck comes from. Use a deck of cards, a spinner, or a slot machine. These are expected to be pure random. Players will quickly understand that they have no skillful impact on these results.

Boom beach warns players when their opponent’s level is too high, and gives players the ability to “scout” before committing. Ensuring players can make the right choice about who to attack based on their Stat level.

Variables that are stats-based are easiest to give feedback before the round begins. Ensure you warn players before they are going into a battle without the necessary upgrades.

As you add strength to these feedback systems, continually play test your game. When players have a clear idea of how to improve their chances, your game will be far more engaging.

Summary

To increase engagement in your game, seek ways to make your systems more transparent in terms of skill-based, stat-based and luck-based variables.

Stats, Skill and Luck each have their own benefits and drawbacks to a game’s success. Ensure you know the focus and balance for your game. Balancing for high levels of all 3 is exceptionally difficult, and can only be accomplished through isolating Stats, Skill and Luck in a transparent way.

To ensure high engagement to a broad audience, be as transparent as possible why a player won or lost in your game. Listing out the variables leading to the outcome and analyzing whether the player has enough feedback is the first step to reducing confusion.

Your goal as a designer: Players should always know how to improve.

Why is it that in the Top Grossing Charts on mobile there are no games with high amounts of player skill?

Where are the Marios? The games that tested your abilities right to the last boss?
Where are the Call of Duties? Games that allow you to play competitively online for hundreds of hours?
Where are the Street Fighters? Games with so much emergence and depth there are books on how to master the controls for the game…

Mobile renditions of these genres may have found a way to get a decent amount of downloads, but none of them have found a sustained spot on the Top Grossing.

Why?

In order to be successful in free to play you need to pace a player’s progression so that they can play your game for months.
For a game to enforce slow progression for all player types,  you need to be able to balance with high precision.

The more player skill effects the outcome of your game – the harder it is to balance for optimal monetization.

How Candy Crush handles Skill

If Candy Crush was balanced to have less luck and more skill, the game wouldn’t be nearly as successful.

For one, it would not have broad appeal. Players without the necessary skill levels would drag behind and leave because of the difficulty, while players of high skill would rip through the content without spending a dime.

But the major reason why Candy Crush Saga is successful is the way that the game paces its content. Candy Crush has a lot of levels, but with 300+ levels, it still takes you months of engaged play to even dent the map. This stalled speed of progression is exactly what is required to be successful in Mobile Free to Play. A game that lasts for months (better yet, years) with a healthy pace of content to keep players engaged. This drives a strong long term retention: a large percentage of players returning to the game after hundreds of days. This KPI is the most important measure when evaluating a games success, we have seen this time and time again at Wooga.

So how does King do it? How do they pace the content so well so that players only reach the 100th level after playing for a month? They do this by varying up the difficulty. The difficulty of levels is not a steady linear increase like in most classic games. Level 55 is not necessarily easier than level 56, level 100 is not necessarily easier than level 200. In Candy Crush (and all of the games that copied the formula thereafter) there are levels that are meant to be easy, and levels that are meant to be hard. A set of levels are designed as easy to make sure that you have moments when you are loving the game and feeling smart/powerful. However, there are also levels that happen more sporadically which ramp up the difficulty exponentially. These levels are “blocking” levels — they are there to be extremely difficult. These levels are required to convert players to payers (give them reasons to use all those boosts), reinforce that the game is not a cakewalk, and that level progression should be celebrated, (see twitter…) but mostly so that there are levels which have to be played over and over and over and over again before you progress – pacing the content.

Florian Steinhoff did a wonderful GDC presentation about this exact balance when he discussed Jelly Splash. I’d really recommend watching it if you have vault access.

The problem facing game designers looking to create new genres in Free to Play comes back to Player Skill. How do you build these blocking levels that are so important to your retention & monetization? If a player has a huge influence on the result of the round (whether they progress or not) then balancing for a skilled player versus an unskilled gamer would be impossible. I can beat the new Super Mario Bros. in a few hours, it takes others with less experience years to do the same. How Candy Crush builds these levels is by making the chance of winning, regardless of your skill, low. Like 5 to 10% (sometimes I’ve heard numbers even lower). But isn’t this frustrating? No! Since the game has so little skill (in comparison to other genres), they can balance these levels to make sure that players consistently come close to reaching the goal. Those near misses everyone talks about. These keep the player feeling like they can beat the level, they just need to play a few more times or convert. With higher player skill, this becomes much more difficult to achieve.

Super Mario Saga

So let’s go with a little bit of an experiment. Let’s take a game that has high amounts of skill and try to pace it without resorting to “dumbing down” the mechanics.

Super Mario is a game everyone loves – lets assume (for this experiment) that you could actually get the same level of control and ease of use out of a mobile version of Mario.

To play the game, the player progresses along a map, working between worlds which have unique content in each world. It takes months of time for artists, designers and developers to build each world. We need to try to make this amount of content last for months in the hands of players as well.

First off – we ramp up the difficulty on some levels like Candy Crush did. This will prevent players from progressing too quickly and force them to master the game. However, then we’d have a retention issue. The game only appeals to a small niche, and most players are dropping out because the game is too punishing and unrewarding to play.

So let’s add an upgrade system! You collect coins from playing, and then you can use those to improve the runners abilities to pass those levels. For unskilled players – they can grind on previous levels if they need the boost, but now they can progress! Games like CSR and Deer Hunter do this very well. Players can upgrade their weapons or car to improve their chances of winning. Blocking levels are directly tied to your upgrade level, and grinding is a core part of the game loop. However, in our Mario game we start to get some issues. Compared to CSR and Deer Hunter, there is substantially more skill in our game. So a player beating a level has a lot more to do with their current skill level than over how much they’ve upgraded. Those blocking levels are not forcing you to upgrade at all, it’s just demanding more skill. So in the end we have no control over the pace of skilled players, and unskilled players are forced to grind. No fun.

Here in the bottom right you can see the recommended gear for completing a level. In order to actually ensure a player NEEDS this upgrade — you need to make sure their skill isn’t overpowered and they can beat the level regardless.

What else can we do?

Games like Hearthstone and Diamond Dash are games of high skill and have performed very well in the Top Grossing charts. These games use Multiplayer PvP to control the player skill.

Tying PvP (Multiplayer) to your progression is one way that high skill games can pace their content. Ranked Mode in Hearthstone is an excellent example of how to do this. You may be a highly skilled player, but matching you against an equally skilled player and tying the result of this match to your progression allows High Skill games to balance out.

Games like Brave Frontier put most of their Skill into the decisions made in the metagame/elder game. The skill from Brave Frontier is not in the battle. Players are asked to mindlessly tap to fight against opponents. In later stages they even have an automated mode when you bore from mindlessly tapping. The skill in Brave Frontier comes from the choices you make as a player about which monsters/fighters you want to upgrade and which ones you take away. These types of mechanics certainly take much longer for players to appreciate and master, but in the end these are the mechanics that drive players to play these games for months rather than put the game down because they’ve bored of the core mechanic.

Takeaways

So what’s your take away? You can say two things –

1. I’m going to be that crazy game designer that cracks this nut. I’m going to go out and design a game with high skill that will dominate the marketplace that has strong retention and amazing monetization to shoot up the top grossing.

2. Or I will sit back with most free to play designers and continue to find ways to subtly water down skill based mechanics so that we can keep players in the game long enough to monetize and turn them into dedicated players.

I strongly believe that the mobile marketplace is maturing. The current marketplace is slowly demanding more skill from their games. Players are becoming fed up with re-hashed mechanics from a couple years ago. Players can see through Candy Crush’s mechanics and are not sticking to these types of games like they used to.

Finding ways to smartly add skill to games will be the key to opening up new genres on mobile.