This two-part series explores the modern practices in mobile game development by interviewing 9 mobile game studios. The work is based on a master’s thesis Conquering the Mobile from Aalto University, Finland wherein you can find detailed references to cases and articles.

“When we had finished almost all of the art for the game, we decided to change the whole visual style,” a mobile game company manager explained as part of the study. The studio had been working with the product for some year, but found the aesthetics just did not resonate with the target audience. “All graphics, whether it was characters, environment, or UI was redone. It was the right move, but we did it too late. Today, we make these changes in the very early stages.”

Mobile game production requires the ability to consciously seek information throughout the development to have the best chance of success in the mobile market. During pre-production, said developers manage to establish an overview of the market opportunities and the various competitors to help shape their strategy. As the production moves forward, studios can seek feedback and validation through game testing – the results of which can often prompt much-needed adjustments.

After the possible soft launch and, of course, the game’s release, successful developers continue to improve the title, making use of live feedback and metrics. However, the research suggests there are considerable differences when it comes to company practices for gathering information and feedback.

Creating understanding in pre-production

It’s fair to say, pre-production doesn’t generate all too much hype within the game industry. It is, nevertheless, a fundamental element in development. It’s during this phase when the studio explores both the competitive landscape and the needs of the market, framing them in terms of team resources and capabilities. In short, the goal is to take the first steps in product strategy and evaluate its financial viability.

It should be noted, however, that carrying out market analysis and being aware of the particular opportunities and threats doesn’t in itself ensure success – rather, it helps avoid potential oversight. It’s only by understanding the key competitors and customer trends that a company in any industry can avoid direct competition with one of the industry giants, heading instead for blue, unsaturated waters.

The research also suggested that the smaller, less experienced studios tend to pay less attention to framing and evaluating the project before starting it. Rather than basing the project on a market possibility, these companies were more likely to base the project on the intrinsic, creative vision of the team. This sometimes led to challenges and rising awareness that the project should evaluated as soon as possible:

“Creating ideas is extremely easy. The difficulty is considering the idea in terms of the development and business. That is more difficult than coming up with an idea, and this requires product management.”

As companies gain experience, they begin to pay more and more attention to pre-production, market research and competitor understanding. Key to this, many of the developers we spoke to made use of several sources of information:

“One is all free and paid market data. There are a lot of information services and banks such as App Annie where you can check the data on other games, draw your own conclusions and do deeper analysis. Also, almost every month we are present in some game industry event, and discuss with other companies.”

Of course, more established studios may well have more resources at their disposal to facilitate such analysis. They often have more employees, and thus more specialised roles. This allows management to spend more time on market research and strategy when compared to smaller start ups where everyone is often required to have their hands on the product.

Some smaller companies have also encountered challenges resulting from insufficient planning. One studio found they developed their game for too niche audience, which in turn made it difficult to acquire users. Another company discovered the scope of the game developed was too large for the team to execute. At the same time, without established position the small teams are under pressure to perform. This creates a difficult tension where the companies have to make plans while running:

“How could I say this, we didn’t have much time or resources to plan marketing, execute it, or think about the business. There is no time to do master plans. Would it be sensible? Yes it would. Our focus was not sufficiently on marketing. Yet, it is easy to be wise on the hindsight.”

The developers that worked with particular target audiences were more aware that they needed to cater to specific player needs. In these cases, the studios sometimes found they were not the target audience themselves. Rather, they were creating the game for a group of people whose playing style might be considerably different from theirs, which led on to them gathering information on their behaviour, needs, and playing style during pre-production.

Difficulties in testing

As a game becomes more and more defined during production, it becomes possible to validate the concept with its target audience through testing. Interestingly, none of the nine companies interviewed had gone to the excess on this score: on the contrary, it could be said some of the problems encountered by said studios come from insufficient or unclear feedback.

When you add to this the fact many studios admitted their most considerable learnings came from feedback, its importance cannot be underestimated. Yet, several companies had found difficulties in gathering insightful and actionable feedback.

One of the interviewed companies had tested the game with friends and family, and found said testers weren’t motivated enough to give structured feedback. Another company found their testers focused on minor usability issues while the company wanted to understand the game experience far more broadly. A third company found feedback was useful in identifying usability challenges and discovering bugs, but contributed less towards understanding what makes the game fun and valuable to the player.

“We have put a lot lower importance on our online testing service provider reviews. The reviewers can evaluate the usability, how to use the user interface etc. That is wonderful with the online service. What it is not very good at is evaluating whether the game is fun.”

Several companies warned against listening to the views of players too closely. The players are likely to see the symptom but not the core issue embedded deep into the game system. The players are laymen in design, and it requires humility and patience to turn their comments into actionable insights.

“We didn’t do soft launch, we did some usability testing with other game developers. We got good feedback. After launch we got some feedback from the players on what they appreciated. However, you can never fully trust on what the players say. You need to deduce what they really want.”

Two of the nine companies actively sought to integrate both qualitative and quantitative feedback, believing the two kinds of data complement each other.

“User feedback is extremely valuable but also extremely dangerous because it is biased. We create hypotheses that we seek to validate through market testing. This market testing leads to numbers, data, and this is explained through user feedback.”

The nature of player feedback proposes a conundrum considering the timing of testing. The closer the product is to the launch, the easier it is for the players to elaborate on their thinking and give precise feedback the designers can take action on. Yet, the later the feedback highlights any issues or challenges, the more considerable the changes required to fix them may be. This makes late feedback difficult to accommodate to. Because of this, several of the companies had explicitly turned towards seeking feedback as early as possible.

“Discovering problems quicker, earlier in development means they cost less to fix. Every single game will have something that is not quite right. The longer it stays not quite right the harder it is to change. If you can find it four months in development as opposed to fourteen months in development, it will cost a tenth as much to change.”

The value of metrics

Soft launching a game in chosen territories was a common practice amongst the nine studios we spoke to. However, the practices behind those launches varied considerably.

Some companies opted for a rather passive approach and did not actively seek qualitative feedback. These developers used the soft launch to optimise the game and ensure its technical performance, with no significant changes made to the game or its mechanics. Some made incremental improvements through specific tests:

“We tweaked the game in soft-launch. We didn’t make any drastic changes. We fine-tuned the difficulty through funnel tests: we identified in which level the players end playing and then adjusted the levels accordingly.”

Analytics were seen as highly valuable, offering an objective, trusted metric of the product performance and its financial viability. Only one of the nine companies stressed the importance of both qualitative and quantitative feedback:

“If you are ever playing a soft launch title I encourage you to send any and all feedback to developers, because it really helps. – – Getting unfiltered player experience and feedback is extremely important. There are certain ways to understand the data without the player text input, but the feedback definitely helps. It gives context to the data.”

Making considerable changes to a game after launch is often challenging. It, for instance, requires taking into the account the existing playerbase and their opinions. One company reflected on their experience in changing the monetisation system in a live product:

“We were aware that if we make some makeshift solution it will be transparent to the users and will not work. We had to change the core logic of the game which required a lot of time and design. If I had a car and I changed change the power line and the engine, its character changes. When well executed, the whole will be balanced. However, if you just change a large engine to a small car it will not work.”

The current ethos and culture in mobile game development does little to support the use of qualitative methods such as interviews and focus groups. Big data is valued, and qualitative methods are sometimes dismissed as unreliable. Many companies have their backgrounds in natural sciences that lean towards quantitative methods and their experience in qualitative methodologies may be more lesser. This creates a vicious circle, where developers skip potentially impactful approaches and, as a result, positive case examples are not created.

Increasing learning

The developers improve the game throughout the production. However, without external feedback studios can find themselves unable to make informed decisions which, in turn, slows the development down or leads to work based on assumptions.

Iteration is the de facto way of creating any design: something is created, evaluated, and improved upon. Instead of creating a final, polished version, the work is produced in iterations that incrementally improve it in terms of its stated goals.

Game designer Jesse Schell goes as far as to state the rule of the loop: “The more times you test and improve your design, the better your game will be.”. The approach has had a significant impact on the startup scene as well through the idea of Lean Startup.

Too often, the critical evaluation of the game occurs too late. One reason for this may be that the mobile game industry lacks specific models that would guide the company focus. The iterative loop is unapologetically vague. It doesn’t state what you should create, nor how it should be tested. Useful in any field of design, it lacks specificity to address any particular industry.

To address the issue, the Design-Process-Stakeholders model was created. It guides the company to consider the key aspects of free-to-play (F2P) mobile game design particularly. It guides the company to map and test its key hypotheses with the stakeholders to validate them. The goal-oriented iterative process is structured and clarified to serve these aims. The model is described in detail in the book Conquering the mobile.

Hunger for learning

Mobile game development company goals are two-fold: to create value for the players by offering enjoyable experiences, and to transfer this customer value into revenue. It becomes difficult to evaluate whether the design is moving closer to this goal if the developer does not seek feedback from their players. Without constantly creating understanding on the product and combining this with sufficient understanding of the market and competitive products the players are engaged with, a developer operates in the dark.

Feedback is a wonderful remedy for inconspicuous ignorance. It is a tool in risk-management allowing the team to identify challenges in a game’s value proposition and improve upon them. In addition to tactical improvements, it facilitates strategic changes to the high-level, long-term product goals.

However, there is even more elevated side-effect: Feedback facilitates individual and company-level learning that spans multiple projects. Professionals in any field should be hungry for feedback that allows them to improve their performance and grow their competencies.

Whether individually or collectively, acknowledging the limitations of our understanding guides people to examine their environment critically. This open-minded process exploration and reflection paves the way for creative and imaginative solutions that also find their audience. It’s through other people’s opinions that we can choose to enrich our perspective.

Lauri Lukka is a Helsinki-based game designer and psychologist focusing on user-centered design.