Games producer King uses Looker to crush it on playtest analysis and the next level

When Candy Crush Saga turned five years old back in November last year, the mobile game’s creator King Digital Entertainment marked the occasion with the release of some pretty impressive statistics.

In its first five years, 1.1 trillion rounds of Candy Crush Saga were played. The total distance travelled by of all those fingers swiping away at all of those mobile screens amounted to some 115,000 kilometres every day – the equivalent of three laps around the Earth. Between them, the three core games in the Candy Crush franchise – namely Candy Crush Saga, Candy Crush Soda Saga and Candy Crush Jelly Saga – had accrued 2.73 billion downloads, the company said.

Every games company dreams of making that kind of splash, which is why playtesting is such an important part of the development process for any new game. In a playtest, a game under development is opened up to a small number of carefully selected players and their subsequent interactions with it are closely observed and analyzed. The aim here is to identify bugs and design flaws before a final product is released to the public.

Playtesting is a big focus at King, which was acquired by interactive entertainment company Activision Blizzard in 2016. King specializes in games that customers can play for two minutes or two hours, put aside and then pick up again later, in order to progress through hundreds (and sometimes thousands) of levels over time. It’s down to thorough playtesting of the first handful of levels of any ‘saga format’ game that the company can tell whether it has another hit on its hands, as Jonathan Palmer, Product Director of Core Data Services, explains:

First-time user experience is very important here. With our saga-format games, you want to see progression. What happens in the first ten levels is crucial. If we keep that funnel healthy, we’ll see a long tail into the game – but if you lose too many users in the early stages, you’re going to have problems later on.

Compelling challenge

As a data analytics specialist, playtesting is a really compelling challenge, he says, not just because of the amount of data created by playtesters, but also because staff in games studios – where designers and data scientists work together on creating the most engaging experiences – expect to get valuable feedback really quickly, he says:

They want very quick access to the data and very quick analysis on top of that, so they can make very fast decisions on how to develop the product. These playtests typically last less than a week and create a lot of data that needs exploring: what features are working, what engagement is like with players, where engagement drops off and so on. The idea here is that they are able to make quick decisions about what to improve on and, ultimately, whether the game is worth pursuing in the longer term.

There are a number of ways that a games company might go about analysing playtest data. At King, Palmer and his team have found their answer in a business intelligence platform called Looker, which might broadly be said to compete against the likes of Tableau and Qlik. The company, also called Looker, is pretty young, having been founded in 2012, and has raised $177.5 million in funding from investors including Kleiner Perkins Caufield & Byers and Sapphire Ventures.

Looker describes its technology as a ‘data platform’, but it doesn’t offer a database or an in-memory engine. Its key differentiator is its data modelling language, LookML, which provides an abstraction layer above SQL that data analysts use to describe how different pieces of data relate to each other and to create dimensions, aggregates and calculations.

It’s this semantic layer that initially got Palmer and his team interested in Looker, because they needed a platform that could provide a ‘single source of truth’ and a common layer for interpreting data. As they continued to assess Looker, he says, they also realized that it provided a highly usable front end that might help beyond playtesting, enabling people from right across the business explore data in a more agile way:

One of the biggest problems with a legacy BI stack tends to be that it’s built to act as a protective layer to a data warehouse. You don’t query the warehouse directly. You query a middle layer of aggregates and, straight away, you’ve made a decision about which data dimensions or measures are available for exploration. If your end user decides they need another ‘take’ on that data, you’ve got to go right back through the layers, right back to ETL.

Where Looker is different is that you’re able to query your data warehouse directly and make much quicker decisions about how you need or want to enrich the data you’re analysing. If that data is available in the data warehouse and you’re able to join to it, then you can explore it. We have a fairly technically literate users at King, people who are able to make small changes in the LookerML layer themselves, as long as they’ve got a basic understanding of SQL. And even if they don’t, it’s possible for developers to help them out, adding more and more data to what’s already available for exploration, without a costly, lengthy development cycle.

Use-case creep

At King, Looker works on top of what the company claims is the world’s largest Exasol cluster and has done a lot to boost the efficiency of games development, according to Palmer. But, he adds, use of Looker is no longer confined to the games studios at King:

These days, we’ve got hundreds of use cases for Looker. One of our biggest success stories has been around performance marketing for user acquisition. What this means is that marketing staff are able to very clearly monitor and measure campaign ROI: so, for all the campaigns we ran yesterday, how did they do, how many installs did they drive, what else did players go on to do and how many players were we able to reactivate through those campaigns?

There are others, too, he adds. In games development, Looker is being used for A/B testing of games, for assessing how games perform against forecasts and targets and for economy balancing within games (a tricky process relating to the pricing of in-game purchases and the value players get from them.) Elsewhere, King’s growing ads business is using Looker to understand ad monetization, user engagement with ads and ad health metrics. And in customer support, it’s being used to analyze help centre traffic and individual agent performance, for example:

The real differentiator here has been how quickly users from across the business have been able to get their hands on data, to add to data to create new analyses and to do so in self-service mode, without relying on centralised help. What we’ve found pretty quickly is that there are far more questions to ask, beyond playtesting, and that Looker can help us to answer them.

Image credit – King