7. Mobile Games Benchmarking

Discussion and methods for (early stage) mobile games benchmarking.

How do you usually approach (early stage) mobile games benchmarking? This article discusses about benchmarking and introduces couple methods (/’schools’) over it.

The article is partly related to (/complements it) also my previous article about importance of Business Modeling (https://gamesalchemy.substack.com/p/6-why-business-modeling-your-f2p):

6. Why Business Modeling Your F2P Mobile Game is Important?

How To Approach Benchmarking?

There seems to be on high level few ‘schools’ to look into:

1. One goes deep in understanding KPIs from macro tools such as Sensor Tower, data ai, AppMagic and other sources of data (ad networks, MMPs, industry reports, industry peers, etc.), and using them in combination for benchmarking.

2. ‘Schools’ that with current commercial sentiment aim, e.g., for hybrid casual games to have 40-50% (or even more) D1 and < $1 CPIs (US) for starting point — to invest further (kinda overlooks benchmarking, but counts in a target profiling of top 5-100 games in certain categories).

3. ‘Schools’ that start with scientific approach where e.g., marketability (KPIs e.g., IPM, and later down the road CPI, are looked into) comes first — afterwards application of 1. and 2., jumps in for further steps.

There's practically nothing wrong with any of these, given one understands what type of business cases it wants to validate and drive further — as well as how much risk one is willing to take and/or how much one needs to de-risk certain things.

And, going further, basically in terms of what data you get and will use, and how you use them, you’ll build your benchmarks from them directly, indirectly, or by combining them through different means, e.g., without limitation, by averages.

Benchmark Types

Here’s a list of usual types of benchmarks:

• Direct Target: Targeting directly specific KPIs set based on benchmarked game(s);

• Adjusted Targets: Targets adjusted per current development stage, which comes from your benchmarks e.g., from live competitor titles;

• Scientific targets: Direct targets focusing on following your scientific process with certain ambition level; and/or

• Category Percentile Targets: Targets aiming to certain category top levels, e.g., 75 percentile, 50 percentile, 25 percentile and/or 10 percentile, as the case applies.

What Else There is to Note?

Here’s a long list of items you need to take into account with benchmarking:

The need to understand commercial facts, such as:

1. How long benchmarked products have been in the market vs. your product’s stage / development stage you’re forming benchmarks for.

2. How much others spent for UA, and how this effects to their KPIs you’re seeing from different sources of data.

3. How aggressive monetization strategies others use, and how this effects to their KPIs you’re seeing from different sources of data.

4. How aggressive UA methods others use from creatives types to scale of creatives experiments, etc. as well as channel mixes, and so on.

5. Differences between GEOs and country strategies.

6. Differences between audience demographics.

7. Differences between targeted audience profiles — from archetypes to motivations to economics behavior, and so on.

8. Seasonal and holiday season differences.

9. Etc.

The need to understand that each data source uses different means for getting KPIs they show:

1. Everything you see can be wrong, right, or close.

2. Instead of one source, sometimes it's better to combine more than one source together.

3. Understanding you need to take things with proper judgement and grain of salt.

When looking more into products, understanding audience and their archetypes, motivations and traits is becoming popular in the mix:

1. Understanding your target audience against benchmarked products' target audience can make a difference.

   1. Differences between targeted audience profiles — from archetypes to motivations to economics behavior, and so on.

   2. Differences between audience demographics.

   3. Differences between GEOs and country strategies.

2. Understanding how you target your audience for UA and communicate towards them is crucial in planning your scaling and GTM strategies.

   1. You can e.g., score your Ads / Creatives with different key factors such as demographics score, motivations score, product USP score, value prop. score, competitiveness score, and so on — and formalize a comprehensive priority list for your Ads / Creatives from that.

3. Understanding how you tailor your product for your audience will get your needles moving, where understanding effects of e.g., USPs and value props. will play big part in estimating how your actuals (actual KPIs you’ll get throughout different qualitative and/or quantitative tests) would differ from benchmarks.

   1. USPs and value props. can be used in point 2 above, which can help with reducing your CPIs and/or improving your audience targeting as well as engagement (this is also a topic that I might potentially write about — let’s see if / when that happens).

Connection to Business Modeling

Overall, on top of benchmarking (at early stage), Business Modeling plays big part in checking the healthiness of the product, which is another topic (you can find it from above and here: https://gamesalchemy.substack.com/p/6-why-business-modeling-your-f2p) of its own; which overall leans to current top of the funnel approach for UA and growth — in terms of e.g., making sure your ROAS profile is moving to right direction.

Business Modeling combined with Benchmarking can also be used for competitive intelligence, e.g., breaking down competitor games and seeing how they engage, monetize and scale / maintain their operations. Percentiles of a category can be also modeled for different purposes

Additionally, the further one goes with game development, that matures the game from early stages to further stages, the more importance Business Modeling will has over benchmarking — but it shouldn’t be forgotten, as there are use cases for it during later stages as well; just in a different format and for some specific purposes.

Extra Read

Furthermore, beyond above points, Benchmarking has a connection (direct / indirect, as the case applies) to Iterative Development, which I’ve covered more here (https://gamesalchemy.substack.com/p/2-get-iterative-game-development):

2. Get Your Iterative Game Development Right

Want to delight your execs and stakeholders in connection with Benchmarking? Read more about expectations management related to this side of things here (https://gamesalchemy.substack.com/p/3-stakeholder-and-executive-expectations):

3. Stakeholder and Executive Expectations Management in Iterative Development

Happy Benchmarking!