What is creative testing? A guide for mobile game creatives

Creative testing is the practice of running structured experiments on ad assets, often comparing variants of an asset against a control to learn which elements of it drive performance. Creative teams typically run tests before committing a budget to a creative or set of creatives. In mobile game user acquisition (UA), creatives often decide how a campaign performs, so creative testing is one of the most impactful levers a performance marketing team has. Creative testing helps to answer which creatives work, for which players, and on which channel.

This guide covers what to test, which method to use, and how to read the metrics games scale on.

What is creative testing (and what is an ad creative)?

Creative testing is the process of testing performance by isolating and comparing the elements of an ad creative to inform which to scale and which to retire. A creative is the asset itself: the video, playable, static image, or interactive end card a player sees and responds to.

Producing strong variants and testing them are separate jobs. (Check out our guide to best practices for ad creatives for the production side of things.) Testing tells you which of those variants your team produced have earned their place in your campaigns.

When you test matters too. Test before launch to check finished assets against your benchmarks, during a campaign to find and scale creative winners, and after to learn why a creative failed and what to include in your next campaign. Mobile game UA teams run all three continuously, as a live campaign always needs new variants.

Why creative testing matters for mobile game user acquisition

For mobile games, creatives are the dominant performance lever in UA. But performance decays over time for any creative. No matter how effective a creative is, if a user sees it enough, it will become stale. This is referred to as creative fatigue.

So for all creative teams, creative sets need to be consistently refreshed. Creative testing is how you know which creative elements to continue using, which need to be updated, and which should be taken out of rotation.

What should you test? Elements and formats

A best practice is to test one layer at a time: the elements inside a creative, and the format the creative takes. Elements are variables that make up a creative asset, such as the hook, visuals, copy, call to action, layout, sound, and characters. Format is the structural choice (video, playable, or video-plus-playable). Often, changing format moves performance more than any single element.

Elements worth isolating:

• Hook (first 1 to 3 seconds): the largest single driver of whether users see the rest.
• Gameplay depiction: real gameplay versus stylized or exaggerated scenarios.
• Copy and on-screen text: value proposition, difficulty framing, and urgency.
• Call to action: how and when you phrase what you want the user to do.
• Layout and orientation: portrait versus landscape, UI density, and end-card placement.
• UGC treatment: creator-style delivery versus produced footage.

Formats worth testing side by side:

• Video: the default across social and game networks, and the fastest to iterate on.
• Playable ads: a short interactive slice of the game. They carry the strongest intent signal, because players self-select by choosing to play.
• Video-plus-playable: a mix of both formats, starting with a short video as the hook that leads users to the playable.

Games teams often under-test interactive formats because they’re harder to produce. Teams can build playables in the Unity Engine or from no-code templates in Unity Playworks and run them on any network, which lowers the production barrier that keeps most teams out of these formats.

A common mistake is running a multivariate test on an A/B-sized budget. If each combination gets too few impressions to reach statistical significance, any winner could be a result of noise.

Metrics: how to read a creative test in games

At the test stage, read installs per mille (IPM) first. IPM is installs per thousand impressions, and it isolates creative performance better than most metrics. It holds impression volume constant. When two creatives run in the same campaign to the same audience, the gap in IPM comes largely from the creative.

Then layer in the rest, in order of how far down the funnel they sit:

• Hook rate or thumbstop rate: the share of impressions that hold attention past the opening seconds.
• Completion rate: the share of video views watched to the end.
• Click-through rate (CTR): clicks divided by impressions.
• Cost per install (CPI): what you paid for each install, and the efficiency read once a creative is live at spend.
• Cost per acquisition (CPA): cost per defined post-install action, such as a tutorial completion or purchase.
• Return on ad spend (ROAS) by day: D7 and D28 ROAS, measured against lifetime value (LTV) expectations.
• Retention: whether the players acquired are still playing.

A creative can win on IPM and lose on ROAS, usually because it attracted players the game didn’t retain. Use the early IPM read to build a creative element shortlist, and let D7 and D28 ROAS plus retention rate settle the final decision.

Two mechanics distort measurement if you ignore them. The first is statistical significance: with small install counts, apparent winners are often random, so hold judgment until each variant has enough signals to trust. The second is the learning phase: platforms reallocate delivery while they calibrate, so a variant judged in its first hours is judged mid-calibration.

A step-by-step creative testing framework

A workable creative testing framework runs in eight steps, from hypothesis to refresh. Each step guards against a specific way tests go wrong: unclear questions, moving goalposts, confounding conditions, and premature calls.

1. Write a specific hypothesis, such as “Leading with boss-fight footage instead of tutorial footage will raise IPM, because it shows the core loop faster.” A specific hypothesis makes an inconclusive result informative.
2. Define the deciding KPI up front. Pick the metric that settles the test before you see any data. That’s usually IPM for early testing and D7 ROAS for scale.
3. Build variants that differ deliberately. For A/B, change one thing. For multivariate, plan the elements so that each test is testing something you’d actually run with.
4. Match the method to your volume. The table above shows what each method needs, and volume determines what you can answer.
5. Set a budget per concept and allocate it evenly. No universal benchmark exists, so derive it: work backward from your CPI and the installs per variant you’d need. At a $2 CPI and a 50-install threshold, that’s roughly $100 per variant. An even budget split helps to make sure delivery doesn’t pick a winner for you.
6. Launch with clean conditions: fresh or comparable audiences, the same geos, the same placements, and the same time window.
7. Read the signal after the learning phase. Give the platform time to calibrate, then check.
8. Scale winners, retire losers, and refresh on a cadence. Move winners to scale budgets while watching whether ROAS holds at higher spend, and queue new variants continuously so production never holds you up.

Watch CPI and IPM as trends rather than absolute numbers. When a top creative’s IPM slides while spend holds steady, fatigue has set in, and that’s your refresh trigger. Teams at high spend generally need new concepts entering rotation weekly or biweekly, with variants of proven winners filling the gaps.

Testing on game networks vs social ad networks

The channel changes both the setup and which numbers you trust. Game ad networks run playables and videos at volume, inside rewarded and interstitial placements in other games, so IPM and install-side metrics are the native read. Social ad networks are often video-first and feed-native, and thumbstop rate, hook rate, and social conventions like UGC do more of the work there.

Intent differs too: someone already playing a mobile game is closer to your product than someone scrolling a feed.

Test across both, and keep separate benchmarks per channel. The same creative will produce different IPM and CPI on Unity Ads and on a social network, and comparing the two directly can lead you to retire a creative that’s performing well in its own context.

Scaling and automating creative testing

Automated and dynamic creative testing lets the system generate or rotate variable combinations. For most game teams, the bottleneck is throughput: the number of tests you can build, launch, and read each week.

Dynamic playables are the clearest example. Instead of shipping five separate playable files, you ship one file with variable elements (art, difficulty, tutorial flow, end-card copy) that A/B test themselves in flight, then serve the winner. That cuts production and trafficking overhead at once.

AI creative testing applies the same logic to generation, and it still needs judgment. Someone has to set the hypothesis, pick the metric that settles the test, and notice when a high-IPM creative is quietly costing you retention. Automation makes a sustainable refresh cadence possible at scale, but it won’t tell you which cadence is right.