7 Ways to A/B Test Your App Store Pages

If someone tells you that you can increase your app conversion rate by 66% by simply changing a graphic element on the App Store product details page, you’ll probably do it without hesitation!

Unfortunately, such an expert does not exist, but this example is just to illustrate: A/B testing is crucial to the performance of the application in the market! For every 1% increase in conversion rate, your revenue will increase by 1%. This is the reality!

At this point, your question should be: "We've been talking about A/B testing for years, but is there a specific testing process?"

In today’s article, we will introduce you to the A/B testing method in detail. After reading it, you will have no excuse not to do A/B testing! Are you ready? drive!

Google Play has its own A/B testing function. If your product is released on this platform, it will be very convenient to do A/B testing. But if your app is released on the iOS platform, it will be more troublesome.

What content needs to be tested?

The answer to this question depends on different situations. In summary: when observing the product details page of the App Store, the content that is most easily overlooked is the content that needs to be tested the most. If your icon is designed to be exquisite and unique, and the screenshots look dull and boring in comparison, you can start with the screenshots, then change to some screenshots with equally eye-catching effects to see which one works better.

According to StoreMaven’s data, videos, screenshots, and icons are the factors that most influence conversion rates. StoreMaven did an impact analysis of different factors, see the figure below:

Where can I find my conversion rate?

If your app is published on Google Play, you can view conversion data for organic downloads under the Acquisition Reports tab. Google Play can display benchmark data, allowing you to see how your app performs compared to similar apps.

As you can see, if your app’s conversions are below the average for your category, Google will tell you how much downloads you need to achieve.

To view your conversion rate for the iOS store, open the Analytics page and tap Conversion Rate. When this option is selected, the graph will show the conversion rate for each day.

If you want to see only data for organic users in the iOS dashboard, set a filter for “App Store Browse” or “App Store Search” as follows:

1. Visual observation

This method can barely be called A/B testing, but it can still help you understand which creatives perform better. If you’re expecting to see a significant increase in conversions, I recommend not relying on this approach.

advantage

• free
• Getting started is simple, just upload new creatives, wait a few weeks, and compare conversion rates before and after

shortcoming

• The effect is not accurate and the rebound is serious
• Even without testing, conversion rates can fluctuate by as much as 50% per month, misleading test assumptions.

No pre-launch testing

You must have a product released before you can use this testing method. More suitable for already stable products.

Long testing cycle

To see noticeable results, you must allow enough time to collect data, say two or three weeks at the least, or a month or two at the most.

In which situations is this approach suitable?

• Your app is only released on iOS platform
• No budget for paid advertising
• There is no login page for the website to run the test
• You are 100% sure that the new change will increase conversions

2. A/B Testing on Google Play

If your app is published on Google Play, you can use this method.

In Google Backend, you can run A/B tests on icons, descriptions, feature graphics, screenshots, promotional videos, etc.

Setting up Google Test is easy. Open the Google Play Dashboard and click here:

There are two types of tests:

Global scope : Global testing will display your test materials to all users around the world. It is more suitable for elements that do not involve text (such as icons) and will not cause problems in understanding due to language barriers.

Local tests : Show localized tests only to users in the target language. It is suitable for testing elements including text (such as titles ), and the test population will be more accurate.

Once you have set up your test and run it, it should look like this:

Once the test begins, all you can do is wait. When the test reaches 90% confidence, it ends and your data is usable. Here is Google’s documentation on how to run A/B testing in Store Listing for your reference:

https://support.google.com/googleplay/android-developer/answer/6227309?hl=en

advantage

• free
• The data is accurate. The reliability of official data from Google is still very high. Of course, statistical significance and "90% confidence intervals" allow for some error.
• Easy to operate. Just submit the content you need to test.

shortcoming

• Only available on Google Play. Hopefully Apple will release its own A/B testing tool , but for now it’s just a dream
• If you have apps in both Google Play and iOS stores, you can test using the Google Play Store Store Listing and then apply the best results to the iOS store.

No pre-launch testing

You need to publish the app first, then test it.

Which situation is it suitable for?

Your app is published on Google Play

3. Third-party testing software

If you have the budget, we recommend using a third-party testing service like SplitMetrics, Storemaven, or Test Nest to simulate your app store pages for A/B testing. According to our experience, the analysis reports they provide are relatively professional.

You just need to create a test campaign, set the variables, and observe the effects.

For example, you could create two mock app pages, each with a different icon. See which ones convert best through Facebook ads or any other traffic source. After the user clicks "Install" on the simulated application page, he will be directed to the product's login page. If the user is interested, he will register here and leave his account information.

Below is an experiment conducted by Illusion Labs where they tested the performance of 3 different icons.

As you can see, the first version of the icon converted the best, and that’s the one they ended up going with.

advantage

• Can do pre-release testing. You can test some elements before the app is officially released
• Allows testing of more variables at once
• Supports attribution links like AppsFlyer to track user behavior . For example, the user's purchase behavior after downloading, etc.

For example, maybe variant A converts 10% more than variant B, but AppsFlyer data shows that users who saw variant B spent 20% more than variant A. Which variable would you choose? Of course it is variable B!

shortcoming

• It’s not cheap, the software you need to run your tests starts at $149 per month (TestNest)
• There will be user loss during the testing process. When users clicked “Install” on the simulated app page, they were not directly redirected to the real app download interface, but required two or more clicks and jumps. In this process, some users will be lost.

In what situations is this approach appropriate?

• Many applications for iOS platform
• Pre-release A/B testing
• Have a testing budget and run paid advertising campaigns at the same time

4. Paid creative testing

Paid advertising refers to promotion services provided by Facebook ads or other advertising alliances . If you have a need to run ads, then you have a need to A/B test your ad creatives.

The operating mode is to run multiple sets of creatives at the same time and test your creatives directly using Facebook ads. It looks like this:

Ad networks other than Facebook can also test icons, descriptions or screenshots in their ads.

For example, if using Chartboost, you could test these:

Here are some additional links from Chartboost: Boost your mobile ad conversions by up to 15% with social metadata.

advantage

• Cheaper than third-party testing software
• Support pre-release testing
• Multiple variables can be tested at once.

shortcoming

• The results will be biased. While this method can help you understand the best conversion method, the data is still not as accurate as professional testing software. This is because the display effect of the ad is different from that of the app store page, and the content seen by users is also different

In which situations is this approach suitable?

• Many applications for iOS platform
• Apps that require pre-release testing
• Apps with advertising budget

5. A/B Testing on Your Landing Page

If your app has a separate login page, this can also be used to test different graphical elements.

You can use services like Optimizely or Google Optimize to test icons, screenshots, descriptions, titles, and even prices directly on your landing page. For example, the following figure:

advantage

• Support pre-release testing
• If you already have a website up and running, all you need to do is install an A/B testing tool and run your tests.

shortcoming

• Since the context on the website is different from the actual app store page, the results may be skewed
• Expensive and time consuming. At least there must be a usable landing page, and the website must have a lot of traffic.

In what situations is this approach applicable?

• Many applications for iOS platform
• Pre-release A/B testing is required
• Have a landing page available and enough traffic to run an A/B test

6. Application wall test

If you have multiple apps under your brand, you can set up an App Wall in another of your apps to cross-promote your different apps. By the way, you can also do some testing here.

advantage

• free. Once you have your app wall set up, all current users will be able to do A/B testing for you for free!

shortcoming

• Since the environment is different from the actual app store page, the results may be skewed
• Depending on the number of users you have, the testing cycle will be extended.

In what situations is this approach applicable?

You need at least one similar app with a decent number of users

7. Social Media Poll Test

You can run a test poll in places like Facebook groups, subreddits, or even show the variables to friends and family. For example:

Here are some platforms where you can start a poll:

• Facebook – App Entrepreneurs and Marketing Group
• Reddit – Feedback Request Example
• Pickfu (paid) - provides you with comparison results of variables. Get 50 votes for just $20

advantage

• Cheap and easy to operate. Simply send your poll to a group of people. Even if you pay to vote, you will get a very cost-effective voting result
• Get feedback from voters

shortcoming

• The accuracy is not high. Often, they may choose the most beautiful or eye-catching icon, but that may not necessarily be the one with the highest conversion rate.

In which situations is this approach suitable?

• Many applications for iOS platform
• Need to collect user feedback

in conclusion

Now, no matter what platform your product is released on, you have no reason not to do A/B testing.

One final tip is that you might want to start testing with qualitative methods (like polling) to make sure you’re on the right track.

If your app is not yet released, you are wondering which version to use for icons and screenshots. Let's do a test to see what everyone's choices are.