Paywall A/B Testing: Optimize In-App Subscriptions for Education Apps

Paywall optimization is one of the quickest ways to increase in-app revenue for subscription apps. But how can you create a high-converting mobile paywall? How do you choose the right price for in-app subscriptions? What subscription durations should you offer? How do you test UI elements, describe the value of your premium features, and choose what trial options to offer? 

In this article, we explore the best practices for creating paywalls for apps in the education category. We also provide some benchmarks and examples to help you optimize your subscription paywalls strategy.

What is paywall A/B testing?

A/B testing, also known as split testing, is a method of comparing two versions of an app feature to determine which one performs better. In the context of paywalls, A/B testing can be used to test different paywall designs, content offerings, and pricing plans. The goal of the A/B testing is to collect the data to make data-driven decisions and grow in-app revenue through paywall optimization.

Paywall A/B testing for in-app subscriptions

Paywalls in each app category tend to have some similar features that work best for their specific category. If you check the pricing strategies, trial offers, and subscription durations of your competitors, you’ll notice that the top-ranking apps often follow the same patterns. Copying their paywalls directly may not be the best move, but getting some inspiration and ideas from relevant products is always useful.

If you’re wondering why your in-app subscription isn’t converting well, consider leveraging paywall A/B testing to create an iterative development process. By retesting and reiterating based on the testing results, you could achieve significant positive changes, including improved trial-to-paid conversion rate, increased retention, increased trial start rate, and increased customer LTV (lifetime value).

Optimize In-app Subscriptions Paywalls for Education Apps

Let’s take a look at the paywalls of the top 15 apps in the Education category.

There are several common features you can easily notice in these top 15 education apps paywalls:

• No more than 2 different subscription durations are offered. The most common durations are annual and monthly subscriptions. 
• 7-day free trial 
• Monthly subscription is almost always priced below 10 euro
• Similarities in UI: subscription plans offered in rows.

Duolingo Paywall A/B testing 

Duolingo is constantly experimenting with its paywalls. As they have highlighted, they are obsessed with A/B testing. Here is a small recap of some of their latest paywall AB testing:

• Set of paywalls testing 3 subscription durations (1 month, 6 months, 12 months) offered in one row as vertical widgets with slightly different prices. 
• Set of paywalls with 2-3 subscription options placed horizontally under each other and same prices. 
• Set of paywalls explaining how the free trial works and some adjustments to the UI

We have tracked some changes to the main paywall of Duolingo in the last 12 months. We have observed that they have been offering €9, €10, and €12 for monthly subscriptions, and €79, €83, and €87 for annual options. Currently, they are using €10 for monthly and €87 for annual subscriptions.

As of May 2023, Duolingo’s subscription flow is made of four consecutive paywall screens:

This sequence includes: 

1. Value of the premium subscription
2. Catchy 0.00 euro CTA
3. Comparison between free and “super” options
4. Explanation of a 14-day trial, similar to Blinkist’s trial screen
5. Most popular pricing plans with a link to all pricing options 

Here are some more examples of AB testing from top-ranking apps in the category.

We have discovered previous iterations of paywalls for popular apps. Below, you can see the comparison of previous and current paywalls.

Photomath Paywall Variations and A/B testing

Mathway Paywall Variations and A/B testing

Picture This Plant Identifier Paywall Variations and A/B testing

Quizlet App Paywall Variations and A/B testing

Brainly Paywall Variations and A/B testing 

NatureID Pro Paywall Variations and A/B testing 

Babbel Paywall Variations and A/B testing

Paywall A/B testing hypotheses 

After checking some examples, you can start your own app’s paywall A/B testing. As the first step, it’s important to define the hypothesis and set clear goals. Some examples of a hypothesis could be that changing the paywall design will result in a specific x% growth for conversion rates/ARPU, or that offering a free trial will increase the conversion to paid user rate by y%. The goals should be specific and measurable, such as “increasing the conversion rate by 10%”.

You can target the following metrics in your paywall A/B testing: 

• Total In-App Revenue
• Subscription Start
• User-to-Trial Conversion
• User-to-Paid Conversion
• Trial-to-Paid Conversion
• Trial Cancellation
• Subscription Cancellation 
• Refunds 

Next, segment your audience, identify the variables involved, and create variations for each variable. For example, if you are testing paywall design, create two or more variations of the paywall page. It’s important to test only one variable at a time to isolate the impact of each variable.

Launch Subscription App Paywall A/B testing 

Select the audience you want to target with the A/B test. You might want to expose only some percentage of your users to the test, launch the test in one or several countries, or launch on one of the major app stores – Apple / Google Play. Additionally, it is important to run tests only on relevant groups of users. Be careful not to include those who already have active subscriptions in case you’re testing an offer for new ones. Otherwise, the data becomes noisy and it takes much more time to get accurate results.

After launching the test, monitor the results to ensure that everything is running smoothly and it does not have an unexpected negative impact on your product or revenue metrics. When enough data has been collected, analyze the results to determine statistical significance and identify the winning variation. To be confident in test results, it is necessary to reach statistical significance for your metric. Usually, a 95% confidence interval is used, which means you can be 95% confident that one variation truly performs better than another, and it’s not a matter of coincidence. 

The ideal test length falls anywhere between 1 and 4 weeks but it largely depends on the number of users exposed to the test. A test may not reach statistical significance in case of low traffic or conversion volume.

Real-Time Results for Apps Paywall A/B Testing

Interpreting the results of an A/B test is critical to optimizing your paywall strategy. The key is to determine statistical significance and identify the winning variation. If you use Qonversion A/B testing module, once the winning variation is identified, you can roll it out to all users without releasing a new app version. And start testing new variables to further optimize your app’s performance.

Best Practices for Subscription Apps Paywalls A/B Testing

Here are some best practices for conducting A/B tests for mobile apps paywalls:

• Test only one variable at a time to isolate the impact of each variable.
• Use statistical significance to determine the winning variation. You can use A/B calculators for this, if this data is not available in the tool you use for A/B testing.
• Test continuously to optimize your paywalls strategy over time. Learn from the top apps in your category.
• Avoid introducing bias into the test by selecting a biased sample or measuring the wrong metrics.
• Monitor the test regularly to ensure that everything is running smoothly.

Paywall A/B Testing with Qonversion 

Qonversion provides a powerful A/B testing tool built for subscription apps. This tool allows you to test and optimize paywalls including in-app pricing, subscription duration, paywall UI, and use custom JSON payload. You can implement and launch AB testing quickly with your existing subscription management solution. You don’t need to process payments through Qonversion to be able to run AB testing, but with Qonversion you are getting the benefits of highly accurate subscription analytics that enable accurate AB testing calculations. With Qonversion A/B testing you can track all relevant metrics including conversion rates and total in-app revenue. You can roll out the winning variant to all users instantly without releasing a new app version to the App Store. 

Additionally, we have launched the fastest A/B paywall experimentation framework. Discover how to conduct paywall experiments for subscription apps in the most straightforward way.