Paywall Experiments for Subscription Apps. Quick A/B Testing Framework.

Paywall A/B testing is crucial for unlocking subscription app revenue growth. However, conducting rapid experiments can be challenging, especially when restricted by app release cycles and the workload of the engineering and product teams. In this guide, we discuss the way to avoid these limitations and enable quick and flexible experimentation. 

Usually you need to be able to run experiments around several main paywall elements, including the following:

  • Main UI elements like image or video that resonates with your users
  • Messaging on the premium features that help persuade users to subscribe
  • Other elements like social proof and reviews to overcome users’ hesitations
  • Subscription price, duration, free trial, or a mix of offered subscription options to increase conversion by meeting users payment preferences 

You can expect to experiment with these elements on the ongoing basis. Since you already know what the elements are, there is a way to set up your paywall based on a template to enable quick experimentation without distracting significant resources of your team.

Let’s start with the guide on using the paywall templates for this purpose. And after that, we’ll take a closer look at what tools you can use to launch and accurately measure monetization hypotheses. 

1. Create compelling paywall design template 

Your paywall needs to convey the benefits of the premium plan, be aligned with your users onboarding experience, and persuade users to start a trial or subscription. The main visual element, like an image or video, needs to be highly relevant with the content of the app. The text elements can be as simple as a short description of the premium plan or more complex with the detailed bullet points highlighting premium features.

Below is an example of a paywall demonstrating these points, that is simple yet pretty powerful, and that can be used as a template. This example contains the following key elements: 

  1. Appealing graphic aligned with app’s use case
  2. Catchy title with the value proposition
  3. Social proof to reassure users
  4. Clear pricing and free trial offer
  5. Clear Call-to-Action button 
  6. Option to skip the paywall if a user is not ready to subscribe yet
  7. Terms, Privacy, and Restore option as required by App Store guidelines
Paywall Experiments for Subscription Apps

You can create the template based on this layout. Each of the important elements can be controlled, changed, and tested individually to optimize paywall’s performance in the future.

Paywall Template 1:

2. Consider adding several paywall templates

It’s okay to start with one paywall template as shown above, but as already mentioned, you are likely to need more freedom in experimentation to address all hypotheses your team can come up with. So, you don’t have to limit yourself with one template. You can think through cases where you need to present several payment options to a user, highlight the details of the premium features, or explain how the free trial works. Then you can create several templates with different UI layouts, keeping in mind the main requirements. 

Below we are showing 3 additional popular subscription app paywall layouts. Let’s have a look at them one by one.

Paywall Template 2:

  • Main image 
  • Premium details
  • Two subscriptions/products with different duration/price placed horizontally
  • CTA
Paywall Experiments for Subscription Apps

Paywall Template 3:

  • Main image 
  • Premium details
  • Two subscriptions/products with different duration/price placed vertically
  • CTA
Paywall Experiments for Subscription Apps

Paywall Template 4

  • Detailed step-by-step trial explanation
  • One subscription offer
Paywall Experiments for Subscription Apps

After creating the templates and adding them to your app, it’s time to decide which template to use, what elements to show, and test any configuration for any of the templates.

3. Configure paywall variants for testing

By having several paywall templates as shown above and controlling their elements, you can create a virtually unlimited set of variations to test. Here are just a few of the possible variants for each of the template: 

Paywall Experiments Variables for Subscription Apps

From this point onward, you can launch monetization experiments using any tool that you prefer. In the following sections we provide guidance on configuring experiments with these paywalls using Qonversion.

4. Control paywall elements with JSON payload

Qonversion’s Experiments gives you a simple tool to control the elements on a paywall variations and run experiments quickly. You can set up key-value pairs for every element:

Paywall Experiments Variables for Subscription Apps

Qonversion SDK returns the JSON with key-value pairs when you need to show the paywall in the app. For example, below you can see this paywall configured with:

  • Different main image
  • Different social proof element
  • Title variations
  • Different CTA
  • Monthly subscription with higher/lower price point
  • Trial vs. no trial option

Based on the SDK payload a user in group A can be presented with paywal 1A, and a user in group B can see 1B variant or any other required variant.

Experiment #1. Paywall 1A vs. Paywall 1B.

Paywall Experiments Variables for Subscription Apps

Experiment #2. Paywall 1B vs. Paywall 1C.

Paywall Experiments Variables for Subscription Apps

Experiment #3. Paywall 1A vs. Paywall 1C.

Paywall Experiments Variables for Subscription Apps

5. Launch paywall A/B testing in Qonversion

To launch a test, make sure that you have Qonversion SDK installed in your app. Then navigate to the Experiments page in your Qonversion account and select “Create a new experiment”. 

The first step here is to formulate the hypothesis, write it down for your future reference, and choose the primary metric for optimisation.

Paywall experiments - Qonversion

Some of the metrics you can set as primary experiment goal:

  • User-to-trial conversion
  • User-to-paid conversion
  • New trials
  • Trial cancellation rate
  • Trial-to-paid conversion
  • New subscriptions
  • Subscriptions cancellation rate
  • Sales (Revenue in USD)
  • Proceeds (Revenue in USD after deducting app stores’ commission)
  • Refunds

You will be able to see all of the metrics above in the test results, so don’t stress to much on what metric to pick here. 

Next to configure the JSON payload for each experiment variant, follow this experiments documentation.

6. User segmentation and traffic share allocation 

Qonversion allows you to segment users and allocate traffic share for A/B testing. This enables targeting specific user segments and control the distribution of traffic between different variations of your paywall. 

User segmentation and traffic allocation for experiments

You are almost ready to launch. One more thing before starting the experiment is to check that each variant looks as expected. You can easily do it using variant ID in Qonversion.

7. Measure experiment performance in real-time

You do need full visibility into what happens to users after they start a trial or a subscription to be able to make any conclusions in case of subscription monetization model. Qonversion A/B testing module uses Qonversion’s best-in-class precise subscription analytics. So you have the full details of the performance in each experiment group and can make correct conclusions. 


Let’s quickly recap what we discussed above: 

  1. Paywall experimentation is about testing a set of common UI elements
  2. You can simplify and accelerate the experimentation by having several popular paywall templates implemented and ready to be configured
  3. With Qonversion, you can configure this unlimited set of paywalls with just a few clicks using JSON payload.
  4. Formulate the hypothesis, setup the targeting and traffic share allocation, and you are ready to launch a new experiment.
  5. Qonversion provides detailed subscription analytics to see real-time experiment results and make conclusions to drive your app’s revenue growth!