Skip to main content

AB Testing

A/B Testing

The A/B Testing section allows you to manage and monitor all your experiments in one place. From here, you can review existing tests, track their progress, and quickly create new ones.

Overview​

On this page, you will find a table listing all your A/B tests. Each row represents a test and provides the following information:

  • Name – The title of the A/B test.
  • Status – Indicates whether the test is Pending (set up but not yet running) or Running.
  • Application – The app where the experiment is being executed.
  • Platform – The operating system of the selected application (iOS or Android).
  • ARPAU – Displays the performance of variation B compared to the baseline (A).
  • End Date – Shows the scheduled completion date of the test.
  • Actions – Use the trash icon to remove a test.

Imagen de aplicación

At the top of the screen, you can:

  • Filter tests by Application, Status, or Platform.
  • Search directly by test name.

Create A/B Test​

This screen allows you to configure a new A/B test for your selected application. Fill out the details carefully to ensure your experiment runs as expected.

Details​

  • Name
    Enter a clear and descriptive name for your A/B test. This will help you and your team easily identify it in the list of experiments.

  • Description (optional)
    Add a short description (up to 300 characters) to provide context or specify the goal of the test.

  • Application
    Select the app where the experiment will be executed.

  • Placements
    Choose the ad placements that will be included in the test.

    Reminder: To A/B test Smart Buffering, you must select all placements of the same ad type.

Variations​

Define how traffic will be split between the two groups:

  • Variation A – Represents the baseline setup.
  • Variation B – Represents the test setup.

You can adjust the distribution slider to control the percentage of users in each group.

  • By default, traffic is split 50/50.
  • If needed, uncheck Equals parts to manually set a custom distribution.

Actions​

  • Create Test – Finalize the setup and start the experiment.
  • Cancel – Discard changes and return to the A/B Testing list.

Imagen de aplicación

A/B Test Configuration​

Once you press Create Test, the A/B test will be created with the details you entered and will appear in Pending status.
In this stage, you can review and configure the test setup before starting it.

General Information​

At the top of the screen, you will see the test name, description, selected application, and the traffic distribution between groups A and B.

Test Groups​

Control Group A​

  • This group is read-only.
  • You cannot edit its configuration, but you can see all the existing settings as they are currently defined in the application. Imagen de aplicación Imagen de aplicación Imagen de aplicación Imagen de aplicación

Variant B​

  • This group is fully editable, allowing you to test new configurations.
  • You can make changes at two levels:
1. Placement Level​
  • Traffic Routing: Add, remove, or reorder traffic routing rules.
  • Mediation Links: Add or modify mediation links and their order.
2. Application Level​
  • Enable or disable Smart Buffering.

    Reminder: To A/B test Smart Buffering, you must select all placements of the same ad type.

Imagen de aplicaciónImagen de aplicación

Actions​

  • Start Test – Once the Variant B configuration is ready, press this button to launch the test.

    • The A/B test will change from Pending to Running.

  • Cancel – Discard changes and return to the A/B test list without starting the experiment.

Running Test​

Once the A/B test is in Running status, you will be able to see the performance metrics directly from the A/B Test list screen.
Each row displays high-level data for Groups B and/or C.

Imagen de aplicación

Detailed View​

This section contains three tabs:

1. Overview​

  • Provides a direct comparison between groups and their key metrics:
    • ARPAU (Average Revenue per Active User)
    • Impressions per Active User (ImpAU)
    • eCPM (Effective Cost per Mille)
  • Includes a comparative chart to visualize performance differences.

2. Analytics​

  • Offers more advanced analytics with additional filters.
  • This is the best place to monitor experiment performance in depth.

3. Groups​

  • Displays the Control Group and all Variant Groups (B, C, etc.).
  • Variables in the variant groups are editable, but keep in mind that changes may affect the validity of your results.

Finishing Test​

You can end the A/B test at any time by pressing the Finish A/B Test button.
When doing so, you must choose which configuration you want to keep (A, B, or C).

  • Once finalized, the test status will change to Finished.
  • You will be able to review the final metrics for all groups.

Imagen de aplicación