Understanding Risk and Expected Uplift in Bayesian Reports

Gain deeper insights into experiment performance with Risk and Expected Uplift metrics in Bayesian reports, helping you make data-driven decisions.

IN THIS ARTICLE YOU WILL:

• Get an overview of the Risk and Expected Uplifts feature
• Know how to access them
• Learn to customize their visibility
• Switch between reports.

Overview

When running A/B tests using Bayesian statistics, understanding the potential risks and expected improvements of each variant is crucial for making informed decisions. Convert offers two key metrics in Bayesian reports:

• Risk: The probability that choosing a specific variant will result in a lower-performing outcome.
• Expected Uplift: The estimated long-term improvement you can expect if the variant is selected.

These metrics provide deeper insights into experiment results, allowing you to confidently determine which variations are likely to perform best in the long run.

Accessing Risk and Expected Uplift in Reports

To view these metrics, follow these steps:

1. Navigate to an active experiment that is using Bayesian statistics.
2. Go to the Report page for the selected experiment.
3. In the results table, find the columns labeled Risk and Expected Uplift.
4. These columns are positioned after "Chance to Win" and before "Status" in the report.
   
   

Customizing Column Visibility

Risk and Expected Uplift are hidden by default to ensure a streamlined reporting experience. Users can choose to enable or disable these columns based on their preferences.

How to Show or Hide Risk and Expected Uplift

1. Click on the table customization options (gear icon) on the report page.
2. In the customization panel, locate the checkboxes for Risk and Expected Uplift.
3. Select the checkboxes to show these columns or uncheck them to hide them.
4. Click Apply to save your changes.
   
   
   

Persistent Visibility Settings

Once enabled or disabled, these settings are saved automatically, meaning your selections remain even after reloading the page.

Switching Between Bayesian and Frequentist Statistics

Since Risk and Expected Uplift are specific to Bayesian statistics, they are not available in Frequentist reports. If you switch an experiment from Bayesian to Frequentist, these columns will automatically be removed from the report.

How to Change Stat Types

1. Navigate to an active experiment that is currently using Bayesian statistics.
2. Click on the Actions button.
3. Select Stats and Settings from the dropdown menu.
4. Choose Frequentist as the stat type and click Save.
5. The Risk and Expected Uplift columns will no longer be visible in the report.
   

Key Takeaways

• Risk helps identify the likelihood of a variant underperforming.
• Expected Uplift estimates the sustainable improvement if a variant is chosen.
• These metrics are available only in Bayesian reports and are hidden by default.
• Users can enable or disable these columns via the customization panel, and their settings will be saved.
• Switching to Frequentist statistics will automatically remove these columns from the report.