Statistical Confidence
      Back to home
      1. Help Center
      2. Configuration
      3. Statistical Confidence

      Statistical Confidence for RPV

      THIS ARTICLE WILL HELP YOU:

      Introduction

      Revenue per Visitor (RPV) measures the amount of revenue generated each time a user visits your site: RPV = Total Revenue/Total Users.

      Since RPV is a non-binomial metric, it does not follow a normal distribution, because the majority of visitors to your site will not convert or make a purchase. As a result, you will discover that RPV’s distribution contains a greater concentration of $0 values and there is no limit on how much a visitor can spend, which may result in your RPV data containing some extreme values.

      Therefore, RPV’s distribution tends to be right-skewed, making the standard T-test less reliable for measuring its statistical significance. Below we describe how we calculate it.

      Binomial vs Non-Binomial Metrics

      A binomial metric is one that can only have two possible values: true or false, yes or no, present or absent, action or no action. In the context of A/B testing these are all metrics that can are expressed as “rates“: goal conversion rate, e-commerce transaction rate, bounce rate, and, rarely, exit rate. 

      A non-binomial metric is a metric in which the possible range of values is not limited to two possible states. These metrics are usually continuous, spanning from zero to plus infinity, or from minus infinity to plus infinity. These are usually "per visitor", "per session" or "per order" metrics, such as: average revenue per visitor, average order value (AOV), average session duration, average pages per session, average sessions per user, average products per purchase, etc..

      So, in short, the major difference comes from how the possible values are distributed. In the binomial case they are strictly enumerated, while in the non-binomial case they can cover all real numbers. This is the main reason why can’t we use the same statistical calculators for binomial and non-binomial metrics.

      Calculating RPV Significance 

      First, we calculate the standard deviation and then we apply Z-Statistics to calculate the probability.