Multi-Armed Bandit (MAB) & Auto-Allocation in Convert

MAB (Multi-Armed Bandit) is a traffic allocation method that helps you dynamically assign more traffic to better-performing variations during the experiment, rather than waiting until the end. This allows faster learnings, better use of traffic, and improved conversions during experimentation.

This feature is now available under Auto-Allocation in Convert for both Frequentist and Bayesian statistical models.

What is Auto-Allocation (MAB)?

Auto-allocation uses MAB strategies to shift more traffic to winning variations as the test progresses. This is different from traditional (manual) traffic allocation, where traffic is evenly split regardless of performance.

Once enabled, MAB continuously analyzes results and allocates traffic automatically using one of the selected strategies:

• Thompson Sampling (Default)

• Epsilon-Greedy

• UCB (Upper Confidence Bound)

How to Enable MAB (Auto-Allocation)

Auto-Allocation (MAB) can be enabled from:

• Summary Page

• Stats & Settings Dialog

When enabled:

• You will see a lock icon if you don’t have access (plan or permission).

• You cannot manually edit traffic distribution.

• A Beta tag will appear next to Auto-Allocation to indicate this feature is in beta.
  

Stats & Settings Configuration

When enabling Auto-Allocation, you’ll be able to configure the following under Stats & Settings:

1. MAB Strategy

   • Dropdown with options: Thompson Sampling (default), Epsilon Greedy, UCB

2. Exploration Factor

   • Default: 0.1

   • Determines how often the system explores other variations rather than exploiting the best one

3. Primary Metric

   • New setting available at both Project and Experience level

     • Experience Level - Under Summary tab > Three dots > Stats & Settings
       
     • Project Level - Under Projects > Configuration > Stats Settings
       

   • Metrics available:

     • Conversion Rate (CR) – Default

     • Revenue Per Visitor (RPV)

     • Average Products Per Visitor (APPV)

   • Used to evaluate the performance of the primary goal and MAB allocation

   • Cannot be changed after MAB is enabled on an active experience
     
     

Primary Goal and Goal Optimization

When switching from Manual to Auto-Allocation:

• You’ll be prompted to confirm the Primary Goal which is the only one optimized by MAB

• If multiple goals exist, only the Primary Goal is used for optimization

• You cannot change the Primary Goal once the experience is active with MAB enabled

  • You’ll be advised to clone the experience to modify the Primary Goal
    

When MAB Can Be Used

• Available for A/B and Multivariate tests

• Works for Web and Full Stack experiences

• Requires Sequential Test Type when using Frequentist model

  • If Sequential is not enabled, Auto-Allocation will not be available and a tooltip will explain the restriction

• Not available for:

  • A/A Tests

  • Deploy Tests

Reporting with MAB Enabled

• Report tables will include an additional column for Traffic Allocation

• A graph will be displayed showing Traffic Allocation Over Time to visualize how MAB adapts
  

Access and Lock Permissions

• MAB is only available to users with the MAB feature included in their Convert plan

• If MAB is not available under your plan, a plan lock tooltip will show

• If your account lacks permission to change stats settings, an access lock tooltip will show

• Plan lock takes precedence over access lock
  

Notes and Best Practices

• MAB is not enabled by default on new experiences

• For users without the MAB feature in their plan:

  • MAB-related settings will not be accessible

  • Attempts to enable MAB via summary will be blocked

• You can save Stats Settings while an experience is still in Draft mode

• Ensure “Auto Allocation” is clearly marked and avoids placement conflicts between Lock icon and Beta tag