How to Validate Convert AB Testing Data Against Shopify Order Data Using Metorik

 

IN THIS ARTICLE YOU WILL:

• Validate Convert A/B testing data with Shopify orders
• Install the Metorik app
• Create customer segments in Metorik
• Segment the Raw Data in Shopify
• Handle cases without the Shopify App
  

Objective

This article explains how to validate your Convert AB testing data against Shopify order data using the Metorik reporting app. By following these steps, you can ensure accurate segmentation and reporting of your A/B test results.

Step 1: Add Metorik to Your Shopify Store

You can do the same with any reporting Shopify app, as long as it provides attribute filtering. Here we use Metorik as an easy example, but there is no advantage over others if they provide order attribute filtering.

Before using Metorik to segment and analyze your data, you need to install the Metorik app on your Shopify store. Here’s how:

1. Log in to your Shopify Admin dashboard.

2. Go to the Shopify App Store and search for "Metorik".

3. Install the Metorik app and follow the setup instructions to connect it to your store.

4. Once installed, Metorik will begin syncing your store’s data, including orders, customers, and carts.

 

Step 2: Use Metorik to Segment Orders Based on Convert Data

Once the test period is complete, you can use Metorik’s segmenting system to filter orders based on the Convert A/B testing data. Here’s how:

Create a Customer Segment in Metorik:

• Go to the Customers page in Metorik.

• Create a segment based on the exps order attribute, which contains the Convert experiment IDs.

• Save this segment to your Saved Segments list.

Match Experiment IDs in the Revenue Report: 

• Navigate to the Revenue Report at Metorik Revenue Report.

• Use the Experiment_experimentid attribute to filter and match orders with specific Convert experiment IDs.

Analyze the Data:

Metorik will now display orders filtered by the Convert A/B testing data, allowing you to validate the performance of your experiments against actual Shopify order data.
These attributes are added by the Shopify App.

Segmenting the Raw Data in Shopify

For users who prefer to bypass third-party apps and use raw data for validation, or if your segmentation needs require more control than an app's interface can provide, we offer a technical guide on directly querying the data. This alternative method is ideal if you are exporting data to a data warehouse (like BigQuery or Snowflake) or using the native ShopifyQL editor. You can learn how to structure your SQL queries to isolate orders by Experiment ID and Variation ID based on the two common data storage formats here: Validating A/B Test Data Against Raw Shopify Orders using SQL

When Not Using the Shopify App

Contact Convert support to help you adapt your Global JS code to one that would do the job.

Additional Resources

For more details on Metorik’s segmentation features, refer to the official documentation:

• Segmenting Orders, Subscriptions, and Carts Based on Customer Filters

Support

If you encounter any issues or have questions, feel free to contact our support team for assistance.