IN THIS ARTICLE YOU WILL:
🎯 Objective
This article explains how to validate your Convert AB testing data against Shopify order data using the Metorik reporting app. By following these steps, you can ensure accurate segmentation and reporting of your A/B test results.
🛒 Step 1: Add Metorik to Your Shopify Store
Before using Metorik to segment and analyze your data, you need to install the Metorik app on your Shopify store. Here’s how:
-
Log in to your Shopify Admin dashboard.
-
Go to the Shopify App Store and search for "Metorik".
-
Install the Metorik app and follow the setup instructions to connect it to your store.
-
Once installed, Metorik will begin syncing your store’s data, including orders, customers, and carts.
Also, there is a 15-day free trial that you can use for the validation.
🧩 Step 2: Add the Provided Code to Convert's Global JS Section
To track and validate your A/B testing data, you need to insert the provided JavaScript code into your Convert project’s Global JS section. Here’s how:
-
Log in to your Convert account and navigate to your project.
-
Go to Project > Configuration > Global JS.
-
Copy and paste the following code into the Global JS section:
// Enable debug logging
const DEBUG = true;
function debugLog(message, ...args) {
if (DEBUG) {
console.log('[Convert Debug]:', message, ...args);
}
}
window._conv_q = window._conv_q || [];
window._conv_q.push({
what: 'addListener',
params: {
event: 'snippet.experiences_evaluated',
handler: function() {
debugLog('Starting experience evaluation handler');
let session_cookie = convert.getCookie('_conv_s');
if (!session_cookie) {
debugLog('Session cookie not found');
return;
}
debugLog('Found session cookie:', session_cookie);
let session_id = session_cookie.substring(
session_cookie.indexOf('sh:') + 3,
session_cookie.indexOf('*')
);
debugLog('Extracted session ID:', session_id);
let exp_list = [];
let variation_list = [];
function processExperiences(sourceData, allData, isHistorical = false) {
debugLog('Processing experiences', isHistorical ? '(historical)' : '(current)');
const variants = [];
for (let expID in sourceData) {
let type = allData.experiences[expID]?.type;
if (type === "deploy") {
debugLog('Skipping deploy type experiment:', expID);
continue;
}
let experience = sourceData[expID];
let variation = experience.variation || {};
let varID = variation.id || experience.variation_id;
debugLog('Processing experience:', expID, 'variation:', varID);
if (varID && !exp_list.includes(expID)) {
exp_list.push(expID);
variation_list.push(varID);
const variantString = `${expID}:${varID}`;
variants.push(variantString);
debugLog('Added variant:', variantString);
}
}
return variants;
}
if (convert.currentData && convert.currentData.experiences) {
debugLog('Processing current experiences');
processExperiences(convert.currentData.experiences, convert.data);
}
if (convert.historicalData && convert.historicalData.experiences) {
debugLog('Processing historical experiences');
processExperiences(convert.historicalData.experiences, convert.data, true);
}
let convert_attributes = {
vars: variation_list,
exps: exp_list
};
debugLog('Final convert_attributes:', convert_attributes);
debugLog('Updating cart attributes');
fetch('/cart/update.js', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
attributes: {
vars: variation_list,
exps: exp_list
}
})
}).then(response => response.json())
.then(() => {
debugLog('Cart attributes updated successfully');
}).catch(error => {
console.error('Error updating cart attributes:', error);
});
}
}
});
Save the changes.
This code will track the variations and experiments a customer is exposed to during their session and store the data in the Shopify cart attributes.
🧪 Step 3: Run the Test for a Determined Period
After adding the code, you need to run your A/B test for a determined period to ensure that Shopify orders are tagged with the experiment ID attributes (exps). Here’s what to do:
-
Launch your A/B test in Convert.
-
Allow the test to run for a sufficient duration (e.g., a week or longer) to generate a meaningful sample size of orders.
-
During this period, the code will automatically tag Shopify cart attributes with the experiment IDs (
exps) and variation IDs (vars) for each customer.
🗒️ Note: Only orders placed during the test period will be tagged with the exps attribute.
📊 Step 4: Use Metorik to Segment Orders Based on Convert Data
Once the test period is complete, you can use Metorik’s segmenting system to filter orders based on the Convert A/B testing data. Here’s how:
Create a Customer Segment in Metorik
-
Go to the Customers page in Metorik.
-
Create a segment based on the exps order attribute, which contains the Convert experiment IDs.
-
Save this segment to your Saved Segments list.
Match Experiment IDs in the Revenue Report
-
Navigate to the Revenue Report: Metorik Revenue Report.
-
Use the
expsattribute to filter and match orders with specific Convert experiment IDs. You can use the vars attribute to filter for the variation data. Make sure you match them with a query likeexpscontains 123456789, being the number the experiment id or vars contains 12315654 with variation id. You can find this numbers in you Preview links.
Analyze the Data
-
Metorik will now display orders filtered by the Convert A/B testing data, allowing you to validate the performance of your experiments against actual Shopify order data.
📚 Additional Resources
For more details on Metorik’s segmentation features, refer to the official documentation:
Segmenting Orders, Subscriptions, and Carts Based on Customer Filters
🛟 Support
If you encounter any issues or have questions, feel free to contact our support team for assistance.