QA Testing Your Experiments Before Putting Them Live

Introduction

The success of your experiments is firstly influenced by whether they are set up properly or not. Therefore, a rigorous Quality Assurance (QA) process is needed before launching any new experiments.

There are several things that you can do to make sure your experiment will display and function properly that are described below.

Staging Environment

If you want to test in separate environments, you can add the tracking code to both the staging server and live environments. Then initially set your experiment site area to the staging server, then when done change it to the live environment, although you don't really need to test this way if you are using the process described below. You can QA your experiments in your live environment without affecting your visitors.

Live Preview

Do a live preview of your variations, which can also be done when your experiment has Draft status.

Make sure your pages render as expected in the most popular browsers for your business. Also make sure that any button, link, or form still works properly. Certain options in the editor could break forms, links and buttons if they are not used properly; see http://support.convert.com/hc/en-us/articles/205160385-Why-Are-My-Pages-Showing-Content-from-the-Page-Where-I-Set-Up-the-Experiment-

To find out how to preview a variation, read the following article: https://convert.zendesk.com/hc/en-us/articles/204506649-How-Do-I-Preview-Variations-

Specific Audience

Place your experiment in "Active" status for a very specific audience that only you can meet the conditions for, and therefore it will run just for you.

To do that, you can create a new Audience rule similar to the following: 
Medium matches exactly qa_test 

You can replace "qa_test" with whatever you like; just replace it with your preferred text in the Audience rule.

 

Now to test this, open a fresh Incognito window in your browser.  Do not open a second tab and make sure you have no other Incognito windows open.  Make sure that you close the window when you are done with the current test and open a new window for each new test. *This is essential.

Before visiting the site, add the following to the end of your URL:
?utm_medium=qa_test (replace qa_test with the value you chose when you created the Audience).

Most of the time you would visit the URL that matches the Site Area conditions.  Depending on how your experiment is structured, in some cases you may start out by visiting a different page and then navigate to the Site Area that triggers the experiment to run.  In either case, make sure you add the query parameter to the very first URL you visit.  This parameter will be remembered even if you navigate to different pages.*

For example, if the URL is http://www.mysite.com, you should visit the URL:
http://www.mysite.com?utm_medium=qa_test

If you follow the above instructions, you will be shown any variation in the experiment, as this is selected randomly.

After you have done the above, and if you didn't already start with the URL that matches the Site Area conditions, you can then visit the URL that matches the Site Area. You should then be viewing either the Original or one of the Variations, depending on which is randomly selected by the experiment.

 Important

Force a specific variation

You can force a specific variation, by using certain query parameters. 

Example: http://www.mysite.com/test-page.html?_conv_eforce=123.678 where 123 is the experiment ID and 678 is the variation ID;

You can find a more detailed explanation here.

 Important

Tip

Combining Parameters

You can also combine query parameters.  For instance, if you want to do a qa test but also want to force a specific variation, you would append the query parameter to the end of the previous parameter with an ampersand (&) between them.  Only the first parameter would have the question mark at the beginning, ie the below example shows both the force variation parameter and qa_test Audience condition paramter combined:

http://www.mysite.com/test-page.html?_conv_eforce=123.678&utm_medium=qa_test

Tip
Tip
Tip

If you are experiencing problems, for instance issues with click goals, please refer to this article for debugging

Have more questions? Submit a request

Comments