Configuration
      Back to home
      1. Help Center
      2. Configuration

      How do Split URL Experiments work?

      Understanding the Mechanics of Split URL Experiments on Convert.com

      In the world of website optimization and A/B testing, Split URL experiments offer a powerful method for testing entirely different versions of web pages against each other. On Convert.com, understanding the precise mechanics of how these experiments function is crucial for setting up effective tests and interpreting results accurately. This article provides a detailed exploration of the internal workings of Split URL experiments on Convert.com, focusing on the step-by-step processes that occur from the moment a visitor lands on a page to the tracking of their interactions.

      Introduction to Split URL Experiments

      Split URL experiments, also known as redirect tests, involve comparing different web pages hosted on separate URLs. This method is ideal for testing:

      • Significant Design Changes: When variations differ so substantially that they require separate pages.
      • Different User Flows: To test alternative navigation paths or user journeys.
      • New Technologies or Platforms: When variations are built using different frameworks or content management systems.

      By redirecting visitors between the original (control) and variation pages, businesses can measure which version performs better based on predefined goals.

      The Mechanics of Split URL Experiments on Convert.com

      1. Initial Page Load and Experiment Triggering

      When a visitor lands on a page where a Split URL experiment is running, the following occurs:

      a. Location and Audience Conditions Evaluation

      • Location Conditions: The Convert.com tracking code checks if the current page URL matches the URL targeting conditions defined for the experiment. These conditions can include specific URLs or patterns matching multiple URLs.
      • Audience Conditions: The script evaluates whether the visitor meets the specified audience criteria, such as device type, geolocation, referral source, or custom attributes.

      Mechanics Insight: The evaluation of these conditions happens during the initial page load through JavaScript code embedded in the tracking script. If both location and audience conditions are satisfied, the visitor becomes eligible for inclusion in the experiment.

      2. Variant Selection

      Once the visitor is deemed eligible:

      a. Random Assignment to Control or Variation

      • The visitor is randomly assigned to either the control or one of the variations based on the predefined traffic allocation percentages.
      • Randomization Process: The assignment uses a pseudo-random number generator to ensure an unbiased and random distribution of visitors across variants.

      Mechanics Insight: Random assignment is critical for ensuring the statistical validity of the experiment. It helps in minimizing biases and achieving a representative sample of visitors for each variant.

      3. Handling of Control Group

      a. Redirection to the Same URL

      • If the visitor is assigned to the control group, they remain on the current page.
      • In technical terms, they are redirected to the same URL they are currently on.

      Mechanics Insight: Redirecting control group visitors to the same URL ensures that the experiment logic treats both control and variation groups similarly in terms of redirection mechanics. This approach maintains consistency in the tracking and data collection process.

      4. Handling of Variation Group

      a. Variation Without Regex Enabled

      • If the visitor is assigned to a variation and the Regex Option is disabled:
        • The visitor is redirected to the Static Variation URL specified in the experiment settings.
        • This URL is predefined and does not change based on the visitor's original URL.

      Mechanics Insight: This straightforward redirection is suitable when the variation page does not need to adapt based on dynamic URL parameters from the original page.

      b. Variation With Regex Enabled

      • If the visitor is assigned to a variation and the Regex Option is enabled:
        • The system performs a Regex Match between the visitor's current URL and a defined Original URL Pattern.
        • This pattern typically includes capturing groups to extract parts of the URL (e.g., query parameters path segments).

      i. Regex Matching Process

      • Matching Groups: The regex pattern identifies specific parts of the original URL and stores them in variables.
      • Match Success:
        • If the regex match is successful, the captured groups are used to construct the Dynamic Variation URL.
        • Substitution: The captured groups are inserted into placeholders within the variation URL template.
      • Match Failure:
        • If the regex match fails, the visitor is not redirected to the variation.
        • Impact on Experiment: This can lead to uneven distribution of visitors if not correctly handled.

      Mechanics Insight: Enabling regex allows for dynamic redirection, preserving parts of the original URL in the variation URL. This is essential when variations need to maintain query parameters or specific path segments for functionality or tracking purposes.

      5. Redirection to the Final URL

      a. Execution of Redirection

      • After determining the final URL (either the same URL for control or the constructed variation URL for the variation group), the visitor is redirected using a client-side redirection method (e.g., window.location.replace() in JavaScript).

      Mechanics Insight: Client-side redirection occurs quickly after the page load begins but can be affected by network latency or script execution delays.

      6. Post-Redirection Tracking and Experiment Counting

      a. Convert Tracking Code Execution

      • Upon arriving at the final URL (control or variation page), the Convert.com tracking code executes again.
      • The tracking code recognizes the visitor's experiment assignment (control or variation) based on cookies or other persistent identifiers.

      b. Experiment Impression Counting

      • Experiment Entry: Only after the visitor lands on the redirected page and the tracking code runs is the visitor counted as part of the experiment.
      • Data Recording: The visitor's interactions, goals achieved, and other metrics are recorded and associated with their assigned variant.

      Mechanics Insight: Delaying the experiment counting until after the redirection ensures that only visitors who successfully reach the intended page are included in the experiment data. This approach helps maintain data accuracy by excluding visitors who may drop off during the redirection process.

      Practical Implications and Considerations

      Handling Uneven Experiments Due to Regex Match Failure

      • Issue: If the regex match fails for certain URLs or visitor scenarios, those visitors will not be redirected to the variation.
      • Result: This can lead to an uneven distribution of visitors between control and variation, potentially biasing the experiment results.
      • Solution: Thoroughly test regex patterns to ensure they match all intended URLs and consider providing fallback mechanisms.

      Mechanics Insight: Proper regex pattern construction is crucial. Tools for testing regex patterns with sample URLs can help identify and correct issues before deploying the experiment.

      Importance of Tracking Code on All Pages

      • Requirement: The Convert.com tracking code must be installed on all pages involved in the experiment, including control and variation pages.
      • Reason: Without the tracking code, visitor interactions and conversions cannot be recorded, rendering experiment data incomplete or invalid.

      Mechanics Insight: The tracking code facilitates communication between the visitor's browser and Convert.com's servers, enabling data collection and variant consistency across sessions.

      Session Consistency and Returning Visitors

      • Visitor Consistency: Convert.com ensures that returning visitors see the same variant to which they were originally assigned.
      • Mechanism: This is achieved through cookies or local storage, storing the visitor's variant assignment.

      Mechanics Insight: Maintaining session consistency is vital for the integrity of the experiment, as switching variants for returning visitors can introduce confounding variables.

      Example Workflow of a Split URL Experiment

      Step-by-Step Scenario

      1. Visitor Arrives: A visitor lands on www.example.com/page.
      2. Condition Evaluation:
        • The tracking code checks if www.example.com/page matches the experiment's location conditions (e.g., URL contains /page).
        • Audience conditions are evaluated (e.g., device type is mobile).
      3. Eligibility Confirmation: The visitor meets all conditions and is eligible for the experiment.
      4. Variant Assignment:
        • The visitor is randomly assigned to the variation group based on traffic allocation (e.g., 50% chance).
      5. Variation Handling:
        • Regex Enabled:
          • The Original URL Pattern is www\.example\.com/page\?id=(\d+).
          • The visitor's URL is www.example.com/page?id=123.
          • Regex Match:
          • Captured Group 1: 123.
          • Variation URL Template: www.example.com/newpage?id=$1.
          • Final Variation URL: www.example.com/newpage?id=123.
      6. Redirection:
        • The visitor is redirected to www.example.com/newpage?id=123.
      7. Post-Redirection:
        • The tracking code runs on www.example.com/newpage?id=123.
        • The visitor's actions are tracked and associated with the variation group.
      8. Data Recording:
        • Any goals achieved (e.g., purchase, sign-up) are recorded under the variation variant.

      Possible Outcomes

      • Successful Redirection and Tracking: Data collected contributes to the experiment's results.
      • Regex Match Failure:
        • If the regex does not match (e.g., missing id parameter), the visitor is not redirected.
        • The visitor may remain in the control group unintentionally, affecting data distribution.

      Best Practices for Split URL Experiments on Convert.com

      Carefully Define URL Patterns and Regex Expressions

      • Test Regular Expressions: Use regex testers with sample URLs to ensure patterns match as intended. There are regex checkers included in the Location conditions as well as in the Variation editor modal. 
      • Include All Variations: Consider all possible URL variations that visitors might use.

      Ensure the Tracking Code is Present on All Relevant Pages

      • Consistent Data Collection: Verify the tracking code loads correctly on both control and variation URLs.
      • Avoid Data Loss: Missing the tracking code can lead to incomplete data and skewed results.

      Monitor Experiment Performance Regularly

      • Check Distribution: Ensure that visitor allocation between control and variation matches expected percentages.
      • Analyze Early: Identify any anomalies or issues early in the experiment to make necessary adjustments.

      Plan for Edge Cases

      • Fallbacks: Decide how to handle visitors for whom regex matches fail or who encounter errors during redirection.
      • Error Handling: Implement error handling in the tracking code or experiment setup to manage unexpected scenarios gracefully.

      Conclusion

      Understanding the detailed mechanics of Split URL experiments on Convert.com enables you to design and execute tests that yield reliable and actionable insights. By comprehending how location and audience conditions trigger experiments, how visitors are assigned to variants, and how redirection and tracking function, you can optimize your experiments for accuracy and effectiveness.

      By paying close attention to the construction of regex patterns, the placement of tracking codes, and the management of visitor assignments, you minimize the risk of uneven experiments and data inaccuracies. This depth of understanding empowers you to leverage Split URL experiments fully, allowing for significant website changes to be tested and validated through data-driven decision-making.

      With this knowledge, you are better equipped to implement Split URL experiments that not only test your hypotheses effectively but also contribute to an improved user experience and enhanced website performance.