VWO Testing Issues and How to Troubleshoot Them

VWO is one of the tools we’ve mentioned in our best A/B Testing tools list. That’s because VWO is more than just a basic testing tool, since it comes with so many add-on features: Besides the experiment platform, it offers heatmaps and video recordings under VWO Insights and VWO FullStack.

With such a great all-around tool comes greater creative capabilities. But this can complicate the process of testing if not done as simply as necessary, which can then result in confusing issues and prevent your experiments from being perfect.

To help with this, VWO has a full knowledge base that features those questions in a VWO issues troubleshooting guides. But if that guide seems too confusing, we’re here to help.

At Brillmark, we’ve been doing A/B testing experiments for all kinds of websites for a long time. This allows us to learn a lot about every major A/B testing tool. Many of our clients come with their own set of problems regarding A/B testing, including with VWO.

Because of this, we’ve segmented the most common points where users get stuck while experimenting in VWO. Common questions, matched with answers from VWO’s troubleshooting guide, are as follows:

Common VWO Testing Tool Issues and Their Fixes

1. The Website Is Not Loading in the VWO Editor

The editor must be used to build A/B tests in VWO. For that, the test webpage needs to load on the editor. Simple — in theory. But when the editor starts to show errors with loading in VWO editor, it leaves the user searching for answers.

Source: VWO

There could be more than one reason for this. Follow the steps below as a checklist to identify the friction and resolve it accordingly:

• VWO suggests adding the VWO smart code on the webpages for the editor to pick it up directly and eliminate the step where it has to load it through the proxy. If the VWO editor uses the proxy to load the website (which only happens when the SmartCode is absent), it may not load properly due to issues with fetching JavaScript (JS), CSS files, and more.
• Try clearing existing cookies and refreshing the page.
• Check if there are any JS blockers (such as ad blockers) installed on the website. If yes, then they shouldn’t be blocking requests from vwo.com and visualwebsiteoptimizer.com. Allow those requests to come in by changing the plugin’s setting.
• Make sure the primary URL is the exact URL of the specific page, as group URLs and URL patterns may not load.
• Verify VWO code installation.
• In the browser’s settings, check if third-party cookies are allowed. The browser must allow third-party cookies, which in this case would be from app.vwo.com or dev.vwo.com. You can add these two URLs as exceptions without having to unblock cookies for all.

2. Visitors Are Not Being Tracked Properly by VWO

While setting up an experiment in VWO or any other A/B testing tool, allocating traffic into percentages for the original and the variation is a mandatory step. If not done properly, it could result in traffic not showing up in the analytics of the variation. The issue is the inconsistency of traffic results being different between experiment analytics and regular analytical tools.

This could happen due to traffic not being allocated properly at the time of experiment framing.

Follow the steps below to explore the issue and resolve it:

1. Where traffic is not being tracked at all:

• Check if the VWO Smartcode is installed on the URL; if not, install it immediately.
• Visitors should land on the experiment-specific URL. Check if it is correctly submitted in “included URLs” or not. If the website has some redirection internally, provide the final URL on which you want the visitors to be tracked as per the experiment.
• If you have included the HTTP version of URL in the “included URLs,” VWO won’t track the visits on the HTTPS version or vice versa, as the tool differentiates between the two. For example, if the website URL is http://www.example.com and it’s accessible through https://www.example.com as well, you must specify the pattern of URLs as http*://www.example.com/.
• Recheck the segmentation conditions to find any faults (if any) and get them resolved.

2. If VWO is showing different analytics than that of other analytics tools:

• If you’re comparing the experiment data of VWO and Google Analytics side by side, you want to know the reasons behind the difference in data between the two. VWO suggests you track only unique visitors and not the repeated users. VWO drops cookies for 100 days, so if a user who has become part of the experiment for the first time returns to the experiment page within those 100 days, they are not considered unique visitors and do not get tracked again.
• In an asynchronous version of VWO smart code, the default timeout value in settings_tolerance and library_tolerance parameters is 2000 and 2500 milliseconds, respectively. If you think the visitors are tracked less than expected, you can increase it to 4000 milliseconds. Then users with slow internet can be included as well.
• In VWO split URL tests, enable the cross-domain tracking option for cases in which the control and variation are on different domains.

Note: VWO won’t be able to track users who have disabled JS and cookies in their browser settings. And, due to segmentation conditions (if you’ve applied any), there could be some sort of discrepancy between data that occur in different analytics tools.

3. Enabling Tests to Run Across Multiple Domains in VWO

To track the goals on multiple domains (also known as cross-domain tracking), it’s necessary to enable the “multiple domains” option while experimenting. This applies to tests in which the main domain being tested includes a goal that says “visits on another page,” which could be some other domain.

That goal will only be tracked when the user lands on that other domain. You must enable the multiple domain options as part of the final steps of the experiment creation process.

Note: One exception to this is that you cannot use multiple domain options if you are self-hosting the JS files.

4. Changes Are Not Appearing in the Variation

You’ve worked on a campaign for so long, and when it goes live, the variation in the experiment shows no change at all. It’s infuriating! And it nullifies the entire reason for A/B testing. That’s why you should always check the preview before making the test live. That way, you can explore the problem, find the error in the steps, and fix it as soon as possible so the test can start successfully.

This could happen in any A/B testing tool if any of the following steps are not followed, but here, our main focus is VWO and their suggestions for resolving this issue:

1. If the VWO code is not installed on the website:

Check if the VWO tracking code is installed on the website. A small preview debugger window appears in the bottom right of the previews, so when the tracking code is not installed, this doesn’t show up (only the original page shows up). The variation will only show up if the tracking code is present.

To install it, copy the tracking code from snippethttp://app.vwo.com/#/settings/code

and then paste it into the <head> section of the campaign page.

Source: VWO

2. If VWO code times out:

VWO code could time out in cases of slow internet; if so, the preview window displays, “VWO test settings timed out.” You can always increase the default time-out values of setting_tolerance and library_tolerance between 2000 and 2500 to 5000 and 5000, respectively.

Source: VWO

3. If there’s inconsistency in the load timing of elements:

When the changes do not upload at the time of their specific script on optimized elements, the change does not show up.

To confirm this, copy the entire code from the VWO editor and find it in the variation page’s console in developer mode. If the changes then show up, it means the elements did not load properly along with the JS scripts.

To resolve this, update the code in the editors through which you framed the experiment.

As shown below, in cases where you add the elements through AJAX after the tracking code is already loaded, you can now update it from the “add JavaScript” section present in the VWO code editor. Click the “poll until the element is changed” option to add the function.

Source: VWO

To do this via editor operation, add the below code snippet to the code editor window. This puts the element on the webpage, then applies the changes in the element of the variation. This method helps when you have added the element later with the AJAX, embedded elements, or JS.

vwo_$(function() {

_vis_opt_element_loaded(campaign_id); //change campaign_id as per the experiment ID

});

For custom JS, use setInterval() function to wrap up the snippet. It can also be done in Dom ready function:

vwo_$(function() {

// changes here

});

4. If styling changes do not get saved in Editor:

When CSS changes are not saved in the editor, try to manually change it with the help of !important property. For example:

#id
{
Color:red !important;
}

5. If Preview Mode shows “Element not loaded”:

This message appears in the preview mode when the element is not present. Fire the CSS path of the element in the console. If it returns blank, that means the element is not there.

Source: VWO

The reason behind it could vary. Find the static CSS path for the element, and add the same to the editor. Get more details from this document by VWO.

5. How to Use the Winning Version as the New Control for Future Campaigns in VWO

Once a variation wins in the experiment, it can then be implemented on the site with full traffic allocation to complete the circle of optimization. To use this variation as a new control for any future A/B test experiments in VWO, follow these steps:

Solution #1: Re-create the changes in the back end of your website.

1. Copy the code snippet of the specific variation changes.
2. Host the new page in place of the previous control of that variation.

Your developer can help you with the coding portions to achieve this. If you lack a skilled developer, you can always hire outsourced A/B testing developers.

Solution #2: Use VWO to deploy the winning version as the new control.

1. Pause the current campaign, then create its clone.
2. Delete all other existing variations except the winning one.
3. It is suggested to always use Clone to make changes in the running variation, as the running variation will have existing users who will keep seeing that version because of the campaign cookies. The Clone campaign will create new cookies, which will include every user as a new user.
4. Add the new variation for the winning variation, which is now the new control.
5. Disable the previous control.
6. Set the winning version from the previous campaign as the base for calculations so it can now be the control for any upcoming experiment.

Best Practices for A/B Testing in VWO

There are many best practices per the types of tests you wish to build in VWO. The following is based on A/B testing best practices, as suggested by VWO. We’ve selected a few of our favorites to cover the most common areas which need an extra push to perform the best:

• While using the VWO editor for A/B testing to make variation pages, do not use the “edit HTML operation” on dynamic changes, as it won’t have the desired effect on them. Use “edit HTML” for minor optimizations to change a single element of a web page.
• Do not use “move” and “resize” for responsive pages, as it may affect their responsiveness. Instead, use “media query” and custom JS code to apply these changes.
• It is suggested to not change the prior value in the middle of the campaign, as it can delay the results because VWO will have to adjust the setting for the new values accordingly.
• To track if the goals are working, use live previews. You can use the guide here to learn how to preview goals.
• There is a “simple” mode activated by default in the URLs. In this mode, you’ll need to add “exact URL match.” To add a URL pattern or run a test on multiple pages, you must use “advanced mode.”