As marketers, our job is to make recommendations to our clients, but we can’t base them on intuition alone. Besides making educated hypotheses on what could be harming usability or causing friction on a site, CRO strategists need a way to see whether our recommendations will positively impact conversions and usability before implementing changes directly on the site. Luckily for us, we have the opportunity to support our recommendations with data using Google Optimize.
This guide will walk you through the basics of setting up Google Optimize, what it can do, and a few things to keep in mind as you start testing.
What is Google Optimize?
Google Optimize is a free testing tool that can help you create and monitor your experiments to help you accomplish your business goals. Whether you are curious if users will be more inclined to purchase if a CTA is red or provide proof to your boss on why you restructured a landing page, Google Optimize can help you prove your points and make your case. There are many reasons to use Optimize, and many things it can do for you.
What Google Optimize Can Do
Within Google Optimize, there are three different types of tests and two site customizations you can use to improve user experience and conversion rates, depending on your needs. In this post, we’ll talk through testing and personalization (we won’t be talking through site banners).
A/B (Split) Tests
An A/B test, also known as a split test, is the most basic and most used test you can perform in Google Optimize. An A/B test allows you to test the risk and reward of a desired change by splitting your audience and presenting two versions of your site: an original version to one group and one with a variable to test to the other group. Optimize will then measure which group had a more successful interaction between the original and the variant, so you can learn whether or not to implement the change directly on your site.
For instance, if you are considering updating a header on your page, you could test your original against the new copy to see whether it can aid in conversion. If you run this test and there is no clear winner, it indicates that the problem lies elsewhere, thus inspiring your next test.
A/B Tests are typically the best fit for most websites, but there are instances where you may want to create more complex test methods.
A redirect test uses the same framework as an A/B test. It still allows you to test two versions of web pages against each other; the only difference is that instead of creating a single change within the test page using Optimize’s visual editor, you simply add the URL of the variation experience. Optimize will then redirect the test audience from your control page to the variation page.
Redirect tests are a great option when variation experiences require more edits than are recommended within the A/B test editor (around 60 lines of code), like when you’re testing a new form format. Some CRO professionals will use redirect tests to compare the performances of two distinctly different pages. While this is great for reducing the risk of introducing a new page layout, we offer a word of caution that it can be difficult to identify why the winning page was more successful.
A multivariate test helps you determine the best performing combination of variants. Using a multivariate test is typically the best fit for sites with very high traffic or baseline conversion rates. For example, let’s say you wanted to test which headlines or images to use on a page, and you have two options for each. Instead of setting up an A/B test, waiting for the results, and then iterating on your findings, you can test which combination of headings and images performs the best all at once. This testing format is great because it can speed up testing, but remember that it requires very large sample sizes in each experience, and exponentially increases the complexity of analysis. This is varsity-level testing.
A multi-page experience is similar to a classic A/B test, but it allows you to create a variant across multiple pages of the site to test a change throughout the user journey consistently. For example, if you wanted to test whether referring to the checkout process as a “cart” or a “bag” helped nudge users to convert more, you could change the CTA on product pages (“Add to Cart” vs. “Add to Bag”), and then change the corresponding title on the checkout page to see which verbiage was more successful.
Personalizations are a bit different than the rest of the test options in Optimize. Think of this option more like a marketing technique, similar to targeted ad campaigns that aim to show the right content to the right visitors at the right time. Personalizations allow you to show a set of changes to your website to display in front of anyone meeting your targeting conditions. Unlike the other test types, personalizations do not use variants and can run forever.
Personalizations can be a stand-alone option if you solely want different users to have different experiences on your site (for example, show a unique phone number when a customer visits from outside your sales area, or add more context to a form if users arrive there from a specific referrer). Still, personalizations can also pair with the findings from other types of tests. If you find that the variant outperforms the original page on a site, you can use personalization to deploy this updated version while the development team changes the site.
How to Use Google Optimize in Four Steps
Now that we’ve talked through what Optimize can do, let’s talk through the how. To run tests in Google Optimize, you must first create an Optimize account and install the Optimize script on your website. We recommend setting up Google Optimize via Google Tag Manager (GTM) for the fastest, most consistent deployment across your site. Once you are up and running, you can start building your tests with these four steps.
1. Create an Account and Add a Container
I’m going to hand you off to Google for the nitty-gritty of setting these up. Google Support has a great guide for setting up your Optimize Account and connecting it to your Analytics Account (a crucially important step for measurement). Once you get into your Google Optimize account, it will prompt you to add a container. This hierarchy is similar to Accounts and Properties within Google Analytics. Once you have your container, it’s time to install Optimize on your website. We recommend that you install Optimize with Google Tag Manager. This will allow flexibility in how and when Optimize is deployed. For example, you can easily pause the tag if you take a break from testing.
Once you have everything set up as described, you will want to go into the preview mode in GTM and make sure your tag works where and how you intended.
Once you confirm that the Optimize tag is firing correctly in preview mode, publish the tag to your live site. Once done, you’re set to create your first experiment in Google Optimize.
2. Set Up Your Test (Create Experience)
Once you are set up in Google Optimize, it is time to select the upper right button “Create Experience” to set up your test and choose parameters.
First, name your experience something that has not been previously used, and be sure to include information that will distinguish it from other tests you may run. Underneath the name, you will include the URL of the page you will be testing, called your editor page.
Next, you will select which type of test you would like to create between A/B, Multivariate, or Redirect.
Create Variant (or URL)
Once you are in your new experience, the first step is to set up the change you want to make on your site by selecting the blue “Add variant” button. Next, choose a name (we will call it “Variant 1”) and select “edit” next to your new variant.
This is the part where you can start making changes to your site that you want to test against the original. The editor allows you to change copy, element order, style colors, and more in a few simple steps. Other more complicated changes could require some help from a developer. At this stage, you will also determine what proportion of test visitors will see the variation experience. We recommend a 50/50 split.
Select Your Audience
Once the variant is set that you would like to test, the next step is to determine who will see the experience. You can set it to appear to all users who visit your site or customize it so that only certain users encounter it. For example, you can specify that only people in certain locations or from certain devices will participate in the experiment.
Choose the Objective
When it comes to choosing an objective or what you want to measure, there are two options. You can either choose from a list of your current Google Analytics Goals or create a custom objective. Custom objectives are useful when the goal of your test doesn’t correspond to a GA goal. For example, you can select a specific event on a webpage, like whether someone clicks on a “Next Step” button to measure whether a change you made on a page influenced a user to start a form on that page.
Determine Test Duration
When we set up tests, the duration will depend on the site’s typical traffic. If it is low, you may need to run a test longer to collect enough useful data. As a rule, we recommend you choose pages for tests that average at least 1,000 pageviews a month over the last three months. This will ensure that your experiment gets steady traffic and can complete in a matter of weeks, not months. If your site doesn’t get the traffic needed for a typical test duration and you don’t want to run a test for months, there are other A/B test strategies for low-traffic sites you can try.
For most sites, the sweet spot for running tests is between two and four weeks. A minimum of two weeks allows us to account for any cyclical nature within the week. A maximum of four weeks allows us to limit the introduction of too many other variables. If you plan to run your tests to statistical significance, we recommend using a tool like the Pre-Test Analysis on A/B Testguide to determine exactly how long your specific test needs to run. Make sure to do this before you start your test.
Semi-Pro Tip: If there’s a page that absolutely positively must be tested quickly but it’s low traffic, consider whether you could use paid ads to drive the right kind of prospects to this page at a higher pace.
Before you launch your test, select your desired settings. First, check that Optimize is correctly installed by clicking the “Check Installation” button. If you’ve set Optimize up correctly, you’ll receive a little green indicator that shows you that you are good to go. Next, we recommend turning on email notifications so you can receive updates about the experience without logging in to check in as often. Depending on your testing needs, select the percentage of visitors to your site who will see the experiment. For speed, we recommend that 100% of your traffic can see the experiment (they will be split based on the variant weight you assigned earlier).
Then, choose an event where the test gets triggered between pageload, continuously, or on a custom event. Pageload is most common and the default option. Continuous activation is a good option for tests on dynamic pages as it will apply the variation’s changes to newly matching elements as they load. A custom event can be used when the event helps determine the test’s audience. For example, if you have a multi-step form and want to determine the influence of a change to the second step, you could trigger your experiment on the “Next Step” button.
3. Launch and Monitor Performance
Once all of the variants, targeting, and settings are in place, it is time to push the shiny “Start” button you have been eyeing. But just because all you need to do now is wait does not mean your work here is done. It is imperative to monitor your test’s performance throughout the experiment to ensure that nothing is broken, that your variant is not creating a worse experience for users, and know when to pull or extend the test’s duration.
Google Optimize makes it very easy to monitor results within the tool. Within the container under the “reporting” tab, you can view the data in various ways. At the top, you can view the sessions over time broken down by each day. This view is important for making sure data is being collected and that there are no alarming drops in numbers. We recommend clicking “View full chart” rather than just peeping in at the thumbnail. We once had a test we let run for four weeks, were checking it weekly, and missed that it had broken ten days in. That was unlucky timing.
The full chart:
4. Analyze the Results
After your test has been running long enough, you can often (not always) start to see a pattern. One way to see how a test is performing is by looking at the “Probability to be Best” section under “Reporting.” If the percentage is 80 or above, it indicates a strong likelihood of becoming the winner over time. In this example, the probability to be best was 100% for our variant, which allowed us to have enough data to implement the variant on the live site.
In some cases, it is helpful to dive deeper than the Probability to be Best to understand the data. Luckily, Optimize does some of this work for you by providing the data that directly compares the performance between the original and the variant. In order to do so, Optimize calculates the modeled conversion rate, which calculates how your variants have performed against a selected objective over time.
There are two ways that Optimize displays the modeled conversion rate. One is in a box and whiskers plot, while the other is in a chart.
There are a few indicators to consider when looking at performance. First, it is important to note whether the blue and black dotted lines diverge. Below is an example of a variant that outperformed the original page, where the lack of overlap shows a clear winner.
Once you find whether a variant helps or hurts your site, it is time to implement the successful change or go back to the drawing board to hypothesize other variants to test.
Tips to Consider When Using Google Optimize
Although you are probably ready to start testing, there are a few things to keep in mind when running experiments in Optimize that could save you in the long run.
One thing to be aware of is potential cross-contamination within your tests. If you run more than one test at a time, avoid this by making sure that your audiences and targeting do not overlap, and that you run one variant at a time. Otherwise, you could end up with unhelpful data, wasted time, and an empty pocket, even if your variants would have been a hit. When we have needed to run two tests on one site at once, we combatted this issue by displaying each test to two different markets to avoid overlap.
One thing you can expect from testing is that you are probably going to fail a few times. There are often times where you will break a test or even just receive disappointing data that shows your variant was not successful. Just know that part of the job is to learn by trial and error. The best way to combat this is to expect and embrace it.
Reach Out to Specialists for Technical Support
One of the ways to limit failure and avoid wasting time and money is to ask experts for help. In some cases, the technical aspects of setting up a test or analyzing complex results can be beyond your expertise. We recommend working with a dedicated CRO or development team to lean on when it comes to creating variants, selecting audiences, and even managing strategies. Not only will they make sure your tests are running smoothly, but they can help ideate future tests to identify areas of improvement on your site.
And there you have it. You’re now armed with the information needed to set up and use Google Optimize, and some best practices along the way. Happy testing!