How to know when to bet the house and when to play it safe when you’re formulating CRO test ideas and UX implementations.
Conversion Rate Optimization is a fine balance. Big, sexy tests with amazing results are the stuff huge contracts with clients and in-house promotions are made of. To get there, though, you have to run smaller, decidedly less-sexy tests with incremental results that don’t necessarily get rave reviews from management. But if you do those smaller tests right, it builds the kind of trust you need in your organization to take the risks required in the big tests.
Pre-Test Questions
To understand risk and reward in testing, you need to start with some basic questions. They will dictate what kind of test you should run and whether it should be a test at all versus an implementation.
How Obvious Will the Change Be to the End User/Prospective Customer?
Will this test appear to be a sea-change to the user, a subtle evolutionary change, or hardly noticeable as they go about their business on your site? Optics can be shaky ground for customers, even if the substance of what you’re asking people to do is relatively fixed.
How Much of the Team Is Involved in Executing the Test? How Big of a Lift Will It Be to Accomplish?
Does this require a CRO, a developer, and a content strategist? Or does this require channel marketers, IT, and leadership decision-makers? The more cooks in the kitchen, the clearer the test design will need to be, and the more calculated the impact to justify the time investment.
What KPI Are You Trying to Affect?
If it’s a navigational KPI vs. a conversion KPI, you can potentially take bigger risks in the design and look to move the needle much more drastically. If it changes how people give you money, you’ll maybe want to be more measured in your approach.
What Kind of Sample Size Do You Expect?
Will tens of thousands of people see this test? Or does this page or part of the funnel only get a few hundred visitors in a month? It might not be worth splitting your audience or trying something groundbreaking if you aren’t going to collect enough data on each variation.
Test Types by Risk
Once you have the answers to those questions, you can determine what kind of test type to run based on what you find. I like to give these tests unique names to identify them by how ambitious they are and how they might appear to customers.
The Superficial
This is the test equivalent of rearranging deck chairs and polishing the brass on a cruise ship. Your customers probably aren’t going to notice, but it might subliminally influence the way they behave.
Test Characteristics: CTA button color changes, CTA button style treatments, navigation reordering, adding iconography
Team Members: CRO, front-end developer (optional)
KPIs: CTR
Risk: Low
Reward: Low
The Copywriter
A different approach to product or service messaging can work wonders, but this test keeps all the load-bearing functionality and structure of your page in place. It spins the thing you’re selling in a new light without disorienting the customer too much.
Test Characteristics: Headline changes, body copy changes, CTA button text changes, navigation text renaming
Team Members: CRO, content strategist, front-end developer (optional)
KPIs: Time on page/site, bounce rate
Risk: Low
Reward: Medium
The Dev Special
This is where you start talking to front-end developers and moving things around in a noticeable way. Your customers who are really paying attention will perceive the change, and it’ll more actively influence their progression through your content and conversion funnels.
Test Characteristics: New images or image treatments, changes to navigation positioning or functionality, changes to layout order of existing content, the addition of new interactive elements
Team Members: CRO, content strategist, designer, front-end developer
KPIs: CTR, time on page/site, bounce rate, funnel start, add to cart
Risk: Medium
Reward: Medium
The Facelift
This is where the website’s end goal doesn’t change, and neither do the baseline navigation objects, but the overall experience can look very different from user to user.
Test Characteristics: Added elements of personalization, content recommendations based on behavior or incoming campaign, dynamic CTA buttons
Team Members: CRO, content strategist, channel marketers, designer, front-end developer, marketing manager
KPIs: Time on page/site, bounce rate, funnel start, add to cart, conversion rate
Risk: Medium
Reward: Medium
The Dev Deluxe
These tests start to really pull at the structural foundations of a website and how customers interact with you. This work might involve back-end developers to restructure how and what data you collect with forms and shopping carts on the site.
Test Characteristics: Changes to contact or lead forms, changes to e-commerce carts or checkout funnels
Team Members: CRO, content strategist, designer, front-end developer, IT, marketing manager
KPIs: Time on page/site, bounce rate, funnel start, add to cart, conversion rate, leads, transactions, lead quality
Risk: Medium
Reward: High
The Overhaul
You’ve given up on the viability of your existing site at this point. It’s fundamentally broken and you need a change, but you want to show some users a completely different version of the site before you cut over to it entirely. This could change how customers think about you altogether.
Test Characteristics: Complete redesign, major functionality and navigation changes, different imagery and typography
Team Members: CRO, content strategist, designer, front-end developer, IT marketing manager, CMO
KPIs: CTR, time on page/site, bounce rate, funnel start, add to cart, conversion rate, leads, transactions, lead quality
Risk: High
Reward: High
Test or Be Tested
Now that you have a framework for evaluating your testing decisions, remember this: hand-wringing and inaction can be just as risky as any test. If you don’t evolve how you present your products and services, the customer’s needs and preferences will evolve without you. Convince your stakeholders to take a leap of faith, no matter how small, and the rewards will always outweigh the risks.
