This week I was planning to write about statistics and A/B testing, as that’s what many of us think about when we think about conversion optimization. But, as I was going through the different modules that the CXL conversion minidegree has to offer, there was one lecture that really helped me put things into perspective.
To do an A/B Test You Need Conversions
Since beginning this minidegree program, one of the issues that has been at the forefront of my mind is how do I help low traffic sites with conversion optimization if I can’t A/B test? For example, to get any kind of statistical significance with A/B testing, the A/B Testing Mastery course instructor, Ton Wesseling mentioned that we should really start our A/B tests when a site has reached 1000 conversions a month. Anything less than that, it will be difficult reaching significance.
While that’s no doubt true, in this CXL video, Marie Polli not only contends that we can A/B test with half that amount and still gain insight, she also mentions several things that we can do to set up low traffic sites for success so that when they get to the point where they reach 1,000 conversions, we’ve already prepared the ground for a successful A/B testing program.
Here’s what it comes down to: CRO is not simply about the data. Sure A/B testing needs a lot of data to make significance, but in the end, that is not what is driving our insight. The central skills that we’re trying to develop as content optimizers are listening and understanding to our customers. CRO is user centric. As Polli contends in her lecture:
“Optimization is not data driven, Optimization is people driven.”
Once we understand that we are driven by our customers and not simply by data, then we can set ourselves up to be good listeners, compilers, and learners of our interaction with our customers no matter if we have one conversion or one million.
Here Polli introduces the CXL Optimization Tower. This is essentially a road map that puts low-conversion companies on a path to a place where they can start start testing and optimizing. Interestingly enough, I found a second video here that list six steps instead of the five that Polli gives in this lecture. All of them are mentioned in the first lecture, she just made them all explicit in the second.
Here are the six steps she suggests we implement to set up our organization on a good conversion optimization path:
Start Measuring Everything
At the lowest level of the optimization tower, we need to set up Google Tag Manager and Google Analytics properly to make sure that we are able to track and have good records of dimensions and metrics on the site.
Auditing GA and GTM then is essential to ensure that our measurements are set up fully and are measuring accurately. Measuring as much as we can fulfills this user-centric goal of listening and seeking to understand our customers.
Pick Low Hanging Fruit
Next we need to make sure that any industry standards of design are maintained on our site.
As users expect a certain feel to an ecommerce site, for example, we need to make sure that we place elements where they would expect them. We might place a shopping cart in the upper right hand corner, because that’s where they usually are located in ecommerce. The goal is to make the user experience as frictionless as possible.
But in doing this, we also must be very cautious. Our intent is not just to copy our competitor’s sites. We have to be careful not to repeat the conversion problems that they have on their sites. Rather, the goal is to find those elements that are commonly expected by the customer and make sure those elements are properly placed on our site.
Qualitative research involves heuristics, user research, and surveys to understand your customers better. Using their voice to write your value proposition, allows your site to feel as if it was designed and written specifically for people like them. This is also enables you to get a feel for any friction or lack of clarity users might be experiencing on a site and to fix these issues.
As qualitative research should take us several months, this will give our properly set up analytics platform to record data that we can then look at to make judgments as to where our customer’s are experiencing problems in their customer journey.
Things like heatmaps and mousetracking analysis can help us understand how customer’s are moving around the site. Sales funnels can show how customers are converting and at what point(s) they might be dropping out before completing a purchase.
Once the website starts reaching approximately 500 conversions a month, Polli suggests that we can start using testing as long as we are very intentional with our hypothesis. We need to test high impact changes to actually get to statistical significance at this point and the should reflect itself in a properly formed hypothesis.
According to Brian Massey and Wesseling in the CXL coursework, a properly set up concrete hypothesis has a problem a proposed solution and a predictive outcome. If (I do this), then (this will happen) because of (this problem). To make the hypothesis even more concrete, Wesseling suggest we add what type of metric we are trying to influence with this hypothesis.
Polli stresses that A/B tests need to be made to high impact changes. As we increase the minimum detectable effect on a conversion calculator (meaning the effect would high impact) the lower sample size we’ll need to reach statistical significance.
She suggests that we can gauge high impact effects by assessing whether or not people will notice the change that we want to implement. While changing a shade of a button color might have some impact, it is not something we will be able to notice with such a small sample size, but changing the content and clarity of our main menu is a bigger risk that just might show significance, for example.
Conversion Science Allows Us to Start Small but Aim Big
Participating in this minidegree program at CXL has been a great experience thus far, but content like this makes it much more relevant for those of us who have it as a goal to start a small agency and thus work with small businesses that might not have the same high levels of traffic as their competitors.
A/B testing might be the ultimate in CRO, the gold standard, but conversion optimization is so much more and these highly actionable points that Polli makes in these two videos definitely unwraps the science of optimization for all, big and small and gives us a window into how all businesses can apply these conversion principles no matter where they stand in their conversion optimization journey.
I’m pumped to help businesses take advantage of this. I think more than ever, what we need are customer-driven businesses, and the research-heavy and people-centered approach of conversion optimization gives us a great way forward.