Don’t just run a test. Plan it.

Digital Marketing / August 08, 2017

Here we outline the steps for creating a simple A/B testing plan, so you know exactly what you’re testing, how you’re testing it, and why.

---

So you want to test some elements on your website or landing page. You’ve created is the best possible version of a page, but you have nagging questions: Which is the best tagline? Where should I place the CTA? How many fields should there be in my form? You can debate the answers with your team for months, but someone else knows the answer: your visitors.

All the best practices in the world won’t answer your question as effectively as running tests. Running well-thought-out A/B tests will quickly show you which combination of on-page elements best motivates your visitors to do what you want them to do.

Running tests on your website can be intimidating at first. It’s yet another process to implement, and a process that can be emotional for people with strong opinions about what works, and what doesn’t.

Most marketers begin their conversion rate optimization journey by running A/B tests with one of the industry’s popular testing tools, like Unbounce, Optimizely, or Visual Web Optimizer. While the tools are relatively easy to use, the challenge comes in deciding which elements to test, and which success metrics to use to determine success.

By creating a simple testing plan, you can be sure that you are using the best metrics and comparing elements that contribute to the success of your business. Not only does this help with your bottom line, but by aligning A/B tests with your organization’s business metrics, you stand a better chance of winning the ongoing support of your management team.

Every A/B test should begin with a plan.

Don’t just jump in. Take some time to think about what you’re going to test, why, and how you will define success. This will help you immeasurably: it will explain why you need to run a test to any naysayers, and it will make it easier for you to communicate and implement the outcome.

Here’s what goes into a testing plan

Define your business objective.

We hope that you already know your organization’s main objective. By beginning a test plan by reiterating your business objective, it helps you ground your test plan by ensuring the decisions you make at each step continue to support your organization’s bottom line.

In light of your business objectives, define the goals of your website.

Hopefully, this is another no-brainer. Now that you have clearly stated your business objective, extend that to your website. Remember: sometimes your website’s goals differ from your core business objective.

Identify the metrics that drive your website goals.

What is the number one success metric you track to know whether or not your website is reaching its goals? In many cases, this will simply be revenue. But maybe you’re interested in something else altogether, such as engagement or shares…  Whatever it is, this is the metric you will measure against.

Identify the elements on your page that advance these metrics.

On the page you want to test, determine which elements (buttons, images, copy, forms, etc.) contribute directly to driving the metrics you will be measuring during your test. By choosing your elements based on the success metrics they drive, your are making sure that your test remains relevant to your business objectives.

Of these elements, choose one that is underperforming, or one you wish to improve.

Here is where you choose what you are going to test. Based on metrics, is there an element that is underperforming? If you’re not sure, is there an element on your website that your team has debated or questioned? One you’re not sure about? Isolate this one as your first test.

Develop a hypothesis to test this element.

Here’s the heart and soul of your test plan. Write down exactly what you want to test, and why. For example: “We are hypothesizing that by increasing the size of our product images we will be able to increase online sales and therefore revenue.”

Identify your success metrics.

What does success look like? In other words, by testing, what are you hoping to improve, how and why? Creating this statement will help you explain the outcome. For example: “We are testing large and small product images to determine whether image size has an impact on conversion rates. The winning result will have a higher product conversion rate with a 95% confidence level.” Now you have clearly identified and documented your process.

Now you are ready to test

This process ensures you are conducting an A/B test that has a tangible impact on your organization’s business goals.

If you replicate this process every time, not only are you clearly defining the what, why and how of each test, you are also creating an ongoing document of your website optimization process. This becomes a perfect template for writing that post-test presentation or case study for management. The end result is an ongoing practice that enables continual testing and improvement throughout the lifecycle of your business.

Need help with conversion rate optimization or A/B testing? Our CRO experts are here to answer your questions. Get a CRO proposal here.