Marketing is constantly changing. Many of the tactics that worked five years ago – or even last year – no longer produce results. The B2B buying cycles are changing. We have reached the point of content saturation. Customer expectations are rising and consumers have more power in the supplier-consumer relationship than ever before. Success now requires a strong focus on customer experience and customer success.

As marketers, we’re breaking new ground and are forced to constantly experiment with new, innovative tactics in order to stay competitive.

Fortunately, the Marketing Automation (MA) systems we use provide tools to make this experimentation easier. An effective way to evaluate new marketing tactics is with A / B testing.

With A / B testing, we can use two variants of the same marketing tactic side by side and compare the results. In this way we discover which of the two is more effective. This removes some of the guesswork from our disruptive marketing experiments and allows us to use data, not intuition, to determine which new tactics to focus on.

Below we dive into A / B testing, discuss when you should use it, and outline some best practices for mastering A / B testing in your marketing organization.

What exactly is A / B testing?

In A / B testing, we use marketing automation to run two approaches to the same marketing tactic at the same time. With the best marketing automation solutions, you can run A / B testing in pretty fine detail.

How does it work from a technical point of view?

We can use email marketing as an example to explain the process. If you are testing two versions of the same email, your MA system will send a sample of each version to two subsets of your total audience.

Your system will then wait a certain amount of time to measure the performance of the two. Which one had the highest open rate? Which one saw the most click-throughs? What caused the most unsubscriptions?

Once there is enough data to determine which version is more effective, your system will propagate that version to the rest of your audience.

Why should you use A / B testing?

It’s important to know which marketing tactics will best target your audience, attract new leads, and get the most lead conversions. If you’re testing an email campaign, the A / B test will tell you which email versions generate the highest open rates and click-through rates and which generate the best leads for marketing.

Experiment with an email campaign

When applying A / B testing to an email campaign, you can experiment with the subject lines of your emails, the copy of the emails, or the images you use. You can experiment on a more detailed level by testing two different fonts, font colors, email template designs, headings, sub-headings, names in the email “From” line, and so on.

Test elements of a marketing campaign

You can also use A / B testing in different parts of a digital marketing campaign. Compare the results of two different landing pages, lead generation forms, or calls to action. You can also test two different marketing campaign sequences to determine the optimal cadence for campaign touchpoints.

Don’t forget statistical significance

As you conduct A / B testing with new tactics, you should apply your test to sample sizes large enough to produce statistically significant results.

If your target audience is 3,000 leads and you only send your first trials to subsets of 10 people, your results won’t be reliable enough to represent your entire audience. Sample size (s) is key to effective A / B testing.

Need a quick refresher on statistical significance? Brush up on the topic.

How do you plan and conduct an A / B test?

A / B testing is about generating data that will lead to actionable insights and will allow you to confidently use the tactics that are most effective for your target audience. What works for one industry may not work for another.

When planning an A / B test, it is helpful to follow and adhere to a set process. This leads to a consistency of your results and strategies. Here is an example of an effective step-by-step process:

1. Define your hypothesis

Determine the question you want to answer. For example, should I get this marketing email from “The [Company Name] Team? “Or does it make more sense to send it by individual salespeople? You’ll assume this will be more effective, but that’s just a guess. Your A / B test will confirm (or not) your assumption.

2. Determine which and how many tactics to test

Are you going to keep it simple and test two subject lines for emails? Or do you also test the sending data to determine which day of the week most emails are opened? If you are new to A / B testing, we recommend testing a variable first; B. an email subject line. It is best to ease your way into the process and learn as you go.

3. Calculate a statistically significant sample size

Do the math and determine the appropriate sample size for each subset of your test so your results can reliably tell you which tactic to use. If you don’t, you’ll be wasting your time as your results won’t accurately predict the results you expect when you roll out your tactic to your entire audience.

4. Test your test

Quality assurance (QA) is critical to effective A / B testing. Take a test drive of your experiment with some test leads in your CRM database. Make sure you are in this group of test leads so that you can go through the process yourself and make sure everything is set up correctly.

Click on every link, fill out every form, open every email, and so on. Then, review the results to make sure the actions you have taken are represented correctly. If any part of the process is flawed, you’ll want to identify it before running your test on actual leads or customers.

5. Set your time frame

How long will you wait while the test group data is compiled before determining the effective tactic and sharing it with your entire audience? The answer is that there is no definitive answer.

The wait time depends on how long it takes to collect enough data for your results to be statistically significant. That depends on your audience size and how fast they act. It is important not to prematurely expand any tactic to the entire group.

6. Provision, measure and analyze

Once you’ve implemented the winning tactic, wait a reasonable amount of time and then measure the results. You may have noticed that while Tactic 1 was more effective during your trial, the results varied significantly when delivered to the entire audience.

When this happens, you may want to take another test and compare the same tactic with another to confirm that this is an effective approach to engaging your audience. There is no harm in retesting a tactic, as you need to understand why a particular tactic was successful.

When should you use A / B testing?

Do not test A / B random tactics out of curiosity. You need to set a goal in A / B testing as this is most helpful when you’re trying to solve a problem or improve something that doesn’t work the way you need it to.

For example, when conversion rates have dropped, it’s time to A / B test new tactics. When customer retention rates start to drop, pull out your A / B testing playbook. If you just can’t generate new leads, it’s probably time to experiment with new tactics.

What do you need to do A / B testing?

First of all, you need to be able to measure certain metrics, most of which cannot be measured without technology. Without software that automates these processes, you cannot measure the click or open rates of email campaigns.

In short, you need a CRM that stores customer and lead data, as well as a marketing automation solution with the ability to run A / B testing. Some CRMs, such as Insightly, contain integrated MA functions to form a unified CRM system. These are the best solutions for effective A / B testing.

Final thoughts on A / B testing

After you understand the basics of A / B testing, as well as the reasons why, when, and how to do A / B testing, it’s time to get to work. Think about when you would like to take your first A / B test trip.

If you don’t have the right technology to conduct A / B testing, now is the time to think about implementing new software – such as a unified CRM – in your company. With software like this, you can do a lot more than just test new tactics. It automates many manual processes, ensures data integrity and gives you a better customer experience as well as many additional benefits.

If you want to learn more about CRM and MA software, you can schedule a free demo with Insightly. We’ll walk you through the benefits you get by using a unified CRM with marketing automation built in.

Request a demo