Why you should be continuously AB testing your website

What do visitors say about your website when you are not around? Although it's hard to say, we all know that it could be the difference between selling your tickets, or not.

You probably have some clues, though, but can you assess it?

Let's say that you change your copy along with some information architecture and see an increase in ticket sales in the next 30 days. Is this enough to say that the changes were successful?

What about seasonality? What about the weather, that big event that brought thousands of people to your city, or all the other variables that can influence your results? How to get rid of them?

Well, you cant.

But there is a very good method that can help you with reducing the impact of those variables: AB testing.


What is AB Testing?

Simply put, in the web environment, AB Testing is a method used to compare the performance of two versions of the same web page or app screen, while exposing them to the same audience, at the same time.

If you have never heard about it, you're probably wondering how to expose different page versions to the same audience at the same time, well, and that's why this test is also known as Split Test.

The secret lies behind the scenes, where 50% of the traffic is randomly assigned to the original page ('control page') and the other 50% to the new version ('variation page'). It attenuates external influences and allows direct comparisons, as both pages are exposed to the same scenario.


Why should you be AB testing your website?

AB Testing allows companies and individuals to prove their hypothesis with data, eliminating the guesswork as much as possible, and implementing changes based on a clear methodology.

Whenever you decide to test your assumptions regularly, you are actually deciding to learn from your own, specific, audience, you are deciding that you're not creating your strategy and making decisions based on external findings, and you are deciding that you'll adapt your website to better serve your own customers.

And the benefits? Well, from this point on, the benefits can be multiple.

Just within the ticket sales example, you could understand how to sell more tickets, how to increase the cart size, how to increase the up-sells, what attractions to highlight, or what call-to-action to use.

It's also allowed to assume that, as those experiments will help you to understand your customers better, they will also help you to enhance your entire customer service, even the offline experiences.

Why? Because by the end of a couple AB tests you'll have a better understanding of your customers' needs, and as more you know about them, better you will be able to serve them.


How can I start AB testing?

AB Testing is a statistic-valid experiment. It means that you should have a representative sample, a clear hypothesis, and a way to analyze it between some confidence intervals.

That said, many tools can help you with setting your experiments up and analyzing the results. One important thing, though is the traffic. You'll usually need to have enough traffic to create valid results, by exposing your audience during a specific timeline (usually 30 days are enough).

The process is simple and should start with a good understanding of the problem, otherwise, you'll test just for the sake of testing, but your results are not going to help you to sell more.

1. Identify the problem or the opportunities of improvement: by looking into your analytics you could find high cart abandonment rates in specific pages, or you could see that some of the forms are converting better than others. This is up to you, but make sure to understand what is your expected outcome.

2. Define the goals: Now that you found opportunities, it's time to define what you want to improve and how you'll recognize success. It could be to increase a button click, the time on page, the ticket sales, or whatever makes more sense for you. Keep in mind that this is what will help you to determine the success of the experiment.

3. Write the hypothesis: After defining your goals it's time to write the hypothesis down. This is where you'll come up with ideas and will explain what you expect from them. Make sure to create a hypothesis that can give you an insightful answer. It means that if you change everything on your page you won't know what is really moving the needle.

4. Create the variation page: From a simple headline to a complete redesign, this is the piece that will translate your hypothesis and be shown to your variation audience.

5. Set the experiment: This is a very important part of the job as well, and tools like Convious can help you out with properly setting this step up.

6. Wait the necessary time (usually 30 days): During this time half of your traffic will be exposed to the original page and the other half to the variation and you'll be able to see which page is generating more positive outcomes regarding your goals.

7. Analyze the results: Although it's tempting to analyze the results before the end of the experiment, you should never do it. Whenever the test is finished you'll analyze the results and understand if your hypothesis was confirmed, or not.

8. Implement or keep testing: In the case that your hypothesis was confirmed, congratulations! It's time to implement it to 100% of your traffic! Otherwise, it's time for your next round of testings!



At Convious, we run AB tests regularly for our clients and we frequently see small changes creating massive uplifts. It's not about the change itself, it's about your audience. AB testing demands knowledge about the problem, and this is where you should spend most of your time.

Every website has opportunities to improve, and this method has been proved so effective that has been adopted by companies of all industries, all sizes, and even by companies that explore offline business models.

You don't need to start with big changes. Try to solve small problems, learn from it, and after some time you'll be asking deeper questions and creating a more relevant hypothesis.


You can always perform AB testings yourself, but if you are looking for the experts in the subject request a free demo with our specialists and let's have a quick chat!


Have you been AB testing your website regularly? Let us know what you think about it by commenting below, and good sales!

New Call-to-action