Tag : A/B Testing

All about email open rates

Posted by TravelMailer at 3:51 pm

Our customers often ask us what ‘open rate’ means, and whether the open rate they are getting is any good or not. We’ve put together the following guide to open rates, which you will now also find in the help section of your account.

What is an open rate?

Open rate is a measure of how many people on an email list open (or view) a particular email campaign. The open rate is normally expressed as a percentage, and at Travel Mailer we calculate it as follows:

So a 20% open rate would mean that of every 10 emails delivered to the inbox, 2 were actually opened.

How do you measure an open?

When each email is sent out, we automatically add a piece of code that requests a tiny, invisible image from our web servers. So when a reader opens the email, the image is downloaded, and we can record that download as an open for that specific email.

It is important to understand that the open rate is not a 100% accurate measure. Recording an ‘open’ can only happen if the readers email client is capable of displaying html with images, and that option is turned on. So if you are sending text-only emails, there is no way to record open rates (the exception is if they actually click a link). Similarly, people reading your html email without images showing will not be recorded as opens.

Another issue is that your readers may have a preview pane in their email client. That preview pane might be displaying your email automatically (and therefore downloading the images) without the reader ever having to click on it or read it.

So you should never take your open rate as a hard and fast number, because you can never know the true figure. It is much better used as general guide, and as a way of measuring the trends on your email campaigns.

What is a typical open rate?

Really, there is no typical open rate. The rate obtained for any list, or group of lists will depend on how it was measured, when it was sent, the size of the list and a zillion other potential variables. There is no shortage of benchmark numbers out there, but even between benchmark figures you will find big variation in the reported open rates.

So instead of giving a specific percentage, we’ve come up with the following chart.

There are certainly some broad trends in open rates.

As list size goes up, the open rate tends to fall; possibly because smaller companies are more likely to have personal relationships with their list subscribers.

Companies and organizations that are focusing on enthusiasts and supporters, like churches, sport teams and non profits see higher open rates

More specific niche topics, like some manufacturing areas also typically have higher open rates than emails on broader topics

Why don’t you just give me a number!

So what if you or your clients just have no idea of what is a reasonable open rate? Based on everything we have seen here at Travel Mailer, and on the other research out there, the bottom line is this:

If you are getting an open rate between 20% and 40%, you are probably somewhere around average.

Very few lists of reasonable size are getting much above 50% open rates from normal campaigns. Your list may have some specific factors that give you higher rates; if so, well done.

However, don’t expect to be getting 80% open rates. People are too busy, inboxes are too full and the measurements are technically limited. If, after all that, you are still interested in seeing specific figures, see the footer for some references you can browse through.

How can I increase my open rate?

There are a ton of elements you can vary to try to entice more of your subscribers to open up your emails. Here are just a few things you could try:

Experiment with your subject lines: Try including details about the content of the email right in the subject line, instead of using your standard subject.

Send on a different day: Are your subscribers too busy on a Wednesday morning to read your email, leaving it languishing down the inbox? Maybe a Friday afternoon email would be welcomed.

Get the important content up the top: Remember that many people will see a preview of your email before deciding to open it or ignore it. Make sure your email is recognizable, and that your key points are in the top third.

The typical open rates in the chart above were derived from Travel Mailer’s own figures.

Using A/B testing to boost your email response

Posted by TravelMailer at 3:20 pm

The A/B testing feature was released earlier this year atop an amazing wave of excitement and anticipation. Many of you may have already had a chance to run A/B tests on your campaigns, however if you haven’t, you will find that its a very effective way to maximize your campaign results and learn about your subscribers. Secondly, it ensures that the message the majority of your subscribers receive is the most relevant choice – this is a win for everyone.

What is an A/B Test?

An A/B test involves two differing emails being sent out to a small portion of your subscriber list, with the most successful (‘winner’) email being chosen from the two after a defined period of time. The winner is then sent to the remainder of your subscribers.

You may have heard this practice being described as ’10/10/80 split’ or ‘multivariate’ testing (however the latter involves changing multiple parts of your campaign). Perhaps you have heard reasons why people don’t use it, such as ‘it’s too hard to do’, or, ‘by the time I get the results from the initial test, it will be too late’. The good news is that we’ve set up a very powerful and easy-to-use interface for your to conduct A/B split campaigns. As the results arrive in real-time, you don’t have to wait until the following day to select your winning email; in fact, we’ll send the winner out automatically.

So… Why test?

There are a number of great reasons why you should optimise your campaigns using A/B testing, including:

The chance to experiment and learn from different subject lines – what will produce the better open rate, ‘Receive 20% off all products at ABC Store’, or ‘Discounts on all products at ABC Store’?
The opportunity to decide what email content is most relevant and responsive – Is layout A better than B? What call to action will work best?
Deciding which From name is best – Do you go corporate ‘ABC Store’, or personal ‘Bill Storeowner’?
No matter what you decide to test, A/B testing will always provide you with useful feedback on your campaigns. For example, you will soon find that the process of choosing the ‘perfect’ subject will rapidly become less of a guessing game and more of an empirical study.

Creating an A/B test campaign

Creating an A/B test campaign is similar to creating a regular campaign – after you click the ‘Create a new campaign’ button, you will see two tabs beneath ‘Step 1: Define the Campaign and Sender’. Click the ‘A/B split campaign tab’ and you will be on your way:

In this example, we’ll be selecting two different subject lines. You will be required to enter differing subject lines for Version A and B of this campaign. You can also personalize the subject line with the recipient’s first name, last name or full name:

Once satisfied, complete ‘Step 2.1: Select the format for this campaign’ as you would on a regular campaign. If you have chosen to send two differing emails, you will be presented with the option to include both of them on this step. Next, you will move onto defining recipients. At ‘Step 2.1 – Select the recipients for this campaign’, select your subscriber list as you would for a regular campaign, then click the ‘Define A/B Split’ button:

In ‘Step 4.1 – Size of test and how you’ll decide the winner’, you can define using the slider what percentage of your subscriber list will receive the initial A/B test emails, then what percentage will receive the winning version. These percentages (A/B/Winner) are entirely up to you, however they cannot be smaller than 1/1/98%, or larger than 25/25/50%. Commonly, 10/10/80% splits are used:

Secondly, you can define what criteria will be used to select the winner. You can select from Open rate, Total unique clicks, or Total clicks on a selected link. This will map back to how you will finally gauge the success of the email campaign, for example, if you are looking to drive visitors to your online store, you may want to select ‘Total unique clicks’ as the criteria for selecting a winner.

Finally, you can select the number of hours or days across which you want to run the A/B test. The default is to ‘Select a winner after 6 hours’, however depending how time-sensitive your campaign is, you may want to select more or less. Note: Setting a testing period of less than a few hours may impact the reliability of the test, as there may be insufficient click and open data generated to accurately determine a winner.

Once you’re done, click ‘Next’.

You will then be presented with a snapshot of the email campaign, including the two subject lines defined earlier. Review, then click ‘Test and define delivery’:

In ‘Step 5.1 – Test your campaign’, you will have the opportunity to test your campaign prior to sending it just as you would a regular campaign. Likewise for ‘Step 5.2 – Schedule campaign delivery’. It’s time to get sending!

Sending and monitoring an A/B test campaign

The excitement all happens once you’ve sent out your email campaign – and at this point, you will see the real-time presentation of results to be quite different from that of regular email campaign sends:

Not only will you be able to see how each version of your creative is performing in the test, but upon completion, you will be able to view the total benefit gained from running the test. This is an excellent way to admire your own handiwork, as well as learn how differing approaches to subject line, content and the from line can alter the results of an email campaign.

This is the first in a series of posts on A/B tests, which we hope will assist you in making your email campaigns more effective (and maybe even make testing fun). Feel free to discuss this post via the comments below.