Results for:

Without or with the newsletter sign-up overlay?

WhichTestWon's Analysis: (Click back to see versions A & B)

– Case Study –


Butlin’s, the popular UK-based family resorts site, teamed up with RedEye, a CRO and marketing automation firm, to conduct this enticing email capture test.


The goal was to determine if an adding a signup overlay would increase newsletter sign-ups, without disrupting conversions.

To minimize interruptions, the team tested the effect of creating an email capture overlay that appeared to returning mobile visitors who spent at least five seconds on site.

The overlay was a three-step signup process and offered a discount code as incentive.

The 3-step overlay appeared like this:

In step #1, visitors were asked to provide their email address. In step #2, the email address field remained auto-populated and the user was prompted to input other contact details. In step #3, the user was given a discount code.

In the version without the overlay, the only way visitors could sign-up was by scrolling to the bottom footer of the page, as show here:

Test Details:

To test the effect of the overlay, over 180 thousand returning mobile visitors saw either the version with or without the overlay. Traffic was split 50/50.

The test ran on Optmizely for two weeks.


The team suspected showing returning visitors a prominent overlay, and offering a promotional code for 10% off the next booking, would greatly increase the number of users signing up for the newsletter.


Winner: Version B – with the overlay  resulted in an astounding 2,618% lift in newsletter sign-ups, compared to the version without the overlay.

You read that right: 2,618% uplift!

Even though the overlay could have been seen as disruptive, it did not affect booking conversions; bookings were the same across both tested versions. 


Using an overlay – even if it’s somewhat intrusive – can be a great way to call attention to important details you want your users to focus on.

As we outlined in our recently published eBook, The Ultimate Guide to Overlays, overlays are almost always an effective strategy for increasing sign-up conversions.

However, there are typically three components that impact the success of an overlay campaign:

  1. Timing – the overlay must be optimally timed to appeal to users at the highest point of engagement.
  2. Placement – The overlay should be optimally placed to capture the most eyeballs on the page.
  3. Incentive – The offer given should motivate users to provide their contact info in exchange for a something perceived as valuable, like a discount code.

In this case, the overlay achieved all three of these objectives.

Because it appeared after 5 seconds, it was optimally timed to appeal to viewers who were engaged with the site and ready to convert.

And, because it was placed above the fold, it was very prominent. Users didn’t need to scroll down to see it. In contrast, in the original version — without the overlay — users had to scroll and go looking for the signup form.

As well, the offer was enticing because users were given a 10% discount code towards their next purchase.

3-Step Process

But, beyond these three tactics, the campaign may have been so successful — achieving an incredible 2,618% lift — because the signup process was broken down into small steps that kept the user engaged and committed.

Since the initial step was only one field, it seemed very simple to fill out. Users only had to decide whether or not they wanted to give their email address.

Those who decided the discount code was worth their email address were then confronted with more required information in step 2. However, at this stage, the user was already invested and committed to the process.

By step 3, the user was rewarded for their efforts with a discount code, ending the transaction on a positive note.

This test shows that, in addition to optimizing the timing, placement, and offer, a highly successful overlay campaign also captures users’ through a seemingly simple stepped sign-up process.

However, it’s important to note, despite the best overlay campaigns, your search engine rankings may be penalized if you use intrusive Interstitial pop-ups that block the mobile browsing experience.

So the end take away is: if you want users to sign-up for your newsletter, make the request obvious, but without being too in their face, aggressive, or disruptive.

Actionable Takeaways:

  1. Delineate the steps. The success of your sign-up campaign may lie in breaking down the process into smaller steps that guide the user along, keeping them engaged and committed with a carrot at the end.
  2. Use overlays wisely. Well there’s plenty of evidence to show overlays work well, avoid presenting your overlays in an aggressive, disruptive manner.

Tell Us Your Thoughts:

When do you use overlays and what tactics do you find work best with them?

Let us know by tweeting @BEHAVEdotorg.

Got a great test you’d like to see featured on the next Test of the Week? Send it on in!

Winning Version


Reader Guesses:

Which Test Won?

  • Version B
  • Version A
Loading ... Loading ...