Alinc Technologies – Cart Test – 2015 Gold Winner
WhichTestWon's Analysis: (Click back to see versions A & B)
Difference between two:
A: Price-Drop notification functionality (represented by a small, green dollar icon immediately next to the item price) active on every item detail page, by default. This feature allowed visitors to submit their e-mail in order to be notified of price drops on the item.
B: Same exact page design, but with the price-drop notification and email functionality feature removed.
Background: The price drop notification link was examined to see if it was negatively impacting sales. Prior to the A/B test, not many e-mails were submitted through this feature, so the tool already showed little effect on direct customer engagement.
Hypothesis: The price drop notification icon negatively affects viewers because it insinuates the price will drop after the viewer has already purchased the product. It, therefore, creates a sense of hesitation and potentially prevents the sale.
Objective: Increase conversions of the “Add To Cart” button, leading to a sale.
Winning version, (B) with no price drop notification lifted conversions 25.6%.
But, Klinke Marketing didn’t just stop here. To be certain about implementing the change, they checked possible variations, based on traffic source. There were none. The numbers matched the overall lift.
They then segmented by visitor type, new versus returning. For Returning Visitors, the version with no price drop notification created a strong a 74.9% lift in conversions. Interestingly, however, for new visitors, this version decreased conversions 6.6%. By investigating segments, the site is now tweaked based on visitor type. The price drop notification icon is shown to new users and removed for returning visitors.
Conducted for: Alinc Technologies is a full-service hardware vendor, specializing in custom HP server configurations, IT solutions, and HP parts and services.
Conducted by: Klinke Marketing, an Atlanta-based boutique marketing agency that focuses on a unified approach to paid search and landing page experience.
-Traffic: Split 50/50; sources included direct, organic search, and paid search
-Run with Optimizely, integrated with Google Analytics
Why we liked this test:
In addition to a strong lift from a simple change, we really liked how this test segmented results by user type and didn’t slap on a one-size-fits-all users approach. This test showed that just because features are available and turned on by default, we should not always accept them at face value.