We recently ran more than one pricing AB test on an ecommerce apparel client to gauge whether or not you should list the savings percentage next to the price of products. Although this might seem to be a “small” test, as per our usability vs. desirability framework, increasing the perceived value of a product is square in the desirability category, which, we argue in that article, typically has a greater chance of affecting conversion rate than usability tweaks (button colors, etc.). In addition, previous tests for this client showed that customers were heavily swayed by price and discounts.
We tested this on the listing page and the product detail page.
The original (“A”) for these tests looked similar to JCPenney.com:
For our client (who is also in apparel), there was also a “was” price and a “now” price — above, JCPenney happens to refer “now” as “after coupon”. Showing the current price (“now”) next to an anchor price (“was”) is a very common presentation of price in apparel ecommerce. Large department stores like Macys.com, Kohls.com, for example, also present their price this way:
But what we tested for this client was adding a simple percentage savings number in addition to the was and now prices. This is how Nordstroms.com presents their price, for example:
Note the “40% off”. That’s what we were interested in. Could that simple “40% off” make a difference?
The hypothesis for why it would help conversion rates is that it could more easily highlight the savings — the thinking being that customers aren’t likely to do math in their head, and when you have an entire listing page of products with two prices each, it’s just too much mental math to internalize. Customer may fail to realize, at a glance, which products are on the steepest discounts.
The opposite argument is that adding more numbers about price could contribute to color and confusion, which perhaps could even hurt conversion rates by making the page look messy (this is a real phenomenon) or by creating more distractions from the CTAs (calls to action).
Pricing AB Test #1: Adding Savings Percentage
We tested presenting this in 2 different colors, so it was an A/B/C test. In both variations the savings percentage was presented as “You Save #%”. In both variations, we added this savings percentage on the product detail page (PDP) and the product listing page (PLP, or “category page”).
The conversion rates to completed order for all three variations were within 0.3% of each other — amazingly close to the same. Here is a snapshot from the Optimizely results dashboard for this client (btw, if you’re curious about our experience with different platforms, here are our thoughts on Optimizely vs. VWO vs. Adobe Target):
You can see the amount of data we collected was significant (yes, this site gets a lot of traffic) — 280,000 visitors per variation for 3 variations, collected over 2 weeks. And yet the conversion rates were nearly identical.
Why did this result in “no difference”? Does this mean that ecommerce shoppers simply ignore percentage savings next to was and now prices?
We actually thought so, until we did a follow up test months later.
Pricing Display AB Test #2: Different Savings Percentages
A key aspect of the previous test, which showed no difference, was that the savings percentage was the same for all products on the site. This site has about 1000 different SKUs at one time, and all of them (except special items) had a 30% difference between the was and now prices.
The fact that adding this percentage did not change conversion rate tells us that listing the savings in a product’s price as a percentage instead of two dollar amounts, by itself doesn’t seem to do much for conversion rate. (Take the usual disclaimer in statements like these that this applies to this site in this instance, there are always exceptions in CRO).
But what we tested next was placing the savings percentage back on PLPs and PDPs during a period when the store had different pricing for different products.
In this test we did not have multiple colors, simply an A and a B variation, with and without the percent off.
This test showed a 2.57% lift in conversion rate from PLPs and PDPs with 99% statistical significance. Revenue per visitor also increased by 2.54% with 95%. This was across 700,000 visitors and 18,000 conversions.
The lift was higher on mobile than desktop. On mobile the lift was 3.61% with 99% significance and desktop only 2.22% lift with, notably, only 89% statistical significance, which by the industry convention of 95% statistical significance to declare a test “significant” would be declared “no difference”.
Nonetheless, even if the lift was for mobile only, it shows a stark difference from the first test which was very much “no difference”.
What does this tell us?
Price Presentation Is a Function of the Price of Other Products
These results tell us that price presentation — a huge needle mover in ecommerce — is not about price from single items. It’s a collective phenomenon. It’s a function of the price of all of your products. Customers view pricing of one product as a part of a collective, where relative differences matter a lot in buying psychology.
The savings percentage was clearly overlooked when it was the same number for all products. But when it changed product to product, it drew the attention of customers and perhaps drew them to certain products with steeper discounts and increased conversion rate. The fact that revenue per visitor also increased means that this was done without simply attracting customers to lower AOV products. The percentage discount mattered, not necessarily the final price.
Overall this suggests the following for ecommerce brands:
- If you have was and now prices and different savings percentages per product, definitely consider testing showing the percentage off
- In general test price presentation carefully, it can make notable differences in conversion rate but stopping after one failed test may leave revenue on the table
If you’d like to read more articles related to this or more ecommerce AB test case studies, here are some suggestions:
- Our usability vs. desirability framework for CRO and ab testing
- Our research study of mobile checkout best practices across the 40 largest U.S. ecommerce sites
- Case Study: Adding a sticky add to cart button on PDPs
- Case Study: Adding Free Shipping messaging in different locations
- Case Study: Adding a Growth Rock coined ‘Link Bar’ to improve mobile navigation
Finally, if you’d like to learn more about our ecommerce CRO agency, you can do so here.
Should you make your add to cart section sticky (that is, it stays visible when scrolling)?
In our work with dozens of ecommerce companies over the last 5 years, we’ve noticed that the design and UX of the add to cart section (buy buttons, options, quantity controls) is debated a lot.
- Should you make the add to cart button sticky?
- Should we make the entire section sticky?
- Should we hide some selections (size, color, flavor) or show all?
- Should we show all sizes as buttons or shove them in a dropdown?
- In what order should we present the different options?
This case study is about the first question.
In this article, we present results from a simple AB test where we tested having the entire add to cart area be sticky versus not sticky on desktop devices for an ecommerce site in the supplement space.
The variation with the sticky add to cart button, which stayed fixed on the right side of the page as the user scrolls showed 7.9% more completed orders with 99% statistical significance. For ecommerce brands doing $10,000,000 and more in revenue from their desktop product page traffic, a 7.9% increase in orders would be worth $790,000 per year in extra revenue.
In our experience, sticky add to cart areas on desktop product pages are less common than on mobile, so this result suggests many ecommerce brands, marketing teams, and store owners may benefit from AB testing sticky add to cart buttons or entire add to cart areas being sticky on their product detail pages.
Note: If you’d like to run AB tests like this to help increase your ecommerce site’s conversion rate or improve user experience, you can learn about working with our ecommerce CRO agency here. Or if you want to read more case studies like this
Building a Data Driven Culture: AB Tests Can Help Settle Endless Debates
When you read about AB tests online, most articles have a nice predictable pattern: (a) We had a great hypothesis. (b) we tested it (c) it worked and got a conversion lift!
But when you do enough AB testing (we run hundreds of AB tests on ecommerce sites every year), you come to learn that most tests don’t end up so neat and clean.
So instead, we urge you to think of an alternative, but also valuable use case of AB testing:
Run a test to learn about your customers even when you could make arguments for either variation being “better” and there isn’t consensus on which one is “obviously going to win” (a phrase we often hear clients use).
Let’s use this article’s example to learn why this “ab tests for learning” use case is so useful: For this sticky add to cart button example, in the traditional design process, people in the company would debate in a conference room (or by email, or in Slack), about having a sticky vs. non-sticky add to cart area on their product page.
They’d go in circles fighting with each other about which is better, which is more on brand, which competitors do something similar, and on and on.
Then, either the higher ranking person wins or an executive steps in and gives their official decision like Caesar at the Colosseum (even though 99% of the time the executive is not a UX expert).
But in reality, neither side knows which will do better. Both arguments are legitimate. And the one important contingent that does not have a seat at the table is the customer. An AB test lets your customers give their input.
And, finally, whichever variation “wins” is important but not as important as the learnings about how your customer thinks, what they prefer, and what is more persuasive to them, all of which you could learn by simply running a test.
So, with all this on our minds, we ran this test.
The Variations: Fixed vs. Sticky Add to Cart Buttons
The variations were simple. One of them (A), the buy box was not sticky and in the other (B) it was sticky. (Aside: Read why we anonymize clients here.)
In this case, the client’s product page had a sticky buy box (variation B) to begin with, and it hadn’t yet been tested. The reason we decided to test this was because there was a content-focused culture around the brand, so we felt it was important to learn how much users want to be left alone to read content versus having a more in your face request to buy following them down the page.
One can make a theoretical argument for both variations:
- Argument for variation A (not sticky): You don’t want the site to act like a pushy salesperson, hovering over your shoulder when you’re just trying to read about the product and its benefits. It will turn people off.
- Argument for variation B (sticky): People can read about the product just fine, and reminding them of the ability to add to cart will increase the percentage of people that do so.
Results: Sticky Add to Cart Button Gets More Orders by 8%
In this test, the sticky add to cart variation showed 7.9% more orders from the product pages with 99% significance. The sticky version also showed an 8.6% increase in add to carts with 99%+ significance. The test ran for 14 days and recorded approximately 2,000 conversions (orders) per variation.
Referencing the arguments for either side above, this test gave the marketing department and us valuable information about the customer (and saved a potentially conversion hurting future change of undoing the sticky add to cart area).
Despite this brand’s heavy focus on content, despite the customers’ needs to read a lot about product uses, benefits, ingredients, and more, having an ever-present add to cart area seemed to be okay with the customer. It did not annoy the customers, and in fact seems to increase the percentage of them that decided purchase. This is a useful learning, despite this test largely being in the usability category, not desirability.
(Note: This is true of this store, it may not be true of yours. Hence we always suggest at the end of case studies that you should “consider testing” this. We don’t say you should just “do” this.)
This learning can actually be extended beyond the product pages to content pages such as blog posts where we can test being more aggressive with product links and placements to see if similar results can be achieved there.
This is why we love using AB testing for learning.
Update: Sticky Add to Cart Button on Mobile Product Pages Also Increased Conversion Rate
Since first writing this case study, we tested multiple UX treatments for a sticky add to cart button on the mobile PDP of this exact same supplement ecommerce store. The smaller screen real estate on mobile devices, of course, means finding CTAs like the add to cart button can be more difficult, so making the add to cart button sticky could improve user experience.
We tested three variations: (A) No sticky add to cart button (B) Sticky add to cart button that simply scrolls the user to the add to cart area on the product page (C) Sticky add to cart button that causes the add to cart area to slide up from the bottom of the page like a hidden drawer.
As of this writing, we’ve observed a 6.5% increase in orders for variation C where the add to cart area slides up like a drawer, with 97% statistical significance. This test has been running for 12 days and also has about 2000 conversion events (orders) per variation.
This suggests that similar conversion gains can be realized on mobile product pages, but the details of how to implement them and the UI/UX that will cause a conversion increase are important. Questions? Let’s discuss in the comments.
If you have questions about whether your store should test sticky add to cart functionality, how to execute it (e.g. Shopify plugins vs. manual coding), or about working with us, you can ask us in the comments, send us an email, or learn about working with us.
We know free shipping is a massive needle mover for ecommerce customers. In this short case study we share results from two AB tests we’ve done that help answer:
Where is the best place to put your free shipping and free returns messaging to get the biggest lift in conversions?
Test 1: Free Shipping messaging placement for furniture ecommerce site increases conversion rate 19%
On the original site, free shipping and free returns was already mentioned in the promo bar at the top of the page which was visible sitewide.
We hypothesized that due to (a) banner blindness and (b) too many competing messages in the promo bar, this message was not getting across.
Where could we place this message that would be least likely to be missed and most likely to influence the buying decision?
We settled on placing it below the add to cart button on the product detail pages (PDP).
We saw a 19% increase in orders with 99.9% statistical significance. The test ran for 2 weeks and recorded over 1,500 conversions.
In many ways, this test is fascinating. In the original, the free shipping and free returns messaging is already mentioned in the promo bar, at the top of the page, sitewide.
How could customers not see this?
This result suggests there is truth to the idea that banner blindness and competing messaging hurts the effectiveness of that message.
If you offer free shipping and free returns, or have other key value propositions (like an active discount code or promotion) you should strongly consider testing where free shipping and returns messaging is placed, and certainly test adding it near your add to cart button on the PDP. Most brands from what we’ve seen either put them in promo bars (not bad) or save them for graphics on the homepage (much worse).
Test 2: Free Shipping copy for a Supplement Company Does Not Affect Conversion Rate
We tested something very similar for a niche supplement company.
In this case, we actually hypothesized it would perform better because there was no mention of free shipping on the site except in fine print. (Definitely not in a sitewide promo bar like the example above).
Just like the above test, we put free shipping copy below the add to cart button:
The only differences were:
- The copy said “Free US Shipping & Returns instead of “Free Shipping & Free Returns”
- There was a dropdown caret that had more details on the 30 day return policy. The schematic above for B (our variation) shows the caret expanded. Upon pageload it was collapsed, i.e. the box with return details was not visible.
After 2 weeks and over 5,000 conversions, we saw no difference in conversion rate between the original and variation. The conversion rates were almost identical!
For this brand we actually tried a few different placements of free shipping copy including in a promo bar and still found it made no difference on conversion rate.
Why could that be?
AB tests tell you what and you have to hypothesize as to why.
In this case it could be several reasons:
- This is a specific, niche supplement space where there are only a few providers and most provide free shipping, so it may be expected by the customer.
- This is a much lower price point than the first example (furniture) so perhaps in the first example the thought of a hefty shipping cost and hassle of returning furniture is a huge friction point that the copy helped assuage.
- The supplement brand is very content heavy, so readers may be far more sure they want to buy after reading up on the details and details like shipping cost don’t matter as much.
- Finally, the customers for the supplement brand may simply be less price sensitive due to its niche characteristic. In fact, later we did pricing tests that also showed little difference (to be profiled in a later case study).
One lesson we’ve learned over and over is that while there are UX patterns that seem to perform better across multiple ecommerce sites, there are always plenty of exceptions. So what works for one site, doesn’t always work for another. The two examples above show that.
So we encourage you to learn, take inspiration, and think critically about the case studies above and how they may apply to your store. Then, we encourage you to run your own tests before simply implementing these UX treatments on your site.
If you’d like to talk to us about improving conversion rates for your ecommerce brand, you can learn more about working with us here.
Recent AB tests we’ve done suggest that many ecommerce sites could see an increase in mobile conversion rate by adding a “bar” of navigation links at the top of their mobile homepage, instead of relying solely on the hamburger menu.
We’re calling this a “Top Nav Link Bar”, or just “Link Bar”.
The Link Bar is an alternative to the much hated “Hamburger Menu”, which hides links behind the famous 3 bars (the hamburger). It’s hated enough to where simply Googling “hamburger menu” returns anti-hamburger menu articles in the top 5 results!
In this article, we’ll discuss the Link Bar concept via two AB test case studies where we saw increases in visits to product pages and purchase conversion rate.
Finally, we’ll also show a set of design examples from popular ecommerce sites that implement a Link Bar concept in different ways.
Our hypothesis is that the Link Bar lets shoppers get to the product pages faster by exposing product and category page links normally hidden behind the hamburger menu. One less click is required and the links are more prominent, so it increases the chances of users proceeding “down funnel” and seeing products.
Let’s get to the two case studies.
Mobile navigation Link Bar increase orders by 5% for an apparel store with 1000 products
First we have an apparel client that has over 1000 products across 9 categories (and multiple subcategories on their site).
So, pretty stereotypical ecommerce company.
What did the mobile homepage look like?
Since we anonymize clients, let’s use the mobile homepage of a well known brand that had a similar layout: Urban Outfitters.
Key characteristics of this mobile homepage (that were true of our client’s mobile homepage):
- Large image based full bleed photos that change depending on the current marketing campaign (about once a month)
- Main navigation hidden inside the hamburger menu
- If you scroll down far enough there are eventually links to categories
Here’s what we tested:
In the variation, we simply added the Link Bar, to the homepage only. There were 9 categories.
Note we didn’t replace the hamburger menu, it’s still there and still is the most thorough way to navigate the different product categories.
But it’s no longer the easiest way — the Link Bar is.
The Link Bar was left-right scrollable and had arrows to help indicate that.
Here are the results.
First, completed orders. After 28 days, we saw a 5% increase with 93% statistical significance:
Note Optimizely’s stat engine uses a more rigorous “two-tailed” statistical significance calculation, which does not give this any significance, but a traditional p-value calculation shows this:
So this is not a “runaway winner” by any means. The industry convention is to declare a winner if it reaches 95% statistical significance or higher when the test reaches your pre-determined number of visitors.
But that is, in the end, a “convention”.
With over 80,000 visitors, 2,300 conversion events per variation, and having run for exactly 4 weeks with the variation leading basically for the entire test, we felt the conclusion was “this is likely a winner and is more likely to perform better by 2% – 5% over longer periods.”
But that’s just one metric (albeit an important one). The story gets more interesting if you look at additional metrics.
Only the exposed listing pages showed an increase in pageviews
Pageviews of the category pages showed clear increases by 10% – 12% (with 99%+ significance), validating one of our critical hypotheses that the Link Bar would send more users “down funnel”.
For example here is the first category link on the left of the quick Link Bar we added (e.g. the “TOPS” link in the “B” mockup above):
The other two category pages showed similar results.
But those pageview increases were only seen for the exposed category links:
What about the links that were “hidden behind the scroll” in other words, you needed to use either the arrow, or scroll to the right to reveal them?
They showed no change in pageviews:
This was consistent for all the category page links that were hidden behind the scroll.
This confirms the original hypothesis of this test: Revealing links to product and category pages will increase the amount of customers reaching them.
Certainly if category pages that were just to the right in our Link Bar didn’t see an increase in pageviews, then hiding all links behind the hamburger menu does the site no favors in terms of getting shoppers to the products.
Takeaways for your mobile site:
- Test putting links to your most popular product categories at the top of your mobile homepage.
- Try making the bar scrollable and see if you can reproduce this result in your store.
- Do you see indications of an increase in completed orders like we did? Maybe your store shows a far more definitive increase in conversion rate than the slight possible lift we saw above.
Case Study 2: Health food brand sees 29% increase with a navigate Link Bar on the homepage
Next we have a very different ecommerce brand, in the health food space with 3 product flavors.
Again, the homepage had copy and images and links but you had to scroll down the page to get links to the 3 PDPs.
So we added the navigation Link Bar just like before:
The variation in this case had links directly to the PDPs of the 3 different flavors (which each had their own PDPs).
After 14 days, we saw a 29% increase in orders with 98% significance.
Traffic to this site was lower, however, so the test got only 139 vs. 107 conversions per variation. This is low. The difference is only about 30 orders, so again we have to put a qualifier that the variation “likely” performed better.
However there was no indication that it would perform worse than not having the links.
Link Bars can help expose customers to new products
In this case, of the 3 flavors, the second and third flavor saw a large increase in PDP pageviews (Chocolate and Strawberry in the mockup above): +25% more visits, and 77% with 99.9% stat significance and over 600 conversions per variation.
But, the most popular flavor did not see much of an increase.
In this case the site was known for their most popular flavor. Historically that was the only flavor for when the brand first launched. Referral links disproportionately went there, blog links disproportionately link to that flavor, and the homepage imagery and copy mostly talked about that flavor
So in this case the Link Bar served to expose more customers to the rest of the company’s offerings.
This is a nice additional benefit of Link Bars. Note that those alternative flavors were inside the hamburger menu also, but as we saw in the first case study, having them exposed on the page (via the Link Bar) showed a definitive increase in visitors to those PDPs.
Conclusions and how to apply this to your own mobile ecommerce site
Taken together both of these tests, on two very different ecommerce stores (1000 products vs. 3 products), suggest a similar theme:
Make it as easy as possible for mobile shoppers to get to your product offerings.
If you have hundreds or thousands of products, put links as close to above the fold as possible to your most popular categories.
In the first example above, a natural iteration of the test (that has not yet been tested) would be to stack the links instead of having them be in one scrollable row.
This will give shoppers an even better overview of exactly what the store offers.
This should send even more visitors “down funnel” and perhaps give the test a more definitive win over the baseline.
If you have only a few products, create top nav links to the product detail pages.
Finally, as always, you should test this yourself. Don’t assume these results will apply to your store.
Both of the AB tests above saw definitive increases in visits to the category or product detail pages, but the increases in order rate weren’t “runaway” winners, which we define as 99%+ significance with hundreds or thousands of conversion events for each variation.
That’s okay though, as we’ve written about before, not all ecommerce stores have the luxury of that much data. That doesn’t mean you throw up your hands and not test anything nor does it mean you should just use the old fashioned method of “debate designs in a room, loudest voice wins, and implement it outright”.
That’s even more dangerous.
Aside: We once had an in house designer form a client ask if they could implement a hamburger menu on desktop because it “looked sleek”. (Facepalm)
This is why testing is important, even if you don’t get picture perfect increases in conversion rate (99% significance, and thousands of conversions over many weeks).
What about desktop? Why is this mobile only?
The reason this isn’t relevant on desktop is because almost all ecommerce sites have exposed links to all categories (and often dropdowns to subcategories, aka a “mega menu”). So this is by definition almost always already implemented on desktop.
It’s just that the space constraints of mobile result in the hamburger menu.
Hopefully this article and this data we shared helps you start to think outside of needing to collapse everything behind the hamburger menu and starts opening up other possibilities.
On that note, our variations aren’t the only way to go about this. Here are several more examples of alternatives to the hamburger menu from different ecommerce mobile sites.
Ecommerce mobile homepage examples
Who is doing this well already?
Here are some other brands that have clear links at the top of the mobile homepage, getting rid of the complete dependence on the hamburger menu:
Gap has a lot of products and categories. They have clear links to the main categories at the top of their mobile homepage:
The use of photos is a nice touch and could possibly increase engagement with the links and clarity for certain stores.
Note the links are not sticky upon scroll, whereas the links to Gap Incs other brands at the top of the page are. Interesting.
Also in the apparel world, Abercrombie chooses to simply split by Men and Women. This is worth testing versus a deeper category split like Gaps above:
For search heavy stores, Lowes.com has a great example of both featuring search and using a suggested area to basically push some category links. We hadn’t been to Lowes.com on this device before so these were likely just categories they wanted to promote (versus a personalized list based on past visits).
Finally, here’s a more bold homepage concept by Cos Clothing, who doesn’t need a thin strip of suggested categories but rather just dedicates the bulk of the homepage to sending shoppers to the right categories.
(Note by the time we published this article the Cosclothing homepage had changed to include a promotion at the top instead of full bleed photos linking to women and men’s departments.)
They have full bleed images for women and men followed by clear links.
We would love to test something this bold with one of our clients.
Final Aside: the homepage is often sacred ground for ecommerce organizations. People fight and negotiate over screen real estate there. So even we, as a third party optimization agency, often have severe restrictions on testing the homepage, much less radically redesigning it. Much thanks to the two clients who let us run the tests featured above.
Not presenting related items at the shopping cart step could be costing many ecommerce stores millions in potential revenue.
In particular I’ve noticed while large, well known brands do this consistently (see examples below), mid-size ecommerce stores often don’t, and that’s likely a mistake.
In this article, we’ll show data from two AB tests where we added a one-click upsells and cross sells.
The first increased average order value (AOV) by $55 (worth millions in annual revenue).
The second increased conversion rate by 13%, which for any 8 figure or greater ecommerce store is also worth 7 figures in extra annual revenue.
Finally, we’ll also show (and analyze) 5 live examples from well known brands of upselling and cross selling related products at the cart stage.
This way we hope you can find an upsell implementation that works for you.
Note: We are a conversion optimization agency exclusively focused on ecommerce. Want our conversion and UX experts to evaluate your upsells or optimize your conversion rate? Learn more about what we do on the homepage or contact us via the red button at the top.
How do upsells and cross selling work in ecommerce?
Some people have all sorts of specific definitions of “upsell”, “cross sale”, and “downsell”.
Quickly, for our purposes, I prefer to use the more general definitions of upselling and cross-selling, which just mean you’re trying to get the customer to increase their order value by presenting additional items they might want.
It may be a more expensive item (upsell). or some add on items (cross sell). But here’s the most common type in ecommerce (discussed in more detail below):
Once you add to cart, Gap is showing 4 additional items I can consider. We’ll discuss the implementation details below (for example here you need to click into each product detail page (PDP), you can’t just add those items to cart) but that’s the idea.
For now, let’s talk strategy.
As the two case studies in this article below show, upsells and cross sells can either:
- Increase AOV
- Increase conversion rate
(If you’re curious how an upsell can increase conversion rate scroll down the second example.)
Let’s start with an AB test that does the former.
Upsells that increase AOV: $2 million/year extra revenue for an online furniture store
Our first example is from an online furniture store. Let’s say in this case that they sell sofas ranging from $850 to $2000+ with an AOV of $1200.
Their most popular sofas are leather, and what’s interesting in this case study is not the sale of the leather sofas, but of a particular upsell: a leather conditioning kit that helps protect the sofa, and costs between $40 – $80.
Something like this:
The conditioning kit is a perfect cross sell for a customer buying a leather sofa. It actively protects and lengthens the life of the thousand dollar or more purchase the customer is already making.
If you’re already spending $1500 on a leather sofa, why not pay $60 to protect it and make it last longer.
But these complimentary accessories were not easy to navigate to on the site at the time of this test. They weren’t promoted heavily.
So we hypothesized that mentioning it as an option at the cart step, and making it very easy to add to cart, would increase AOV.
Building our AB test from the hypothesis
You can turn a hypothesis into an actual UI/UX treatment in many different ways and this step is critical. Our hypothesis was:
Offering a leather conditioning kit as a one click upsell when a customer adds a sofa to cart will increase AOV and thus total revenue.
But how should we actually offer the leather conditioner in the cart?
With a photo?
As a one line item?
Do we add some copy to really “sell” it or keep it low key?
Will any of these decisions possibly hurt sofa conversions itself?
We opted to start low key because we felt that the change of going from not mentioning that leather conditioner at all to mentioning it was a big enough change.
Our variation design:
The pink strip is what we added.
We coded the plus icon to add the conditioning kit to cart on click. If the customer clicked the name of the conditioning kit instead, it took them to its product detail page (PDP).
Typically we run tests for around 2 – 4 weeks, but we ran this test for 41 days (nearly 6 weeks)! Why so long?
Because what we were looking for here was change in AOV, but, the current AOV was above $1000, and the leather conditioning kit costs between $42 and $84.
So we were trying to detect a pretty small change.
After 41 days, over 4000 transactions and $5,600,000 revenue tracked, and AOV increased by $55, with 92% statistical significance.
The AOV increase held steady for the last 4 weeks of the test with statistical significance sitting in the 90% – 95% range the entire time.
Here is a plot of quantity sold per week of the upsell’s product SKU in Google Analytics’ ecommerce report:
Previously they were selling around 40 – 80 conditioning kits per week. Once we turned on the test (which means only 50% of users saw the variation), sales jumped immediately to 150 – 180 per week.
In fact, the warehouse ran out of leather conditioning kits when we turned this test to 100% of traffic and we had to turn it off temporarily until they could order more.
This increase in AOV, on average, was worth an extra $180,000 per month in revenue (that’s over $2,000,000 of extra revenue per year!).
Takeaways for your site
Ask yourself: Are there complimentary, lower priced products that pair with your main product(s) really well?
Walk through the typical buying and checking out funnel.
- Is it obvious to customers that these products exist? It should be.
- Is it easy for them to add them to cart? It should be.
- Does the copy position them in a way that makes it clear they compliment the primary products? It should.
Upsells that increase conversion rate: 13% increase in orders for a health food store
This second case study surprised even us when it happened.
Building on the success of upsell tests like the one above, we decided to test something similar for a online health food brand that sold nutrition bars (same disclaimer).
The key difference from the example above though is this: They only sold that one product in 3 different flavors.
That’s it. There were no other products. All 3 flavors had the same price point.
So how do you offer an upsell when you largely just have one product in 3 flavors?
It’s not the case that customers didn’t know about the other products: On the homepage, all 3 products were mentioned. In the navbar, all 3 products were mentioned. Even on each PDP the other 2 flavors were mentioned.
What we decided to do is this: When a customer adds a product to cart and an add-to-cart “drawer” slides in, we decided to offer a single “pack” of one of the other flavors at a discount (A below).
Packs typically cost around $8, but customers can only buy 2, 6, 12, or 18 packs (AOV for this site was around $57).
So when a customer added one of these to cart, our upsell offered a single pack of another flavor for $6 (B and C). That’s it, you can only add 1 of the alternate flavor, but you get a slight discount on it.
We tested two variations that were functionally similar but had slightly different designs (white border versus colored background).
Results: No change in AOV but an increase in conversion rate
Our hope for this test was that this would get more customers thinking about adding multiple flavors to their cart and thus increase AOV. In other words that they wouldn’t just stop at the single pack but decide “Well let me also add more of the other flavors”.
That didn’t happen.
But what did happen was positive. We simply saw an increase in orders (“conversion rate”) on the site as a whole.
Specifically we saw a 13.4% increase with 95% significance with over 1,000 conversion events (orders).
We tested two variations over 2 weeks with a slight design difference and both showed the 13% increase over the original (no cross sell) with 95% significance.
Why did a cross-sell increase conversion rate?
Why did adding a single nutrition bar of an alternate flavor increase conversion rate?
Our hypothesis is that customers simply wanted to take advantage of the “deal” on the alternate flavor. They get to the site, browse the flavors, pick a flavor, add it to cart.
Stop and think about what your mindset, as a customer, would be at that exact moment: A part of you will have a slight doubt about your flavor choice:
“Hmm, maybe that other flavor was better?”
“I wonder what that would taste like?”
“Should I go through with it and buy this?”
In our variation, at that moment, customers saw a small $6 discount offer on one of the other flavors .
We think for some fraction of customers this was enough to push them over the edge to buy.
Basically, the cross sell acted as a discount or add-on special offer that encouraged more purchases of the main product.
Takeaways for your site
If you don’t have upsells like the first example that compliment the main product and could possibly increase AOV, can you instead offer a similar product at a slight discount?
Are there multiple flavors or varieties of your product that customers likely debate about choosing?
Can you offer one of the other flavors at checkout at a slight discount?
Upsell and Cross sell examples in Ecommerce
Finally, for inspiration, here are a X upsell/cross sell examples from well known (and sometimes well optimized) ecommerce brands (in the U.S.).
Under Armour: Customers Also Bought
The most common types of upsells are in large SKU stores (in particular apparel) where they suggest other similar products when you add one to cart:
If you’re not testing something like this and you have a store with many products (over 20), you should test it immediately. Start without fancy algorithms and just put your most popular items there.
When testing these, try testing these implementation and UI details…
Test different algorithms or logic for suggesting products. This many not be easy to do with front end AB testing alone, but can be done. If you’re curious how, contact us to discuss.
Test number of items presented. Try minimizing carousels like what Under Armour does and show 4, or even 6 items if space allows.
Test showing and not showing product prices or even titles. You may be thinking “What?!” but this has precedence.
Gap: No Prices in Suggested Products
Bare Necessities: No Prices or Product Names
In general our experience is that showing more products and less carousel arrows is better. Requiring clicks will reduce the number of users seeing products.
Wayfair: Accessories Upsells
Exactly like the first case study at the beginning, Wayfair shows very complimentary accessories to large furniture items added to cart:
Try testing these UI and implementation details:
Test the number of upsell items. In our first client example at the beginning, we followed up the test profiled in this article with many other tests that also presented other upsells to the couch. They didn’t make much of a difference. Nothing beat just suggesting the conditioning kit as a single upsell.
Test different add to cart functionality. Above Wayfair lets you choose color and quantity. In a test not profiled here, allowing multiple quantities did worse than simply suggesting adding a single quantity in one click of a particular upsell. But this is very store dependent so should be tested
Harry’s Razors: Modal for Details
Similar to Wayfair, but at much smaller price points, when I add a razor, I get suggestions for sensible, complimentary products I can add.
A click on the plus sign, doesn’t add the balm, however, it pops open a modal where I can choose details:
This is an interesting choice. I’d be curious if they have tested this versus just a one click add to cart that defaults to Quantity 1 and the most popular size.
Especially for products like shave balm where customers aren’t expecting to be able to choose a size (unlike say, jeans) and it’s not obvious why someone would want to add more than 1, I would think this is a must test issue.
Want us to evaluate your upsells or optimize your conversion rate? Contact us on the homepage or via the red button at the top.
eCommerce Copywriting Case Study: 3 Advanced Tactics We Used to Increase Product Page Conversion Rates 13.9%
We partnered with copywriter Brian Speronello of Accelerated Conversions on this project. This case study was written by Brian and edited by Devesh and Brian.
Writing powerful eCommerce copy that improves conversion rates is one of the most effective ways to generate more revenue for your online business. Just by changing the words your visitors read when they come to your site, you can generate significant increases in sales for your company.
For example, online direct-to-consumer mattress brand Amerisleep was able to achieve a multi-million dollar increase in annual revenues just by rewriting their product details page. A/B testing showed that checkouts increased by 13.9% with 98% confidence.
Note: Our agreement with Amerisleep prevents us from publishing their actual product page traffic and conversion rates but for the integrity of the test we’re showing the confidence level and hundreds of conversion events per variation. The test was also run for multiple buying cycles.
GrowthRock performed the conversion testing for this project, and Brian Speronello from Accelerated Conversions (that’s me) wrote the copy.
In this eCommerce case study, I’m going to share three advanced copywriting techniques that helped drive the 13.9% increase in orders on the new product page for Amerisleep.
eCommerce Conversion Copy Tactic 1: Go Beyond Bush-League Benefits
If you’re reading this post, I’m going to assume that you’re at least moderately interested in conversion and copywriting strategy. You should already know why it’s important to promote benefits ahead of features.
(If not, before you invest time going over this case study, you should read more about the basics of copywriting. That way you’ll be able to get the most out of the strategies presented here).
This section is NOT about benefits versus features.
It’s pretty easy to tell the difference between a benefit and a feature. The former describes a result for the reader. The latter describes a function of the product or service.
Detecting Bush-League Benefits is more difficult, because it’s a matter of tone and perspective.
Bush-League Benefits are actual benefits that the target audience will get from your product or service. But they’re benefits that the end user doesn’t care about.
Effective copy doesn’t just focus on benefits before features. Sometimes it has to go through several layers of benefits to reach the deepest desires of the audience.
As an example, let’s look at a classic marketing and copywriting analogy: the electric drill.
A New Version of a Classic Marketing Tale
As the old saying about marketing a drill goes, the benefit of a drill is a hole in the wall. That’s the immediate result (benefit) that the user gets from the features the drill has. If you wrote copy for a drill that promoted how it puts holes in your wall, you would technically be promoting its benefits, since you’re not talking about features like torque or battery life.
But is “a hole in the wall” really what the reader wants?
Does a drill customer wake up in the morning and say “man, I really need some holes in my wall?” (And if so, why not just buy a few kegs and invite over the local frat? Problem solved.)
So while “a hole in the wall” is certainly a benefit of owning a drill, it’s a Bush-League Benefit because it doesn’t tap into the real desires of the audience. It doesn’t connect with a problem or desire that the audience actively thinks about on a regular basis.
So what’s the real benefit of having the drill?
Is it having holes that are a precise size? Is it the drill’s ability to make holes in a wall with little physical effort? To find out, you have to ask yourself why the customer would want the holes in the wall.
You’d probably come up with an answer like “to hang family portraits.” This response is far more emotional than “holes in the wall.” And it makes for a more compelling benefit to the reader.
For example, you can imagine a family moving into a new home and dreaming about hanging their family portraits above the fireplace. That’s how you know you’ve gone deeper than a Bush-League Benefit. From there you could write your copy targeting “hanging family portraits” as the real benefit of your drill.
But you’d be wrong. (Sort of…)
Because while the benefit of “hanging your family portraits” is definitely more emotional and compelling to the reader than holes in their wall, you can still go deeper.
What would happen if you asked “why?” again, researching the reason a family would want to hang their family portraits? They might say something like “We just moved into a new house, and we want to make it feel like home.”
Now you’re getting somewhere.
Which drill would you rather buy?
- Drill A: Our high-powered electric drill will put precision-sized holes in your walls.
- Drill B: A new drill from our company will let you hang your family portraits so your new house finally feels like home.
No contest, you’d take Drill B. Even though Drill A mentions a benefit (holes in your walls) it’s a Bush-League Benefit because it doesn’t connect to the emotions and desires of the audience.
The Challenge with Bush-League Benefits
The challenge with going beyond Bush-League Benefits is deciding when you’ve discovered a benefit that connects with the reader’s emotions enough to make them want to purchase. It’s also why I said Bush-League Benefits are a matter of tone and perspective, so you’d only be “sort of” wrong to use “hanging family portraits” as the main benefit in copy for the drill.
Maybe your audience wants their house to feel like home because they’re part of an elite social circle and care about their status or impressing their neighbors. Or maybe it’s because they’re looking to be featured in Better Homes & Gardens magazine or on HGTV.
Those desires are even more compelling than “making the house feel like home.” You could write your copy to focus on them, if you found they applied to the majority of your audience, and probably get even better conversions.
So it’s up to you to decide how deep you need to go based on your product or service and your audience. As a rule of thumb, the more expensive, complicated, or important your offer is for the reader, the more your copy needs to connect with their deepest hopes and fears.
But if you want your copy to convert well, it must connect the benefits of your product or service with needs and desires that are emotional and top-of-mind for the reader.
You can’t settle for Bush-League Benefits and expect to get a lot of buyers.
Going Beyond Bush-League Benefits with Amerisleep
If you asked novice copywriters to tell you the benefit of a new mattress, they’d probably say “getting more sleep.” And they’d be getting tricked by a Bush-League Benefit.
Don’t get me wrong, sleep is great. And because most people understand that, copy that uses sleep as the main benefit can get decent results all by itself.
However even though sleep can be an effective benefit on its own — in fact because it’s so easy to settle for using sleep as your main benefit — it’s actually a Bush-League Benefit that has tricked many copywriters. That’s because probing further into the reader’s desire for sleep reveals even more powerful motivators.
One of the biggest changes that we made on the Amerisleep product page was adding the following section. It’s short, but it plays a key role in connecting the Bush-League Benefit of “more sleep” with deeper aspirations of the audience. It relates “more sleep” with two highly emotional and nearly universal desires: An improvement in physical health and better mental performance at work.
This section positions sleep as the solution to better health, better job performance, and a higher quality of life. And it’s a mistake to trust the reader to make these connection on their own.
You have to remind readers about the important role sleep plays in how we feel each and every day before they start to care about getting more of it.
Only once the reader is actively craving sleep will “more sleep” move from a Bush-League Benefit to a real one. This section helps create that desire for sleep in the reader, and in turn it makes the other parts of the page where we talk about increasing sleep more persuasive.
So when it comes to your own copy, make sure that you connect the immediate benefit of your product or service to deeper and more compelling desires the audience has.
Otherwise you’ll just be peddling Bush-League Benefits.
eCommerce Copy Tactic 2: How to Shut Down the One Competitor Every Business Has in Common
There is one competitor that every single business in the entire world has in common. One competitor who is taking money out of the pockets of every company on the planet.
And my guess is that this competitor is not on your radar, even if you spend a massive amount of your time and effort analyzing your place in the market.
Can you name it?
It’s the status quo. Otherwise known as inaction or doing nothing.
If you analyzed the behavior of prospects who ultimately do not buy from you, which of the following situations do you think is more common?
- They spend their money with one of your competitors instead of you.
- They decide to save their money and not to buy from anyone.
Let’s say you run a website and you convert 10% of your visitors to customers. That’s an outstanding conversion rate, and yet 90% of your visitors still do not purchase.
Does that mean 90% of your visitors wind up buying from your competitors?
Obviously not! Otherwise their own conversion rates would be off the charts.
The market of people who explore their options but ultimately decide to do nothing is much larger than the market of people that choose your competition over you.
Rather than spending the majority of your time butting heads with your competitors, you should spend more of your time working to convert prospects who would otherwise do nothing at all. Not only is this a bigger market, but because your competition is likely focusing on trying to beat you, it’s also relatively uncontested.
Copy Techniques for Overcoming Inaction
No matter who you are or what you sell, your clients and customers are always going to prefer doing nothing to buying from you. Doing nothing is cheaper, easier, less uncertain, less mentally demanding, less socially and politically risky, and less stressful.
The only way to compete with the status quo is to vividly show the reader how doing nothing will lead to him or her being worse off. This is usually a great place to leverage loss aversion, where you show readers what they will lose or miss if they don’t take action. Another option is to ask the reader a question that follows the general outline “If you don’t do this, what will you do that is better?”
Here are some hypothetical examples from a cross-section of industries that show how you could use loss aversion and questions about alternative choices to make readers aware that doing nothing is actually worse for them:
- Appliances: If you don’t upgrade to a new, energy-efficient washing machine, it will cost you up to $1,000 more every year on your water and electricity bill. (Notice I didn’t say our energy-efficient washing machine. This is any of them. We’ll get to your company’s specific offer later.)
- Conferences: Last year people who attended our conference on average were able to add $300,000 to their bottom line by the end of the year using the techniques we teach. It’s your choice if you decide to join us or not — we’re not going to give you the hard sell. But if you don’t attend, do you have a better plan for adding $300,000 to your profits in the next 12 months?
- Dating Coach: I know the idea of working with a dating coach can make some people feel embarrassed — after all, shouldn’t we just naturally know what to do? The hard truth is that it’s not natural, and the social systems that used to guide us have gone away. So ask yourself what’s more embarrassing, working with someone to help you improve an important (maybe the most important) part of your life? Or doing nothing and continuing to get rejected by the people you’re attracted to and waking up next to an empty pillow every morning?
Overcoming Inaction with Amerisleep’s Copy
We applied this principle on the Amerisleep product page in the same segment of copy from the last example. We talked about the benefits of sleep, waking up without pain, not feeling groggy at work, and an overall better lifestyle. Then we said:
The number of daily problems that go away with a good night’s sleep is astounding — but that only happens if you actually do something to improve your sleep.
If you just keep things the same, you’ll keep getting the same disappointing results. And don’t you deserve better?
By clearly addressing how doing nothing makes readers worse off, it takes away a lot of the mental excuses they can use to justify maintaining the status quo.
It’s only after you’ve convinced readers that they need to take action to address a problem or desire they have — period — that positioning yourself as the best choice among your competitors pays off.
How to do that effectively is what we’ll be discussing next.
eCommerce Copy Tactic 3: Position Yourself as the Market Leader Using Comparisons
Once your prospects decide to take action on a problem or desire they have, their mindset changes. Before that, their number one question is “Should I even worry about this right now?”
After your prospects make up their mind to move forward though, they begin to ask “Who should I hire/How should I solve this problem?” instead.
When your prospects reach this stage in the decision-making process, it finally becomes effective for you to spend time proving how your offer is superior to your competition.
Here are a few techniques for making prospects see your company as their best choice, even if you operate in a crowded market.
Absolute Comparative Statements
Absolute comparative statements say that one thing is absolutely better than the other. Claims like better, bigger, and the best are absolute comparatives.
(If you’re wondering “doesn’t that make all comparative statements absolute?” I’ll show you how that’s not true in the next section on Faux Comparatives.)
The biggest mistake copywriters make when using comparatives is not being specific. If you’re going to use a comparative statement, you need to explain compared to what.
Too many companies will say “Our product is the best!” But that doesn’t answer the question “the best compared to what?” And without the “compared to what” piece, the comparison is ineffective.
Here are two examples of absolute comparatives from the Amerisleep product display page. I’ve bolded the comparative term and underlined the “compared to what” part of the sentence.
- On top of that, our foam is also the most environmentally friendly. Our patented foam-making process uses plants to replace some petroleum, and is the only manufacturing method that exceeds the standards of the Clean Air Act.
- Our foam is also better than traditional memory foam because it recovers its shape faster.
Absolute comparative statements are the most powerful way to make your offer appear superior than the competition. But what if you can’t say your product or service is absolutely better than the competition in any measurable way?
In that case, some copy trickery can help you give the impression you’re better than your competition, when really you’re only claiming to be equal.
I call these copy tricks “Faux Comparatives.”
How “Faux Comparatives” Let You Turn Equality into Superiority
Tell me what this sentence means:
- No other mattress is more supportive than Amerisleep’s Revere bed.
Most people would believe it says the Amerisleep Revere bed is more supportive than any other mattress…
But in reality it only says that no one is better — which means there could be many others that are equal.
Take another look at the sentence, this time with the implied meaning in parenthesis:
- No other mattress is more supportive than Amerisleep’s Revere bed (but there are several others that are equally supportive).
If you’re in a market where the offers are all relatively equal, you can claim that no one is better and be factually accurate. No one can say you’re making false advertising claims.
If the audience interprets “no one is better” to mean that “we are the best,” that’s their misunderstanding. Your job is to make the case for your product or service in the most compelling — and truthful — manner possible. If that misunderstanding works out in your favor, lucky you.
In some respects all marketing and copywriting is presenting the truth in the most flattering light. As long as the statements you make are true, choosing the words that make your copy the most convincing to the audience is just doing your job as a writer.
Here’s an example of how we used this Faux Comparative strategy on the Amerisleep product display page in the headline before describing the materials inside each mattress:
This headline suggests that Amerisleep’s mattresses are more carefully engineered than any other brand. It’s a great message to introduce before explaining what goes inside each mattress. However in reality all it says is that, while there aren’t any mattresses more carefully engineered, there may be others that are of equal quality.
Another Faux Comparative method that gives the impression of superiority, when you are actually only claiming equality, is adding “one of” to an otherwise absolute comparison.
Instead of saying that you are the best, the most, or the biggest, you can say you are one of the best, the most, or the biggest in your category.
Like with the last Faux Comparative, the reader will focus on the comparison that you are the best, most, or biggest in your category. They will think you are one of a select few who are at the front of the market, but in reality you could be one of many who are all equal.
We used this approach in the sub-headline for the copy from the previous Faux Comparative:
- Our innovative and proprietary materials let us build the one of most comfortable mattresses ever
We also used it in the eco-friendly section:
In both cases, it makes Amerisleep seem like they’re at the top of their market. But in reality, it could actually mean they are just one of many companies with similar characteristics.
Increasing Conversions from Your eCommerce Copy
You’ve just learned three eCommerce copy techniques that can increase your sales by double-digit percentages. But your conversions only go up if you invest the time to rewrite your copy. If you simply move on with your day now, nothing changes and your sales will stay the same.
So before you close this page, set aside time in your calendar or add an item to your to-do list to incorporate the three copy strategies from this article. Because if you’re not going to do anything with these ideas, why did you spend 15 minutes reading this case study? Without applying the tactics it covered, the all you just did was 15 minutes of mental masturbation.
(And if this section feels familiar…remember what I said about your biggest competitor being inaction?)
You can use the following checklist to apply the lessons from this case study:
- Highlight the primary benefits in your copy. Ask if they are emotionally engaging. If not, come up with benefits that your audience relates to more deeply — and if you can ask actual customers for feedback, even better.
- Add a section to your copy that discusses the costs, risks, and problems that your prospects will experience if they don’t buy from you.
- Find comparison terms, like more, better, and the best. Make sure that each one includes a “compared to what?” part of the statement.
- Where possible, add “no one is more…” and “one of the…” Faux Comparative statements so readers think aspects of your business that are only equal to the competition are actually superior.
But what if you’re legitimately focused on other priorities and can’t make the time to rewrite your copy — even though you want to? Or what if you want to go beyond these three tactics and get the maximum increase in your site’s sales and conversion rate?
Ecommerce Marketing Case Study: How an Online Shoe Retailer Retargets Customers for Less than $0.03 per Click with Email
Today I’m going to show you an ecommerce marketing case study of how a French online shoe retailer used product bonuses to retarget visitors for one one-hundredth the cost of Google or Facebook retargeting.
Retargeting by email also gets him click through rates in the 14%+ range — a massive improvement from typical PPC retargeting click through rates.
He did this without spending hours on social media.
And he did this without fancy web design tricks.
He also didn’t have to spend tons of time creating new content — he just used what he already had.
Let’s meet him. (more…)
What if you could increase free trial sign ups…by a lot…by eliminating the word “free” from your landing page?
Well, I have a case study where we did just that.
We increased clicks by 139% and free trial sign ups by 73% by making a few simple changes to a SaaS home page.
The company is Wedbuddy.com. It lets users design a great looking wedding website really easily. It’s a cool company with a cool product. And they wanted more visitors to sign up for a free trial.
They already had a great looking homepage (screenshots below), so the question was: what changes could get more people to sign up instead of bouncing?
You would think emphasizing a no-risk, no-credit-card-required free trial would help.
Or you would think emphasizing all the cool features would help.
And you would certainly think adding more testimonials would help.
Alas, in this case, you’d be wrong. (more…)