Growth Rock Database of Ecommerce A/B Tests

Growth Rock Database of Ecommerce A/B Tests

Below is a running list of all ecommerce AB tests we’ve run for clients for the past 7 years. You can read more about why we created this below. We update this database monthly.

Tests are organized by sections on an ecommerce site (Home, Nav, Listing, PDP, Checkout, Blog) and labeled with a mix of the 8 Purposes (Brand, Discovery, Product Appeal, Product Detail, Price & Value, Usability, Quantity, Scarcity) of every test as per our Purposes Framework for Ecommerce CRO.

Jump to read more about this database. Learn more about working with us here.

2020-12-14 18:35:13

2020-12-14 18:39:37

2020-12-14 18:40:57

2024-08-22 23:17:09

2020-12-14 18:51:33

2020-08-17 16:44:38

Home
Homepage Products and Recommendations
Position of personalized product recommendation container (higher vs. lower on the page) Discovery
Swipeable carousel of recommended products vs a 2X2 Grid (Mobile only) Usability, Discovery
Show vs Hiding product discounts and removing separate product benefits section Price & Value, Product Appeal
Showing product bundles only for new visitors to see if it affects new visitor purchase behavior Discovery
Show vs Hiding a quiz to help customers find the right product Discovery
Show all products in a 4 column grid (like a listing page) directly on the homepage Discovery, Usability
Adding links to product categories in the middle of the homepage so categories aren’t only hidden inside hamburger menu Discovery
Popups
Hiding vs. showing an email capture popup on the homepage Usability
Redesign disclaimer popup with icons instead of paragraphs of text Usability
Homepage Hero Unit
Add product hotspots on hero unit creative to let customers easily buy products models are wearing Discovery
Using two types of animation via JS vs. CSS to measure page load speed Usability
Casual vs professional headline copy to catch user’s attention Brand
Testing different hero unit product photos Product Detail, Product Appeal
Adding featured customer reviews in different parts of the homepage Brand
If user has items in the cart, show the items photo in homepage hero unit and mention they are not done checking out Usability
Auto-playing vs pausing promotional video in the hero unit Usability, Brand
Displaying different copy depending on how user found website (ads, search engine, etc) Brand, Usability
Testing link vs. button styling on “shop now” CTAs in homepage hero unit Usability
Show vs Hiding email newsletter signup form within hero unit Usability
Promoting products in the hero unit vs. content and an email signup form Discovery, Usability
Homepage Content and Design
Testing a completely different upgraded homepage design Brand
Design of live chat icon – circular vs rectangular Support, Usability
Show vs Hiding customer support phone call option Support, Usability
Effect of additional link to “Sale” page Usability
Simple vs. longer appointment booking form Usability
Shorter vs longer homepage with more information, video, and testimonials Brand
Homepage with only products vs more information and reviews Discovery, Product Appeal
Testing the effect of Lighthouse support icon on conversion rate Support, Usability
Sending add to cart button clicks on homepage to cart vs. to the PDP Discovery
Significantly reduce copy and move call-to-action higher on page Usability
Placing newer products higher on the homepage Discovery
Homepage Social Proof
Testing effect of quotes from publications with logos Brand
Adding images of celebrities who use the products for social proof Brand
Show vs Hiding “Reseller Ratings” to increase customer trust Brand
Add more UX emphasis (brighter, larger) to a charitable/social good aspect of the brand Brand

Navigation, Headers, Footers
Desktop Navigation
Sticky vs. fixed desktop header navigation Discovery
Redesigning a header navigation (desktop) Discovery
“Sale” vs “Clearance” copy and color emphasis on that nav link Discovery, Price & Value
Categorizing certain products in different groups with different namings Discovery
Redesign mini cart dropdown to emphasize was/now anchor prices in cart items Price & Value
Show vs Hiding sticky header with additional product categories Discovery
Show vs Hiding background color vs background image in navigation bar Usability
Show vs Hiding “Deals” vs “Deals and Savings” in navigation Usability, Price & Value
Reducing number of categories in the top navigation to the most popular products Discovery, Usability
Show vs Hiding link to “Sales” page in navbar Price & Value
Top of Page Promo Banner
Sticky vs. fixed top of page promotion banner Price & Value, Usability
Design optimization of promotion bar Price & Value, Brand
Showing and hiding a countdown timer for when a sale ends Price & Value, Brand
Show vs Hiding closeable banner with rotating discounts Price & Value, Usability
Sticky vs non-sticky navigational bar Usability
Closeable sticky vs non-sticky discount code banner on mobile Usability, Price & Value
Show vs Hiding thin vs thick discount code banner Usability, Price & Value
Show vs Hiding bright colored text on promotional banner Usability, Price & Value
Testing a revolving carousel promotion bar vs. a static promotion bar Price & Value, Usability
Show vs Hiding banner that emphasizes free shipping Price & Value
Mobile Hamburger Menu and Navigation
Linkbar in addition to hamburger menu Discovery, Usability
Test scrollable linkbar vs. linkbar in a grid Discovery, Usability
Bring emphasis to the hamburger menu with text vs icons vs colors Discovery, Usability
Test a mega menu inside mobile hamburger menu including swiping vs dropdowns Discovery, Usability
Test removing subcategories inside mobile menu and re-arranging and grouping menu items Discovery
Adding a number of items indicator on the mobile cart icon Usability
If users have item in the cart, show a dropdown from cart icon with item name and price savings Usability, Price & Value
Placing content in two columns vs stacking to improve load time Usability
More clearly demarkating different categories of products inside the hamburger manu Usability
Adding “Menu” text under the hamburger icon, to make it clearer for all users that it’s a menu Usability
Testing fewer product categories inside the hamburger menu Usability, Discovery
Sticky vs Non-sticky full header navigation on mobile Usability
Live Chat
Test rectangular tab vs bottom right circle for live chat widget (Mobile Only) Support, Usability
Showing and hiding mobile live chat Support
“Contact” vs “Live Chat” icon Support, Usability
Adding a support icon and text link in the global navbar (instead of a support icon in lower right corner) Support, Usability
Add clear, prominent button on the PDP to “Order by phone” or talk to a stylist during business hours Support, Usability
Footer Contents
Showing vs. Hiding UGC (User Generated Content) in Footer Brand, Product Appeal

Listing or Collections
Layout and Design
Adding ability to filter items according to specific product details (color, size, etc) Discovery, Usability
Showing vs Hiding Quick Shop or Quick Look option for every product Usability, Product Detail
Showing specific product details or stats near photo of every product Product Detail, Product Appeal
Showing product images from multiple angles of product on mouse hover Product Detail, Product Appeal
Adding color and refinement product filters for each specific product category Product Detail, Product Appeal
Showing vs Hiding brand logos under each product photo Product Detail, Brand
Adding product dimensions and measurements next to each product listing (furniture) Product Detail
Replace full product photos with zoomed in photos that show more product detail Product Appeal, Product Detail
Longer descriptions under product photos vs only product names Product Detail, Product Appeal
Testing different product names on listing page Product Appeal, Brand
Show vs Hiding available sizes of products on mouse hover Product Detail
Show vs Hiding “pre-order” product label on pre-order products Scarcity, Product Appeal
Testing different arrangements of filters in the left sidebar of a product listing page Discovery, Usability
Show vs Hiding different product images (item alone, item removed from packaging, etc) Product Appeal
Testing different layouts and photo sizes of products on the mobile listing page Usability, Discovery
Increase size of product images Usability, Product Appeal
Show vs Hiding long vertical page vs shorter stacks of products Usability, Discovery
Add beautiful stylized, image heavy category banners on top of each category page Brand
Product Recommendations
Adding interactive quiz to help user choose one product and reduce number of products on page (mobile) Discovery, Usability
Showing only the most popular products and hiding remaining behind “see more” link Discovery, Usability
Show vs Hiding a table to compare different products and their features Product Detail, Product Appeal
Interactive product recommendations based on what option(s) user selects Discovery, Usability
Page Navigation
Showing vs Hiding number of products on a page Discovery, Usability
Showing vs Hiding recently viewed products container Discovery, Usability
Showing vs Hiding related categories at the top of the page Discovery, Usability
Testing more specific vs. more general terms for filter copy Discovery, Usability
Showing vs Hiding a carousel displaying various product styles at the top of the page Discovery, Usability
Redesign of listing page to look similar to homepage, have larger photos, more immersive, modern design Brand, Product Appeal
Show vs Hiding additional brand images Brand, Product Appeal
Show vs Hiding several vs fewer products on page Usability, Discovery
Placing items in a vertical vs horizontal layout Usability
Tesitng different value propositions in listing page headline Brand
Adding product benefit icons Product Detail, Product Appeal
Rearranging products by type instead of alphabetically Usability
Adding additional information on who would benefit most from specific products Product Detail, Product Appeal
Adding logo and copy about specific patented material used in products Product Detail, Brand
Show vs Hiding more product options Discovery
Discount Codes and Promotions
Adding banners with discount codes and applying discount directly to product as percentage Price & Value, Usability
Adding product-specific discount codes underneath each individual product Price & Value, Usability
Displaying calculated savings as a percentage vs. dollar savings Price & Value
Making was vs now pricing more prominent Price & Value
Improving newsletter email signups by personalizing popup copy to product categories Brand, Usability
Show promotional pricing for new customers only Price & Value
Social Proof Design Adjustments
Adding product review star ratings under each product Product Appeal, Usability
Mobile Navigation
Showing vs Hiding a sticky filter bar on mobile for easy access Usability, Discovery
Adding a clickable “scroll to top” button on mobile Usability
Test a mega menu inside mobile hamburger menu with fewer categories Discovery
Allow mobile users to easily select (and remove) size filters Product Detail, Discovery
Showing vs Hiding additional, more specific, listing pages Usability
Make sitewide header sticky on scroll vs. not sticky on scroll Usability

Product Detail Pages
Social Proof Design Adjustments
Showing customer UGC (Instagram) photos of customers using the products Brand, Product Appeal
Changing placement of additional product options on the page Discovery, Product Detail
Showing customer reviews as a clickable link option at the top of the PDP Brand, Product Appeal
Adding “featured in” social proof icon on each product image Brand
Show celebrity Instagram images and testimonial within description Brand
Show curated, featured customer testimonials prominently on the PDP Brand
Move customer reviews above product description Brand, Usability
Show vs Hiding “Quality Guaranteed” icon on product Brand, Product Appeal
Show vs Hiding customer testimonials in a highlighted box Brand
Price and Savings Emphasis
Emphasizes savings as a percentage off the original price Price & Value
Resize the rows of related product images so customers can see more content on the screen Discovery, Usability
Collapse parts of the description to highlight important details Product Detail, Product Appeal
Show vs Hiding additional options to allow user to bundle products for a discount Quantity, Discovery
Show vs Hiding “Free Shipping” under purchase link Price & Value
Show vs Hiding all available Sale products instead of a single product Price & Value, Discovery
Adding was vs now and percentage discount Price & Value
Show vs Hiding emphasis on low price guarantee Price & Value, Brand
Show vs Hiding default selection as subscription vs one-time purchase Quantity, Usability
Testing a full “build a box” flow vs regular ecommerce add to cart flow Usability, Quantity
Test preset quantity options (3, 6, 9, 12) vs. letting users choose any integer quantity Quantity, Usability
Showing a progress bar to receive a free gift after a certain cart size (to try to increase AOV) Quantity
Testing effect of 25% off promotion copy near add to cart button Price & Value
Showing anchor price and savings amount of the subscription option to encourage more subscription sales Price & Value
Testing different product pricing to find the optimal price point Price & Value
Testing different free shipping thresholds (to increase AOV via free shipping threshold) Price & Value, Quantity
Showing a countdown timer on a promotion to create pricing urgency Price & Value
Show vs Hiding anchor pricing (crossed-out original price) Price & Value
Show vs Hiding “Free Shipping” and “Free Returns” text under “Add to Cart” button Price & Value, Brand
Testing “Subscription” as the default selection on page load vs “One Time Purchase” Quantity, Usability
Show a discount to new visitors only with a “My first purchase” to encourage more first time purchases Price & Value
Product Details
Add option to view other products with the same print pattern Discovery, Product Detail
Add option to quickly access different products on the page in a pop-up window Discovery, Usability
Add category of “Suggested Items” to promote the best selling products Discovery
Add related categories of products near the Reviews Discovery
Moves description to top and bottom of product options Product Detail, Usability
Show vs Hiding specific details of products and increasing photo size Product Appeal, Product Detail
Significantly increasing the amount of information about product Usability, Product Detail
Show vs Hiding various sections of product description Product Appeal, Product Detail
Make product image move when clicked Product Appeal, Usability
Change size dropdown to show smallest (and cheapest) option vs. most popular option Price & Value, Usability
Emphasize specific value proposition in PDP copy Product Appeal, Brand
Start image carousel with zoomed in product image instead of full size product image Product Appeal, Product Detail
Show vs Hiding full page popup that links info on how to choose the right product Discovery, Product Appeal
Changing copy to highlight most impressive benefits of a specific product Product Appeal
Add lifestyle images of customers using products in their homes Product Appeal
Adding bullets of benefits near call to action and move call to action up Product Appeal, Usability
Adding customer-sourced suggestions on how they use the product throughout the day Product Detail, Product Appeal
Adding list of product benefits above order button Product Appeal, Usability
Adding copy of what a product tastes like Product Appeal
Moving link to size guide to a more prominent part of the PDP Product Detail, Usability
Testing a large redesign involving product images, featured testimonials, free shipping copy and more Product Appeal, Price & Value
Testing product names: a scientific variation vs a more colloquial name Product Detail, Product Appeal
Testing single product quantities vs multiples Quantity
Changing product names from “Trial” pack to “Variety” back Product Appeal
Testing “single-serve” vs “travel pack” product names Product Appeal
Test different default selected flavors on page load for food PDPs Product Appeal, Product Detail
Showing vs. hiding afterpay or affirm payment option to see if it affects conversion rate or AOV Price & Value
Testing “Variety” vs “Trial” vs “Sample” product name and description Product Appeal
Product Recommendations
Testing a recommended products container Usability, Discovery
Show vs Hiding dropdown with more products so users can create a bundle Discovery, Quantity
Adding option to quickly add accessory items to order Discovery, Quantity
Increasing the number of recommended products Discovery, Usability
Showing upsell items that compliment current product Discovery, Quantity
Showing upsell bundle options for those buying one item Quantity
Add larger quantity “bundle” option as a promotional offer below add to cart button Quantity
Design Adjustments
Show vs Hiding in-person retail store information Brand
Show vs Hiding copy and logo of specific patented materials used in the product Brand
Adding reviews and live chat banner when users attempt to leave page Brand, Usability
Show vs Hiding link to popup of current shipping times Product Detail, Product Appeal
Testing casual vs professional copy on “Buy Now” order link Usability
Testing copy and design emphasis of appointment form call to action Usability
Geotargeting: custom appointment form calls to action based on user location Usability
Testing “Pre-order” vs “Sold Out” copy for out of stock item Product Appeal, Brand
Testing loading a cart drawer (slides in from right side) instead of loading separate cart page Usability
“Add to Cart” area is sticky (fixed in one position) vs not sticky Usability
Adding professional images of people using item Product Appeal
Video and Images
Showing thumbnail images under product carousel vs. dots or a line indicating there are more images Product Appeal
Adding a video in gallery to describe product Product Appeal
Adding an explainer video to explain all options of a product underneath photo gallery Product Appeal, Product Detail
Testing an explainer video specifically on a bundle product Product Appeal, Product Detail
Test a promotional video on different positions on page Brand, Product Appeal
Adding explainer images (images with overlays of text with product details) in image carousel of PDP Product Detail
Showing vs. Hiding image of model wearing apparel product in image carousel of PDP Product Appeal
Brand vs Competitors Highlight
Show vs Hiding chart of benefits of brand versus competitors (with regards to specific products) Brand, Product Appeal
Adding photo comparing product details (dimensions) vs. competitors Product Appeal, Product Detail
Mobile Navigation
Adds sticky functionality to “Add to Cart” on mobile to make it more easily accessible for customers Usability
Displays breadcrumbs to easily return to the main product category and keep shopping Discovery, Usability
Replaces dots with image thumbnails to indicate “additional images” on mobile PDP Usability, Product Appeal
Adjusting placement of review stars and social media buttons Usability, Brand
Adding “Sale Ends” countdown timer on mobile Price & Value, Brand
Add copy about how easy checkout is to complete on mobile and how much customers saved Usability
Shorter vs. longer product descriptions and copy on PDP Usability, Product Detail
Show vs Hiding copy to emphasize ease of checkout on mobile Brand
Reduces amount of content and emphasize calls-to-action Usability
Show vs Hiding call-to-action instead of hiding it by default Usability
Show vs Hiding buttons with pricing options vs dropdrown with prices Usability, Price & Value
Display promo code in a different place for first-time visitors Price & Value, Usability
Show vs Hiding sticky “Add to Cart” button on bottom of screen Usability
Reducing content to scannable bullets on mobile (to encourage reading more of the PDP on mobile) Usability
Showing product image carousel first vs product title and description first on a mobile PDP Product Appeal

Cart and Checkout
Discount Codes and Promotions
Adjusts placement of discount code on page and makes it sticky Price & Value
Showing vs Hiding promo code input box Price & Value, Usability
Showing vs Hiding additional savings and adjusting text size Price & Value
Adjusting text color to highlight savings Price & Value
Showing vs Hiding discount code countdown timer on checkout page Price & Value
Automatically adding discounts to products in cart Price & Value, Usability
Show vs Hiding detailed summary of discount savings Price & Value
Show vs Hiding duplicate promotional banner of when discounts will end Price & Value, Usability
Show vs Hiding one active promo code near promo code box Price & Value
Show vs Hiding discount code for user instead of automatically applying code Price & Value, Usability
Show vs Hiding shipping in cart instead of in final step in checkout Price & Value, Usability
Show vs Hiding a bar that shows progress to Free Shipping discount Price & Value, Quantity
Page Design and UX
Reduces legnth of checkout order form Usability
Showing vs Hiding summary of products in cart in first step of checkout Usability, Discovery
Adding additional proceed to checkout button at the top of cart page Usability
Showing vs Hiding suggested products before finalizing payment Discovery, Quantity
Showing vs skipping a separate login page during checkout flow Usability
Show vs Hiding detailed summary of customer order Usability
Show vs Hiding estimated shipping dates at the top of checkout page Usability
Show vs Hiding additional prompt to confirm product added to cart Usability
Show vs Hiding link to FAQ page Usability
Removing various trust badges and benefit copy on proceed to checkout modal to reduce distractions Usability
Show vs Hiding link to view cart on proceed to checkout modal Usability
Two page vs one page checkout Usability
Show vs Hiding estimated shipping dates in both cart and checkout Usability
Manual vs automatic address fill-in Usability
Reducing elements in header so customer can focus on checkout Usability
Testing effect of making phone number a mandatory field during checkout Usability
Show vs Hiding optional information to complete checkout Usability
Show vs Hiding why user bought the product Usability
Allow users to choose from one-time and subscriptions (or vice versa) within cart Price & Value
Payment Options
Add Apple Pay payment option Usability
Product Recommendations
Show vs Hiding prompt to add additional product to order Discovery, Quantity
Show vs Hiding prompt to add specific product to order Discovery, Product Appeal
Show vs Hiding additional products when adding items to cart Discovery, Quantity
Show vs Hiding multiple upsell products Discovery, Quantity
Show vs Hiding one upsell item based on what product customer has in cart Discovery, Quantity
Adding an upsell popup when users are ready to checkout Quantity
Show vs Hiding upsells under customer’s cart items Quantity
Show vs Hiding upsells under order total Quantity
Personalized Improvements
Showing vs Hiding popup to save cart by creating an account Usability
Allow user to save products in cart for later on desktop or mobile Usability
Show vs Hiding prompt to create an account Usability
Show vs Hiding link to popup to “Save Your Bag” Usability
Highlighting and emphasizing savings during entire checkout process Price & Value
Show vs Hiding all saving details of order total (savings, crossed out price, free shipping) Price & Value
Dropdown Mini-Cart
Showing vs Hiding PayPal button checkout option in cart Usability
Cart popup vs loading a separate cart page Usability
Security Design Adjustments
Adding additional secure payment icons to cart Brand
Adding PayPal Merchant credentials Brand
Adding shopping guarantee Brand
Mobile Navigation
Showing vs Hiding sections of checkout to reduce page length on mobile Usability
Adding a link to save cart for later (aimed to increase user registrations and cross device purchasing) Usability

Blog
Discount Codes and Promotions
Adding blog-only discounts when mentioning or linking to products Price & Value
Social Proof Design Adjustments
Adding customer star ratings above product that’s mentioned in blog post Brand, Product Appeal
Adding more calls-to-action and increase number of displayed user reviews Discovery, Brand
Page Navigation
Changing links to products using different colors so they visually pop more in a wall of text Usability
Show vs Hiding link to product with bolder color Usability
Show vs Hiding table of contents and moving product link higher on page Usability
Show vs Hiding affiliate disclosure Usability
Sticky vs. fixed navigation bar with new category labels Usability, Discovery
Product Details
Adding product benefits and history of brand Product Detail, Brand
Adding a callout box that highlights product as product of choice for a given blog post Usability, Brand
Making calls-to-action and product links specific to the topic of the blog post (vs generic “view product” links on all posts) Usability
Product Recommendations
Recommending individual products vs combined package products Discovery
Testing different flavors or models of a product in certain blog posts Discovery
Recommending a selection of products instead of just one Discovery
Mobile Navigation
Redesigning mobile blog layout with smaller images and text plus adding hamburger menu Usability

Purpose of This Ecommerce AB Test Database

We made this database to show the extent of our experience testing and optimizing ecommerce sites. For example, if you’re thinking of testing a specific part of your ecommerce site — say, your mobile hamburger menu —  you can scroll to the navigation section and see all of the mobile hamburger menu tests we’ve done. Or if you’re thinking about testing upsells during checkout, you can scroll to the Checkout section and see what upsell related tests we’ve done.

The secondary purpose is to show the greater ecommerce community where us and our clients have focused time and attention in AB testing over the years. For example, you can see a lot of focus on parts of an ecommerce site like: pricing details on the PDP (product detail page), both desktop and mobile navigation tests, product recommendations, homepage messaging, and more. 

Finally, this page gives us a place to link to and organize our longer case studies, which takes us to…

Where are the Results of These A/B Tests? 

We don’t post the results of the A/B tests on this database for a two reasons. 

First, it would skip over the nuance and richness of A/B test results that we explained in depth in our article on why we base our A/B tests on questions and not hypotheses. A/B testing is not just about getting a binary result “So did it win or not?” As we explain in that post, we want a richer answer. To do this we multiple goals in each test and view each test as a way to learn more about the customer, not just as a one time gimmick to increase conversion rate (“this color vs. that color!”). As time allows, we write detailed case studies for certain tests and link to it above. That said, we do think you can learn from overall trends and patterns from multiple tests on certain parts of ecommerce sites, which brings us to…

Second, we think it’s the privilege of our clients to gain access to these years of accumulated learnings on how to design high converting ecommerce sites. If interested, you can learn more about working with us here