Growth Rock Database of Ecommerce A/B Tests
Below is a running list of all ecommerce AB tests we’ve run for clients for the past 7 years. You can read more about why we created this below. We update this database monthly.
Tests are organized by sections on an ecommerce site (Home, Nav, Listing, PDP, Checkout, Blog) and labeled with a mix of the 8 Purposes (Brand, Discovery, Product Appeal, Product Detail, Price & Value, Usability, Quantity, Scarcity) of every test as per our Purposes Framework for Ecommerce CRO.
Jump to read more about this database. Learn more about working with us here.
Home | |
---|---|
Homepage Products and Recommendations | |
Position of personalized product recommendation container (higher vs. lower on the page) | Discovery |
Swipeable carousel of recommended products vs a 2X2 Grid (Mobile only) | Usability, Discovery |
Show vs Hiding product discounts and removing separate product benefits section | Price & Value, Product Appeal |
Showing product bundles only for new visitors to see if it affects new visitor purchase behavior | Discovery |
Show vs Hiding a quiz to help customers find the right product | Discovery |
Show all products in a 4 column grid (like a listing page) directly on the homepage | Discovery, Usability |
Adding links to product categories in the middle of the homepage so categories aren’t only hidden inside hamburger menu | Discovery |
Popups | |
Hiding vs. showing an email capture popup on the homepage | Usability |
Redesign disclaimer popup with icons instead of paragraphs of text | Usability |
Homepage Hero Unit | |
Add product hotspots on hero unit creative to let customers easily buy products models are wearing | Discovery |
Using two types of animation via JS vs. CSS to measure page load speed | Usability |
Casual vs professional headline copy to catch user’s attention | Brand |
Testing different hero unit product photos | Product Detail, Product Appeal |
Adding featured customer reviews in different parts of the homepage | Brand |
If user has items in the cart, show the items photo in homepage hero unit and mention they are not done checking out | Usability |
Auto-playing vs pausing promotional video in the hero unit | Usability, Brand |
Displaying different copy depending on how user found website (ads, search engine, etc) | Brand, Usability |
Testing link vs. button styling on “shop now” CTAs in homepage hero unit | Usability |
Show vs Hiding email newsletter signup form within hero unit | Usability |
Promoting products in the hero unit vs. content and an email signup form | Discovery, Usability |
Homepage Content and Design | |
Testing a completely different upgraded homepage design | Brand |
Design of live chat icon – circular vs rectangular | Support, Usability |
Show vs Hiding customer support phone call option | Support, Usability |
Effect of additional link to “Sale” page | Usability |
Simple vs. longer appointment booking form | Usability |
Shorter vs longer homepage with more information, video, and testimonials | Brand |
Homepage with only products vs more information and reviews | Discovery, Product Appeal |
Testing the effect of Lighthouse support icon on conversion rate | Support, Usability |
Sending add to cart button clicks on homepage to cart vs. to the PDP | Discovery |
Significantly reduce copy and move call-to-action higher on page | Usability |
Placing newer products higher on the homepage | Discovery |
Homepage Social Proof | |
Testing effect of quotes from publications with logos | Brand |
Adding images of celebrities who use the products for social proof | Brand |
Show vs Hiding “Reseller Ratings” to increase customer trust | Brand |
Add more UX emphasis (brighter, larger) to a charitable/social good aspect of the brand | Brand |
Navigation, Headers, Footers | |
---|---|
Desktop Navigation | |
Sticky vs. fixed desktop header navigation | Discovery |
Redesigning a header navigation (desktop) | Discovery |
“Sale” vs “Clearance” copy and color emphasis on that nav link | Discovery, Price & Value |
Categorizing certain products in different groups with different namings | Discovery |
Redesign mini cart dropdown to emphasize was/now anchor prices in cart items | Price & Value |
Show vs Hiding sticky header with additional product categories | Discovery |
Show vs Hiding background color vs background image in navigation bar | Usability |
Show vs Hiding “Deals” vs “Deals and Savings” in navigation | Usability, Price & Value |
Reducing number of categories in the top navigation to the most popular products | Discovery, Usability |
Show vs Hiding link to “Sales” page in navbar | Price & Value |
Top of Page Promo Banner | |
Sticky vs. fixed top of page promotion banner | Price & Value, Usability |
Design optimization of promotion bar | Price & Value, Brand |
Showing and hiding a countdown timer for when a sale ends | Price & Value, Brand |
Show vs Hiding closeable banner with rotating discounts | Price & Value, Usability |
Sticky vs non-sticky navigational bar | Usability |
Closeable sticky vs non-sticky discount code banner on mobile | Usability, Price & Value |
Show vs Hiding thin vs thick discount code banner | Usability, Price & Value |
Show vs Hiding bright colored text on promotional banner | Usability, Price & Value |
Testing a revolving carousel promotion bar vs. a static promotion bar | Price & Value, Usability |
Show vs Hiding banner that emphasizes free shipping | Price & Value |
Mobile Hamburger Menu and Navigation | |
Linkbar in addition to hamburger menu | Discovery, Usability |
Test scrollable linkbar vs. linkbar in a grid | Discovery, Usability |
Bring emphasis to the hamburger menu with text vs icons vs colors | Discovery, Usability |
Test a mega menu inside mobile hamburger menu including swiping vs dropdowns | Discovery, Usability |
Test removing subcategories inside mobile menu and re-arranging and grouping menu items | Discovery |
Adding a number of items indicator on the mobile cart icon | Usability |
If users have item in the cart, show a dropdown from cart icon with item name and price savings | Usability, Price & Value |
Placing content in two columns vs stacking to improve load time | Usability |
More clearly demarkating different categories of products inside the hamburger manu | Usability |
Adding “Menu” text under the hamburger icon, to make it clearer for all users that it’s a menu | Usability |
Testing fewer product categories inside the hamburger menu | Usability, Discovery |
Sticky vs Non-sticky full header navigation on mobile | Usability |
Live Chat | |
Test rectangular tab vs bottom right circle for live chat widget (Mobile Only) | Support, Usability |
Showing and hiding mobile live chat | Support |
“Contact” vs “Live Chat” icon | Support, Usability |
Adding a support icon and text link in the global navbar (instead of a support icon in lower right corner) | Support, Usability |
Add clear, prominent button on the PDP to “Order by phone” or talk to a stylist during business hours | Support, Usability |
Footer Contents | |
Showing vs. Hiding UGC (User Generated Content) in Footer | Brand, Product Appeal |
Listing or Collections | |
---|---|
Layout and Design | |
Adding ability to filter items according to specific product details (color, size, etc) | Discovery, Usability |
Showing vs Hiding Quick Shop or Quick Look option for every product | Usability, Product Detail |
Showing specific product details or stats near photo of every product | Product Detail, Product Appeal |
Showing product images from multiple angles of product on mouse hover | Product Detail, Product Appeal |
Adding color and refinement product filters for each specific product category | Product Detail, Product Appeal |
Showing vs Hiding brand logos under each product photo | Product Detail, Brand |
Adding product dimensions and measurements next to each product listing (furniture) | Product Detail |
Replace full product photos with zoomed in photos that show more product detail | Product Appeal, Product Detail |
Longer descriptions under product photos vs only product names | Product Detail, Product Appeal |
Testing different product names on listing page | Product Appeal, Brand |
Show vs Hiding available sizes of products on mouse hover | Product Detail |
Show vs Hiding “pre-order” product label on pre-order products | Scarcity, Product Appeal |
Testing different arrangements of filters in the left sidebar of a product listing page | Discovery, Usability |
Show vs Hiding different product images (item alone, item removed from packaging, etc) | Product Appeal |
Testing different layouts and photo sizes of products on the mobile listing page | Usability, Discovery |
Increase size of product images | Usability, Product Appeal |
Show vs Hiding long vertical page vs shorter stacks of products | Usability, Discovery |
Add beautiful stylized, image heavy category banners on top of each category page | Brand |
Product Recommendations | |
Adding interactive quiz to help user choose one product and reduce number of products on page (mobile) | Discovery, Usability |
Showing only the most popular products and hiding remaining behind “see more” link | Discovery, Usability |
Show vs Hiding a table to compare different products and their features | Product Detail, Product Appeal |
Interactive product recommendations based on what option(s) user selects | Discovery, Usability |
Page Navigation | |
Showing vs Hiding number of products on a page | Discovery, Usability |
Showing vs Hiding recently viewed products container | Discovery, Usability |
Showing vs Hiding related categories at the top of the page | Discovery, Usability |
Testing more specific vs. more general terms for filter copy | Discovery, Usability |
Showing vs Hiding a carousel displaying various product styles at the top of the page | Discovery, Usability |
Redesign of listing page to look similar to homepage, have larger photos, more immersive, modern design | Brand, Product Appeal |
Show vs Hiding additional brand images | Brand, Product Appeal |
Show vs Hiding several vs fewer products on page | Usability, Discovery |
Placing items in a vertical vs horizontal layout | Usability |
Tesitng different value propositions in listing page headline | Brand |
Adding product benefit icons | Product Detail, Product Appeal |
Rearranging products by type instead of alphabetically | Usability |
Adding additional information on who would benefit most from specific products | Product Detail, Product Appeal |
Adding logo and copy about specific patented material used in products | Product Detail, Brand |
Show vs Hiding more product options | Discovery |
Discount Codes and Promotions | |
Adding banners with discount codes and applying discount directly to product as percentage | Price & Value, Usability |
Adding product-specific discount codes underneath each individual product | Price & Value, Usability |
Displaying calculated savings as a percentage vs. dollar savings | Price & Value |
Making was vs now pricing more prominent | Price & Value |
Improving newsletter email signups by personalizing popup copy to product categories | Brand, Usability |
Show promotional pricing for new customers only | Price & Value |
Social Proof Design Adjustments | |
Adding product review star ratings under each product | Product Appeal, Usability |
Mobile Navigation | |
Showing vs Hiding a sticky filter bar on mobile for easy access | Usability, Discovery |
Adding a clickable “scroll to top” button on mobile | Usability |
Test a mega menu inside mobile hamburger menu with fewer categories | Discovery |
Allow mobile users to easily select (and remove) size filters | Product Detail, Discovery |
Showing vs Hiding additional, more specific, listing pages | Usability |
Make sitewide header sticky on scroll vs. not sticky on scroll | Usability |
Product Detail Pages | |
---|---|
Social Proof Design Adjustments | |
Showing customer UGC (Instagram) photos of customers using the products | Brand, Product Appeal |
Changing placement of additional product options on the page | Discovery, Product Detail |
Showing customer reviews as a clickable link option at the top of the PDP | Brand, Product Appeal |
Adding “featured in” social proof icon on each product image | Brand |
Show celebrity Instagram images and testimonial within description | Brand |
Show curated, featured customer testimonials prominently on the PDP | Brand |
Move customer reviews above product description | Brand, Usability |
Show vs Hiding “Quality Guaranteed” icon on product | Brand, Product Appeal |
Show vs Hiding customer testimonials in a highlighted box | Brand |
Price and Savings Emphasis | |
Emphasizes savings as a percentage off the original price | Price & Value |
Resize the rows of related product images so customers can see more content on the screen | Discovery, Usability |
Collapse parts of the description to highlight important details | Product Detail, Product Appeal |
Show vs Hiding additional options to allow user to bundle products for a discount | Quantity, Discovery |
Show vs Hiding “Free Shipping” under purchase link | Price & Value |
Show vs Hiding all available Sale products instead of a single product | Price & Value, Discovery |
Adding was vs now and percentage discount | Price & Value |
Show vs Hiding emphasis on low price guarantee | Price & Value, Brand |
Show vs Hiding default selection as subscription vs one-time purchase | Quantity, Usability |
Testing a full “build a box” flow vs regular ecommerce add to cart flow | Usability, Quantity |
Test preset quantity options (3, 6, 9, 12) vs. letting users choose any integer quantity | Quantity, Usability |
Showing a progress bar to receive a free gift after a certain cart size (to try to increase AOV) | Quantity |
Testing effect of 25% off promotion copy near add to cart button | Price & Value |
Showing anchor price and savings amount of the subscription option to encourage more subscription sales | Price & Value |
Testing different product pricing to find the optimal price point | Price & Value |
Testing different free shipping thresholds (to increase AOV via free shipping threshold) | Price & Value, Quantity |
Showing a countdown timer on a promotion to create pricing urgency | Price & Value |
Show vs Hiding anchor pricing (crossed-out original price) | Price & Value |
Show vs Hiding “Free Shipping” and “Free Returns” text under “Add to Cart” button | Price & Value, Brand |
Testing “Subscription” as the default selection on page load vs “One Time Purchase” | Quantity, Usability |
Show a discount to new visitors only with a “My first purchase” to encourage more first time purchases | Price & Value |
Product Details | |
Add option to view other products with the same print pattern | Discovery, Product Detail |
Add option to quickly access different products on the page in a pop-up window | Discovery, Usability |
Add category of “Suggested Items” to promote the best selling products | Discovery |
Add related categories of products near the Reviews | Discovery |
Moves description to top and bottom of product options | Product Detail, Usability |
Show vs Hiding specific details of products and increasing photo size | Product Appeal, Product Detail |
Significantly increasing the amount of information about product | Usability, Product Detail |
Show vs Hiding various sections of product description | Product Appeal, Product Detail |
Make product image move when clicked | Product Appeal, Usability |
Change size dropdown to show smallest (and cheapest) option vs. most popular option | Price & Value, Usability |
Emphasize specific value proposition in PDP copy | Product Appeal, Brand |
Start image carousel with zoomed in product image instead of full size product image | Product Appeal, Product Detail |
Show vs Hiding full page popup that links info on how to choose the right product | Discovery, Product Appeal |
Changing copy to highlight most impressive benefits of a specific product | Product Appeal |
Add lifestyle images of customers using products in their homes | Product Appeal |
Adding bullets of benefits near call to action and move call to action up | Product Appeal, Usability |
Adding customer-sourced suggestions on how they use the product throughout the day | Product Detail, Product Appeal |
Adding list of product benefits above order button | Product Appeal, Usability |
Adding copy of what a product tastes like | Product Appeal |
Moving link to size guide to a more prominent part of the PDP | Product Detail, Usability |
Testing a large redesign involving product images, featured testimonials, free shipping copy and more | Product Appeal, Price & Value |
Testing product names: a scientific variation vs a more colloquial name | Product Detail, Product Appeal |
Testing single product quantities vs multiples | Quantity |
Changing product names from “Trial” pack to “Variety” back | Product Appeal |
Testing “single-serve” vs “travel pack” product names | Product Appeal |
Test different default selected flavors on page load for food PDPs | Product Appeal, Product Detail |
Showing vs. hiding afterpay or affirm payment option to see if it affects conversion rate or AOV | Price & Value |
Testing “Variety” vs “Trial” vs “Sample” product name and description | Product Appeal |
Product Recommendations | |
Testing a recommended products container | Usability, Discovery |
Show vs Hiding dropdown with more products so users can create a bundle | Discovery, Quantity |
Adding option to quickly add accessory items to order | Discovery, Quantity |
Increasing the number of recommended products | Discovery, Usability |
Showing upsell items that compliment current product | Discovery, Quantity |
Showing upsell bundle options for those buying one item | Quantity |
Add larger quantity “bundle” option as a promotional offer below add to cart button | Quantity |
Design Adjustments | |
Show vs Hiding in-person retail store information | Brand |
Show vs Hiding copy and logo of specific patented materials used in the product | Brand |
Adding reviews and live chat banner when users attempt to leave page | Brand, Usability |
Show vs Hiding link to popup of current shipping times | Product Detail, Product Appeal |
Testing casual vs professional copy on “Buy Now” order link | Usability |
Testing copy and design emphasis of appointment form call to action | Usability |
Geotargeting: custom appointment form calls to action based on user location | Usability |
Testing “Pre-order” vs “Sold Out” copy for out of stock item | Product Appeal, Brand |
Testing loading a cart drawer (slides in from right side) instead of loading separate cart page | Usability |
“Add to Cart” area is sticky (fixed in one position) vs not sticky | Usability |
Adding professional images of people using item | Product Appeal |
Video and Images | |
Showing thumbnail images under product carousel vs. dots or a line indicating there are more images | Product Appeal |
Adding a video in gallery to describe product | Product Appeal |
Adding an explainer video to explain all options of a product underneath photo gallery | Product Appeal, Product Detail |
Testing an explainer video specifically on a bundle product | Product Appeal, Product Detail |
Test a promotional video on different positions on page | Brand, Product Appeal |
Adding explainer images (images with overlays of text with product details) in image carousel of PDP | Product Detail |
Showing vs. Hiding image of model wearing apparel product in image carousel of PDP | Product Appeal |
Brand vs Competitors Highlight | |
Show vs Hiding chart of benefits of brand versus competitors (with regards to specific products) | Brand, Product Appeal |
Adding photo comparing product details (dimensions) vs. competitors | Product Appeal, Product Detail |
Mobile Navigation | |
Adds sticky functionality to “Add to Cart” on mobile to make it more easily accessible for customers | Usability |
Displays breadcrumbs to easily return to the main product category and keep shopping | Discovery, Usability |
Replaces dots with image thumbnails to indicate “additional images” on mobile PDP | Usability, Product Appeal |
Adjusting placement of review stars and social media buttons | Usability, Brand |
Adding “Sale Ends” countdown timer on mobile | Price & Value, Brand |
Add copy about how easy checkout is to complete on mobile and how much customers saved | Usability |
Shorter vs. longer product descriptions and copy on PDP | Usability, Product Detail |
Show vs Hiding copy to emphasize ease of checkout on mobile | Brand |
Reduces amount of content and emphasize calls-to-action | Usability |
Show vs Hiding call-to-action instead of hiding it by default | Usability |
Show vs Hiding buttons with pricing options vs dropdrown with prices | Usability, Price & Value |
Display promo code in a different place for first-time visitors | Price & Value, Usability |
Show vs Hiding sticky “Add to Cart” button on bottom of screen | Usability |
Reducing content to scannable bullets on mobile (to encourage reading more of the PDP on mobile) | Usability |
Showing product image carousel first vs product title and description first on a mobile PDP | Product Appeal |
Cart and Checkout | |
---|---|
Discount Codes and Promotions | |
Adjusts placement of discount code on page and makes it sticky | Price & Value |
Showing vs Hiding promo code input box | Price & Value, Usability |
Showing vs Hiding additional savings and adjusting text size | Price & Value |
Adjusting text color to highlight savings | Price & Value |
Showing vs Hiding discount code countdown timer on checkout page | Price & Value |
Automatically adding discounts to products in cart | Price & Value, Usability |
Show vs Hiding detailed summary of discount savings | Price & Value |
Show vs Hiding duplicate promotional banner of when discounts will end | Price & Value, Usability |
Show vs Hiding one active promo code near promo code box | Price & Value |
Show vs Hiding discount code for user instead of automatically applying code | Price & Value, Usability |
Show vs Hiding shipping in cart instead of in final step in checkout | Price & Value, Usability |
Show vs Hiding a bar that shows progress to Free Shipping discount | Price & Value, Quantity |
Page Design and UX | |
Reduces legnth of checkout order form | Usability |
Showing vs Hiding summary of products in cart in first step of checkout | Usability, Discovery |
Adding additional proceed to checkout button at the top of cart page | Usability |
Showing vs Hiding suggested products before finalizing payment | Discovery, Quantity |
Showing vs skipping a separate login page during checkout flow | Usability |
Show vs Hiding detailed summary of customer order | Usability |
Show vs Hiding estimated shipping dates at the top of checkout page | Usability |
Show vs Hiding additional prompt to confirm product added to cart | Usability |
Show vs Hiding link to FAQ page | Usability |
Removing various trust badges and benefit copy on proceed to checkout modal to reduce distractions | Usability |
Show vs Hiding link to view cart on proceed to checkout modal | Usability |
Two page vs one page checkout | Usability |
Show vs Hiding estimated shipping dates in both cart and checkout | Usability |
Manual vs automatic address fill-in | Usability |
Reducing elements in header so customer can focus on checkout | Usability |
Testing effect of making phone number a mandatory field during checkout | Usability |
Show vs Hiding optional information to complete checkout | Usability |
Show vs Hiding why user bought the product | Usability |
Allow users to choose from one-time and subscriptions (or vice versa) within cart | Price & Value |
Payment Options | |
Add Apple Pay payment option | Usability |
Product Recommendations | |
Show vs Hiding prompt to add additional product to order | Discovery, Quantity |
Show vs Hiding prompt to add specific product to order | Discovery, Product Appeal |
Show vs Hiding additional products when adding items to cart | Discovery, Quantity |
Show vs Hiding multiple upsell products | Discovery, Quantity |
Show vs Hiding one upsell item based on what product customer has in cart | Discovery, Quantity |
Adding an upsell popup when users are ready to checkout | Quantity |
Show vs Hiding upsells under customer’s cart items | Quantity |
Show vs Hiding upsells under order total | Quantity |
Personalized Improvements | |
Showing vs Hiding popup to save cart by creating an account | Usability |
Allow user to save products in cart for later on desktop or mobile | Usability |
Show vs Hiding prompt to create an account | Usability |
Show vs Hiding link to popup to “Save Your Bag” | Usability |
Highlighting and emphasizing savings during entire checkout process | Price & Value |
Show vs Hiding all saving details of order total (savings, crossed out price, free shipping) | Price & Value |
Dropdown Mini-Cart | |
Showing vs Hiding PayPal button checkout option in cart | Usability |
Cart popup vs loading a separate cart page | Usability |
Security Design Adjustments | |
Adding additional secure payment icons to cart | Brand |
Adding PayPal Merchant credentials | Brand |
Adding shopping guarantee | Brand |
Mobile Navigation | |
Showing vs Hiding sections of checkout to reduce page length on mobile | Usability |
Adding a link to save cart for later (aimed to increase user registrations and cross device purchasing) | Usability |
Blog | |
---|---|
Discount Codes and Promotions | |
Adding blog-only discounts when mentioning or linking to products | Price & Value |
Social Proof Design Adjustments | |
Adding customer star ratings above product that’s mentioned in blog post | Brand, Product Appeal |
Adding more calls-to-action and increase number of displayed user reviews | Discovery, Brand |
Page Navigation | |
Changing links to products using different colors so they visually pop more in a wall of text | Usability |
Show vs Hiding link to product with bolder color | Usability |
Show vs Hiding table of contents and moving product link higher on page | Usability |
Show vs Hiding affiliate disclosure | Usability |
Sticky vs. fixed navigation bar with new category labels | Usability, Discovery |
Product Details | |
Adding product benefits and history of brand | Product Detail, Brand |
Adding a callout box that highlights product as product of choice for a given blog post | Usability, Brand |
Making calls-to-action and product links specific to the topic of the blog post (vs generic “view product” links on all posts) | Usability |
Product Recommendations | |
Recommending individual products vs combined package products | Discovery |
Testing different flavors or models of a product in certain blog posts | Discovery |
Recommending a selection of products instead of just one | Discovery |
Mobile Navigation | |
Redesigning mobile blog layout with smaller images and text plus adding hamburger menu | Usability |
Purpose of This Ecommerce AB Test Database
We made this database to show the extent of our experience testing and optimizing ecommerce sites. For example, if you’re thinking of testing a specific part of your ecommerce site — say, your mobile hamburger menu — you can scroll to the navigation section and see all of the mobile hamburger menu tests we’ve done. Or if you’re thinking about testing upsells during checkout, you can scroll to the Checkout section and see what upsell related tests we’ve done.
The secondary purpose is to show the greater ecommerce community where us and our clients have focused time and attention in AB testing over the years. For example, you can see a lot of focus on parts of an ecommerce site like: pricing details on the PDP (product detail page), both desktop and mobile navigation tests, product recommendations, homepage messaging, and more.
Finally, this page gives us a place to link to and organize our longer case studies, which takes us to…
Where are the Results of These A/B Tests?
We don’t post the results of the A/B tests on this database for a two reasons.
First, it would skip over the nuance and richness of A/B test results that we explained in depth in our article on why we base our A/B tests on questions and not hypotheses. A/B testing is not just about getting a binary result “So did it win or not?” As we explain in that post, we want a richer answer. To do this we multiple goals in each test and view each test as a way to learn more about the customer, not just as a one time gimmick to increase conversion rate (“this color vs. that color!”). As time allows, we write detailed case studies for certain tests and link to it above. That said, we do think you can learn from overall trends and patterns from multiple tests on certain parts of ecommerce sites, which brings us to…
Second, we think it’s the privilege of our clients to gain access to these years of accumulated learnings on how to design high converting ecommerce sites. If interested, you can learn more about working with us here.