Conversion case studies are great. They show real results from real sites (instead of cliche top 10 lists). But one issue with them is that you only see the nice, shiny, glorious end result.
You don’t often see the thinking and strategy that led to those results. That’s what I’m going to be presenting here (and in future conversion teardowns).
I’ll take a site that I haven’t optimized, haven’t seen the analytics of, and don’t have an affiliation to, and I’ll break down how I would think about optimizing their conversion rate.
Here’s my goal: This will show you how to start thinking about your own site’s conversion strategy — from the beginning. Not just “try a red button color”, but actual processes for doing user research, learning about customers, finding good test hypotheses, and where to prioritize your efforts.
Let’s get started.
The Site: Cratejoy.com
Subscription boxes are boxes of stuff that arrive at your place regularly, and you pay for it every month (or so). Cratejoy is like the farmer’s market of subscription boxes. You can go there and find all kinds of stuff that you can have delivered to you every month: food, books, shaving supplies, cosmetics, whatever.
For example here’s a box of candy and snacks you can get for $12.95/month:
So you get a bunch of snack food delivered to your door for a low price every month. All you have to do is pay $12.95/month and watch out for diabetes.
And (this is key) you can also sell your own subscription boxes.
So they have an interesting conversion optimization problem: optimize for two different sides of a transaction.
We’ll discuss that in a bit, but let’s first discuss the CRO process.
My 5 Step CRO Process
You’ll see this as a pattern in future conversion teardowns, so let me outline how I approach the CRO process:
2. Browser Testing
3. User Research
4. Your Own Walkthrough
Since this is the first conversion teardown, let me explain the process in one sentence before we go step by step through the process for Cratejoy:
First, figure out what you should be measuring and measure that (1. Goals), then make sure stupid mistakes aren’t killing conversions (2. Browser Testing), then begin getting a deep understanding of what users want, what they’re trying to do, and where they’re getting stuck (3. User Research), then walkthrough the site yourself and think about what is clear, what is not, what works, what doesn’t (4. Walkthrough), and at the same time, with all the prior info in hand, make hypotheses of how you can more easily give users what they want better, faster, or with less barriers (5. Hypotheses).
If you follow this process instead of just testing random ideas that folks around the office have, you’ll get more sustainable results from your conversion optimization efforts.
Let’s go through the 5 steps for Cratejoy.
1.The Conversion Goal: Buyers or Sellers?
With marketplaces like this, the first thing you have to identify when starting to “optimize” a site, is the goal: Are you optimizing to get more sellers selling, or to get more buyers buying?
Don’t overlook how important this is for a marketplace site. There are two totally different visitor types and thus conversion goals:
1. Customers who want to buy subscription boxes.
2. Vendors who want to sell subscription boxes.
You have to be very careful on every page about who you’re speaking to. If you mix and match messages too much, you’ll confuse both sides.
Why B2C marketplaces focus on buyers first and sellers second
Notice that most established B2C marketplace businesses have their main homepage speak to buyers above sellers.
For example, here’s the homepage of Airbnb:
The entire hero unit is dedicated to you starting your search to book a place, and one button on the top is about becoming a host.
In a totally different industry, WordPress themes, here’s Themeforest.net’s homepage:
Again, the most prime screen real estate is about finding themes (for the buyer), and there’s one totally non-descript link in the navbar for selling themes.
Reason #1: Buyers are the source of cash
One reason Airbnb, Themeforest and a bunch of other marketplace companies do this (Ebay, Amazon, Etsy, even Craigslist) is because buyers are the foundation for the entire business. They are the source of cash. The demand they generate is the motivation for sellers to sell in that marketplace.
Reason #2: There are a lot more buyers
The second reason is that these are consumer based businesses (B2C), so the number of visitors of each type are heavily swayed in the buyers favor, so it makes sense that the site would focus on buyers’ needs. In a B2B marketplace business (say, a business that hooks up companies who need heavy machinery with companies that sell heavy machinery), the number of customers is far lower, so the ratio isn’t as buyer heavy.
So, buyers make the whole business run, and there’s a lot more of them. Makes sense to have the website focused on them, right?
Cratejoy’s Homepage: Unclear Focus
Unlike Airbnb, Themeforest, and others, Cratejoy uses premium homepage real estate to appeal to sellers, and it’s sandwiched between copy and functionality for buyers, these leaves the user distracted and unsure on the best next step:
Here are the main goals they should measure in Google Analytics (GA):
New to Google Analytics? Check out this great guide from Justin Cutroni: Google Analytics E-Commerce Tracking.
1. Purchases (through the ecommerce integration of GA)
2. Popular subscriptions
3. Clicks on these elements of the homepage (among others):
4. Setup a funnel through the “subscribe now” step.
1. Clicks on the homepage to start.cratejoy.com
2. Funnel through the store creation process.
If you do not have an ecommerece site I highly recommend you check out the article: Conversion Goal? It Depends on Your Business Model from Crazy Egg’s blog.
2. Browser Testing
Browser compatibility issues exist in almost every site. There are people likely viewing your site in a browser that are seeing errors and that’s drastically reducing your conversion rate.
For example, look at this GA breakdown of conversion rate vs. Internet Explorer version number from a client of mine:
IE 8 and 9 have a seroius conversion problem compared to other versions. And when we multipled the reduced conversion rate by traffic and average value of each conversion, we learned this was costing them $150,000 a year in revenue.
So, let’s look at Cratejoy.
I did these tests in Browserstack.com (as usual, no affiliation. In general, I don’t do affiliate links on my site, so just know when I link to something it’s not cause someone is paying me 10 cents for you to click it or some nonsense).
Here’s a sample of OS/Browsers where I saw obvious UI issues. Since I don’t have access to their GA, I can’t see how many people arrive every month from each browser, but they can. So they should do this browser testing for all of their major browser types according to GA.
Windows 8/Chrome 22
Some search bar/nav issues
Windows 7/IE 8
Windows 7/IE 9
Only a few subtle issues compared IE8.
No need to belabor the issue, but the general principle here is that you should test on all browsers and devices where you see non-neglible traffic.
Simply use a service like Browserstack to walkthrough a good chunk of your site.
3. User Research
Now we’re getting to one of my favorite steps: figuring out what people want (and what irritates them).
On page surveys
I’d definitely put a Qualaroo style survey on Cratejoy’s homepage, and their “start selling” page (start.cratejoy.com).
What I (and others) find works best is open-ended qualitative questions, with a few more pointed yes/no’s thrown in. As Dave Chaffey, founder of Smart Insights, explains in an article by Raphael Paulin-Daigle:
“We’re fans of asking open questions on specific pages in the journey to get additional insight you can’t get from analytics tools.”
So on the homepage, I’d ask:
- What kind of subscription boxes are you looking for?
- Have you ever bought a subscription box before?
- Are you finding what you’re looking for?
- What else do you want to know before trying a subscription box?
In terms of non-qualitative questions, I’d also ask:
- Are you looking to buy or sell subscription boxes?
- Have you ever thought of selling a subscription box?
People will then start answering the open-ended questions by basically telling you their problems and desires. Some will be short; others will be long.
Pro Tip: After I get 100 or more responses, I categorize them by subject. Then I make a simple histogram of the responses, so we can all clearly see the subjects users are asking and talking about the most. Here’s an example for a recent SaaS client:
Most people wanted to know more about how the software works, setup, etc. (We did a deeper analysis of all the how it works questions as well). And a ton of people just wanted to know price, which was not mentioned on the homepage.
So on page surveys will give you good insight into what live users want to ask for, complain about, and compliment you on.
For ecommerce type sites like this (I know this isn’t “traditional” ecommerce, but site functionality is similar: lots of product options, search, browse by categories, etc.), screen recordings of real users are a great way to figure out how users actually browse, add items to card, and check out.
They require patience. Watching people browse your site over and over again can get tedious, but they can reveal interesting behavior that you haven’t noticed before.
For Cratejoy, I’d be curious about a few things:
- How do you users actually find a subscription box they like? Trending? Similar? Search? Categories? Reviews?
- Do they use the trending subscriptions?
- What about sidebar items?
- Do they linger on reviews?
- Do they click through the different photos?
- Do they use the “people also bought” section?
- How many boxes do they typically browse before making a purchase?
Some of these are things that you could measure with careful click and event tracking in Google Analytics, but most people don’t do that. Other things (how long they linger on reviews or certain descriptions) are not easy or possible to measure with GA. (page time is not the same as time on a certain section of a page).
Live User Testing
Finally, an invaluable technique is to use actual live, paid users to go through your site and give you feedback while they browse and after they’re done. According to marketing consultant Kristina Allen, “There is no better way to find out how visitors behave on your site and what makes them convert at a higher rate than through user experience (UX) tests.”
You can do this with usertesting.com and competitors (I have no affiliation). Or, you can just put an ad on craigslist and ask for certain type of people and offer how much you can afford per video.
For a wide reaching B2C business like Cratejoy, it’s easy to find users that fit their target customers. You can watch them browse and have them give you feedback about what they like and don’t like.
For Cratejoy, I’d be very curious about the above questions from live users AND what else they ask for that I haven’t thought of.
- Are they asking for more photos?
- Are they curious why I have to go to a different site to actually purchase (I am!)
- Do they even notice that you can browse by category way on the bottom of the homepage?
4. Your Own Walkthrough
Your own walkthrough is the last step after you have all the user research in hand, so you already have some ideas of things that could be hampering conversions.
Now, it’s time to slowly go through the site, page by page (or page template by page template) and list:
- Things you can do better to answer user concerns
- Ways you could express value more clearly
- Places to remove distractions and clutter
- Ways to help users navigate more easily to what they want
- Other ways to add elements of persuasion
Check out my sample walkthrough of Cratejoy, here:
Obviously, a full site walkthrough will take a lot longer than 12 minutes, but you get to see the kinds of questions I ask that I’d end up testing.
Here are some of the questions I’ve asked myself when walking through Cratejoy’s site:
- Are trending items the first things you should see on the homepage? Is that really how users browse for subscription boxes?
- What about categories. I might be interested in subscription services, but I’m totally overwhelmed with what I’m seeing.
- Clarity clarity…am I shopping for a subscription or am I starting a business??
- I want the best ones, not the newest ones or hottest ones.
- I want to know what the box has in it. Not just a photo.
- Why do I have to go to a seperate looking page to buy? Can I trust them?
Finally, after all the information from the above steps is gathered. You should list them out into hypotheses. That is, variations to certain pages or page types to test.
Test them, of course, with your favorite A/B testing software like Optimizely or VWO.
Watch the video above to see some hypotheses that naturally fall out of the walkthrough.
Get a bonus user testing video plus more teardowns like this free
I’ll be walking through more sites on a regular basis. If you want to get future teardowns in addition to other conversion tips, as well as the recording of my user testing walkthrough with someone who has never seen the video before, click here and let me know where to send the video links and future teardowns. It’s totally free.
Have you gone through Cratejoy.com yourself? If so, what would you test that I haven’t thought of? Let me know in the comments.