The first part of our analysis involved examining the qualitative data collected through the screen-recorded user tests we mentioned in our previous post.
As you might recall, we ran a series of tests on 5 users (as recommended by Jakob Nielsen). When choosing users, we were looking for semi-frequent Yelp users. Our 5 users included 3 users who had Yelp accounts and 2 who did not (since some of our tasks required accounts). Both groups of users were assigned common tasks like finding a restaurant and less-common tasks like finding an event. By choosing all kinds of tasks, we hoped to cover the spectrum of general use cases.
In this piece, we’ll uncover the insights and explain how they might influence our redesign of Yelp’s website.
We started qualitative analysis by watching our UserTesting videos and annotating interesting moments (UserTesting’s platform lets you mark up videos). While analyzing our user tests, several interaction patterns started to surface. To quickly summarize, we learned that the Search bar was one of the most mandatory features and that it was very easy to use— if users knew specific search criteria (such as the name of a business). Some of the other features weren’t as intuitive, though, as you’ll discover in the discussion below.
1. The Search bar was the main starting point.
All 5 users depended on the Search bar, even for tasks that could be accomplished by browsing through Categories (such as finding a cool restaurant without being given many guidelines). In fact, 4/5 users went directly to the search bar when they were looking for a restaurant. The other user started her task by using the Categories, and she defaulted to the Search bar after finding it “overwhelming”.
It’s also worth mentioning that we asked users to “find” a restaurant (not to “search” for one) because we didn’t want the language to bias their actions.
The results, of course, means that we will definitely want to make sure that search functionality is still prioritized in the redesign.
2. The Events tab wasn’t very noticeable.
For one of the tasks, we asked the two users without Yelp accounts to find an event in their area that seemed fun for the weekend. Our goal was to see if they would use the Events tab at the top of the page, which would seem like a natural place to go. One user defaulted to the beloved Search bar, while the other browsed through the Arts & Entertainment category within the “Best of Yelp” section.
The results clearly show us that if we want an easy way for users to find Events, we’d either need to redesign the Events tab (or entire top navigation), or replace Events with a completely different feature.
3. Bookmarking frustrated users, and nobody created Lists.
We wanted to see how users save locations for later reference. With Yelp, there’s two ways to accomplish this. Users with existing accounts can either bookmark a location, or create a list. We asked Group 1 (three users with Yelp accounts) to save a couple locations to look at later. In our wording, we chose “save” so that we wouldn’t accidentally mention the features and cause bias.
Here were the results from our three users:
- One user saved businesses using Bookmarks, but complained the process was too time consuming
- One user started to save businesses using Bookmarks, but abandoned it since it was also too time consuming
- One user couldn’t figure out how to save businesses
The two users who used Bookmarks both mentioned that it would be nice to bookmark a business directly from the search listings (instead of visiting each individual business, and then saving them with Bookmarks). You can actually hear him thinking out loud in the below video (click to play in a new window):
What this means is that the new design needs to incorporate a simpler method of saving a business to return to later.
4. It was very fast and easy to find a specific venue.
As you’ll see in our previous post, all 5 users were asked to find a specific business and check if it was open at a certain time. They all completed the task and rated the task as “Very easy”. The Search bar, of course, was used to complete all the tasks. Since this action was quite simple and easily accomplished, we decided we probably wouldn’t need to tweak it much in our redesign.
5. Photos were the most helpful for identifying ambiance.
When we asked all five users to find a restaurant with a certain ambiance, none of them used the search bar (surprisingly). Three users browsed through photos of the restaurant on Yelp, one clicked out to check out the restaurant’s website, and one actually said that the pricing symbols ($, $$, $$$, $$$$) was enough to tell if a restaurant had the right ambiance. You can hear some of their thoughts in the video below (click the play icon and it will open in a new window).
These findings lead to two takeaways:
1. Photos are an almost mandatory part of the Yelp experience and significantly influence business choices.
2. Yelp’s Search bar and filters don’t really account for ambiance. For our redesign, we either need to include a way to filter for types of ambiances, or we can leave it unaltered (since photos seem to be quite sufficient).
6. Filters are useful but could use some improvement.
When we asked all 5 participants to find a restaurant that could seat a group of 15, three of the five chose the “good for groups” filter. One user chose the “make a reservation” feature and then scrolled until she found the appropriate restaurant. At another point, another user tried to select two categories (Bars and Pubs) to filter his choices, but one actually disappeared when he clicked the other. You can see this happen below, much to his surprise (click the play button).
Clearly, the filters were a feature that were used by a majority of the testers. But given the accidental de-selection upon checking two different options under “Categories”, the functionality could be improved. In the next article in this series, we discuss quantitative testing and you’ll see how our card sort helped to reshape Yelp’s filters to better match natural thought processes.
7. The pricing categories needed to be clearer.
When we asked people to find a restaurant under $20, two users were confused by the pricing categories and weren’t sure if their $20 budget would be considered $, $$, or $$$. One actually said she wasn’t sure what the symbols meant, while another selected the wrong pricing symbol. The other three users all correctly chose the $$ symbol.
When you select a filter, price ranges are not shown for these symbols. These symbols are only defined once you visit a specific restaurant’s page. Because they weren’t clearly defined, it was of course left up to subjective interpretation (which leads to confusion). Budget is usually a strong influence on people’s decisions, so our redesign needs to better define the price ranges for the dollar symbols.
Smart user testing leads to better designs
Just because nobody has complained about certain features of a website doesn’t mean the visitors are using the site effectively. Yelp as a whole is quite usable, but there is always room for improvement. In fact, as you’ll see in our full e-book, sometimes a couple simple tweaks can go a long way. In the next part, we’ll look at the quantitative tests (card sorting, tree testing, click testing) run by Optimal Workshop.
If you’re interested in learning more about applying this process to your own design, feel free to sign up below for an early copy of the e-book. We explain different methods of user testing, how to find the right insights, and how to follow a design sprint to be creative and efficient.