What is Usability Testing and How to Run It

Usability testing is a crucial part of the design thinking process. It’s an opportunity for UX teams to present their solutions to those whose problems they’re trying to solve—a nerve-racking and exciting experience!

UXPin is the world’s most sophisticated prototyping and usability testing design tool. Using UXPin with Merge technology, designers can create high-fidelity prototypes with final product functionality using fully interactive and ready code components. With UXPin, you get accurate testing and meaningful participant feedback. Sign up for a 14-day free trial to experience advanced prototyping and testing with UXPin!

What is Usability Testing?

Usability testing (also called usability studies) tests user interfaces and flows with real users. A UX researcher (moderator or facilitator) will ask a participant to complete a series of tasks (usually on a digital product prototype) while observing their behavior and actions.

The moderator might ask the participant to verbalize their thoughts and actions so UX researchers can understand how the person thinks and feels as they use the prototype.

Why do UX Teams Conduct Usability Tests?

Usability testing provides UX teams with valuable feedback and user insights, including:

• Validating design concepts to solve users’ problems
• Exposing usability problems to fix
• Discovering opportunities for improvement
• Learn more about the users

Usability testing is an iterative process of testing, exposing issues/learning about the user, making adjustments, and retesting.

The ultimate goal of the usability testing process is to fix and improve prototypes as much as possible before the design handoff, where engineers start the development process.

Usability vs User Experience Testing

There is often confusion and debate over the difference between usability testing vs. user experience testing. It’s incorrect to use these terms interchangeably because they refer to different areas of testing.

• Usability testing – tests the product’s usability and performance—user interfaces, navigation, microinteractions, etc.
• User experience testing – tests the user’s enjoyment and satisfaction while using a digital product.

While we define these terms differently, UX researchers test usability and user experience at the same time. 

For example, one UX researcher might observe how the participant completes a task during a usability study (usability testing), while another researcher studies the user’s actions and behavior (user experience testing).

These are two important metrics to consider during product testing. If a user can complete a task, UX designers might see this as a job well done. But what if the user was frustrated during the process? They’ll likely switch to a competing product with a better user experience.

Types of Usability Testing

There are two primary usability testing methods:

• Moderated tests
• Unmoderated tests

UX teams can apply both of these methods to remote and usability lab (face-to-face) testing.

Moderated

During a moderated usability study, the facilitator interacts with the participant, asking them to complete tasks while observing and asking questions.

UX teams can conduct usability studies in a lab or remotely using Zoom, Skype, or purpose-built testing tools.

Advantages of moderated usability testing:

• Moderator ensures the participant understands and completes the tasks correctly
• Moderator can engage with the participant, asking questions about their actions, responses, and behavior
• The above points provide UX teams with accurate, meaningful feedback

Disadvantages of moderated usability testing:

• Requires lots of planning to find a venue, participants, align with team schedules, organize equipment, etc.
• Requires more resources, increasing testing costs
• Limited participants due to cost and time constraints

Unmoderated

During an unmoderated usability study, the facilitator is absent but provides the participant with instructions to complete a series of tasks.

The participant might complete these tasks in a lab environment, the field (where the users will typically use the product), or remotely.

Advantages of unmoderated usability testing:

• Researchers can test multiple participants at the same time
• Fewer resources make unmoderated testing significantly cheaper

Disadvantages of unmoderated usability testing:

• Relies on participants understanding tasks and instructions without guidance—might lead to uncompleted tasks or inconsistent test results

Usability Testing Methods

Card Sorting

Card sorting is an early-stage usability method for testing element hierarchy and establishing information architecture. 

The moderator presents a group of topics or categories for the participant to sort—usually by importance or group by relevance.

Paper Prototyping

Paper prototyping is another early-stage usability method where UX teams test user flows and information architecture.

UX teams rarely test paper prototypes with participants because usability tests are expensive, and paper prototyping doesn’t provide meaningful user feedback.

Still, paper prototypes can provide some insights into the user’s navigational expectations.

Digital Low-Fidelity Prototype Testing

Digital low-fidelity prototypes use a series of wireframes to test user flows and simple navigation. Like paper prototypes, low-fidelity prototyping provides limited feedback about the user experience.

High-Fidelity Prototype Testing

Testing high-fidelity prototypes allows UX teams to get accurate, meaningful feedback. Participants use a fully functioning replica of the final product to complete tasks.

UXPin Merge lets designers connect design elements with interactive components that also devs use to create high-fidelity prototypes with final product functionality. By designing with code components, participants can interact with a UXPin prototype better than any other design tool.

Why not try usability testing with UXPin’s 14-day free trial!

Click Tracking

Click tracking examines where users click or tap on a prototype. UX designers can use this information to see where participants most frequently click (or tap on mobile).

Click tracking can help validate link structure or whether participants can easily identify buttons and CTAs.

Eye Tracking

UX researchers use eye-tracking devices to learn how participants explore user interfaces or what elements catch the eye first. These insights can help UX designers decide how to prioritize screen layouts or where to place CTAs.

How to Conduct a Successful Usability Test in 6 Easy Steps

It’s crucial to have a plan and objectives for usability testing. Without a plan and goals, UX researchers won’t know what to test or the value of the test results.

We’ve broken usability testing into six easy steps:

1. Define Goals
2. Choose the Test
3. Create User Tasks
4. Write a Research Plan
5. Conduct the Test
6. Draft a Report

Step 1 – Define Goals

The first step is to define the usability study’s goals. These goals might ask broad or specific questions, for example:

• Broad: Which checkout method can user’s complete the fastest?
• Specific: Does animating a button increase clicks?

It’s important to prioritize goals and limit testing to a specific question you want to answer—like testing an eCommerce checkout flow or completing a new user sign-up process.

It’s tempting to make the most of usability test sessions and get as much feedback as possible, but this could lead to user fatigue and inaccurate results.

Step 2 – Choose the Correct Test

Once you know what you want to test, you can choose a suitable usability testing method.

In our free eBook, The Guide to Usability Testing, we outline 30 different usability testing methods, which to apply, and when.

We divide usability tests into four categories:

• Scripted — These tests analyze the user’s product interaction based on set instructions, with specific goals and individual elements. (tree testing, hallway usability tests, benchmark testing)
• Decontextualized — Ideal for preliminary user testing and user research (see the difference between user testing and usability testing). These tests don’t involve the product necessarily, but analyze generalized and theoretical topics, targeting idea generation and broad opinions. (user interviews, surveys, card sorting)
• Natural (or near-natural) — Researchers analyze users in the environment where they’ll use the product most often. These tests examine how users behave, pinpointing their feelings with accuracy, at the cost of control. (field and diary studies, A/B testing, first-click testing, beta testing)
• Hybrid — These experimental tests forego traditional methods to take a distinctive look at the user’s mentality. (participatory design, quick exposure memory testing, adjective cards)

Once you select the testing method(s), you can share them with the team, summarizing your goals and tactics in a usability planning document.

Step 3 – Create Your User Tasks

Everything you present to participants during usability testing, including questions and phrasing, impacts their response.

Usability tasks are either open or closed, and your tests should incorporate a healthy mix of both:

• Closed – A closed task offers little room for interpretation—the user is given a question with clearly defined success or failure (“Find a venue that can seat up to 12 people”). Closed tasks produce quantitative and accurate test results.
• Open – By contrast, participants can complete open tasks in several ways. These are “sandbox” style tasks: “Your friends are talking about Optimal Workshop, but you’ve never used it before. Find out how it works.” Open tasks produce qualitative and sometimes unexpected results.

Moderators must be mindful of how they phrase questions to avoid bias. 

For example, you want to know how a user will find a gift for their mother on an eCommerce store. If you phase the question as “can you search for a mother’s day gift in our store?” it might suggest that the participant use the search function instead of following their natural intuition. This question also sounds more like an instruction than a question.

A better way to phrase this question might be, “how would you find a mother’s day gift in our store?”

Step 4 – Write a Usability Research Plan Document

A usability research plan document should cover seven sections:

• Background — In a single paragraph, describe the reasons and events leading to the user research.
• Goals — In bullet points, summarize what the study hopes to accomplish. Phrase the goals objectively and concisely. Instead of “Test how users like our new checkout process,” write “Test how the new checkout process affects conversions for first-time users.”
• Questions — List out around 5-7 questions you’d like the study to answer
• Tactics — Where, when, and how UX researchers will conduct tests. Explain why you’ve chosen this particular test.
• Participants — Describe the type of user you are studying, including their behavioral characteristics. You could even attach personas for more context.
• Timeline — Recruitment start dates, test start and end dates/times, and when stakeholders can expect results.
• Test Script — Include a script if you have one prepared.

Encourage suggestions and feedback from stakeholders to ensure everyone is on board, and you haven’t missed anything.

Further reading – The Plan That Stakeholders Love: The One-Pager.

Step 5 – Conduct the Test

Here are some times for conducting usability testing:

• Make participants comfortable — Remind participants that you are testing the product, not their capabilities. Creating a script beforehand will ensure the moderator’s instructions, phasing, and tone are consistent.
• Don’t interfere — This avoids bias and may reveal user behavior insights you hadn’t predicted. Meaningful insights come from participants interacting with the product in ways you never expected. Observe natural human behavior and let them inspire feature improvements.
• Record the session — Teams may need to review usability studies at a later point. 
• Collaborate — A Rainbow Spreadsheet allows everyone to record their interpretations of each usability test. UX researchers can compare notes to see what they observed in common and any unique observations. Rainbow Spreadsheets worked well to summarize results for stakeholders during our team’s Yelp redesign exercise. (Source: Rainbow Spreadsheets by Tomer Sharon)

Step 6 – Draft a Report

A usability study report is an effective way to summarize results to share with stakeholders. 

Here are some tips for compiling a usability report:

• Don’t be vague – “Users couldn’t buy the right product” isn’t very helpful because it doesn’t explain why. Explain the specific issue from a UX perspective—“users couldn’t buy the right product because they couldn’t find the search bar.”
• Prioritize issues – Categorizing and prioritizing usability issues will help design teams know where to start. We recommend categorizing the report (e.g., Navigation Issues, Layout Issues, etc.) and using color codes for priorities (Low/Medium/High).
• Include recommendations – Lastly, you’ll want to include the team’s recommendations to fix usability issues, so stakeholders know there is a solution.

Other considerations for a usability report include:

• Formal usability report
• Supporting charts, graphs, and figures
• Previous testing documentation 
• Videos or audio tracks of the test

Usability Testing With UXPin

UXPin is a comprehensive design and prototyping tool. Unlike other design tools, UXPin doesn’t require plugins and apps to fulfill wireframing, mockups, prototyping, and testing requirements.

And UXPin does so much more! With states and interactions in your prototype you can enhance participant’s product interaction while providing you with meaningful feedback and results!

Try UXPin for 14-days and experience a whole new world of UX design, prototyping, and usability testing!