The Skeptical Designer’s UX Process

Great UX designers are great analysts.

They question everything. They’re aware of biases. They validate assumptions through triangulation. They think in terms of screens and systems.

As a designer, you oftentimes face situations in which a coworker (or even fellow designer) enthusiastically explains a business problem that needs solving.

Before jumping on board, your first instinct must be to question the problem. Skepticism needs to drive design, otherwise we become victim to our own excitement.

In this post, we dive into:

• How to ask the right questions upfront.
• How to ruthlessly prioritize the right features.
• How to check your enthusiasm with an MVP.

Never settle – question everything

We first learned of the 5-question approach in Chris Thelwell’s blog post.

As Head of UX at Envato, Chris asks his team these questions before every project. Let’s examine how to use each question to reveal dangerous assumptions and knowledge gaps.

1. What do we know we know?

You want to discuss anything that will influence your decisions in the project. For example, lessons learned from previous projects, existing user research insights, even opinions held by certain stakeholders.

The first question also helps you reveal assumptions disguised as facts. When someone says something they believe to be true, ask them for supporting evidence.

Let’s say you’re tasked with redesigning an analytics platform. Trial-to-signup conversions have plummeted. Something needs to change.

A product manager might say “Customers with the highest lifetime value buy our analytics platform because we offer the most customizability for reporting ”. That statement is a fact only if you can verify against prior user research and feedback. Otherwise, you must consider it an assumption.

Keep a running tally of assumptions vs. facts. Mention every assumption as an answer to the second question below.

2. What do we know we don’t know?

Now you reveal even more  assumptions.

At this point, a marketer might say “I’m not sure if our competitor’s aggressive ad retargeting of our trial users is stealing away our on-the-fence users”. You would list the response as an assumption in the format of “Competitive ad retargeting might be stealing market share”.

It’s also worth discussing the consequences if your assumptions are proven wrong. For instance, if you later discover that your most valuable customers aren’t buying your platform for its customizable reporting, how will that impact the project?

The goal here is to identify the risk behind your assumptions.

As you can tell, our above assumption carries a high degree of risk. If you dive into redesign and learn after the first round of usability testing that a reworked reporting feature makes no difference, your team just wasted untold hours and dollars.

As you start seeing the different risks behind assumptions, you start to naturally prioritize the questions that initial user research will seek to answer.

Research should reveal most answers, but some might be unanswerable (e.g. determining the effectiveness of a competitor’s retargeting ad). In those cases, at least you’ve pushed that risk to the surface where stakeholders can evaluate early in the process (e.g. proceed with the project, or push to backlog).

3. What do we actually know, but think we don’t?

The answers to this question might naturally arise as other people chime in to the previous questions.

For example, a marketer might answer question #2 with “I’m not sure if people churn due to cost or frustration”, and your user researcher might explain a series of usability reports that points to frustration.

Nonetheless, it’s worth revisiting all the answers you’ve categorized as assumptions. Ask the team if they worked on similar projects in the past, and if they revealed any insights regarding the assumptions. You don’t want any facts miscategorized as assumptions since you’ll later waste time researching previously validated information.

4. What are the unknown unknowns?

This questions sounds strange, but pay close attention.

Have we failed to discover anything so far that could completely ruin the project? What haven’t we thought about yet?

When you ask this question, you’re turning the grey zone of “unknown unknowns” into illuminated “known unknowns” that your team can prioritize based on risk.

For example, the question might prompt a developer to say “Well, now that you bring it up, we actually found out three weeks ago during load testing that our reporting feature can lag by up to 4.5s for data sets exceeding 100 rows”.

That information might be an “unknown unknown” for everyone outside of the development team, but you’ve now added it to everyone’s list of assumptions to test.

You’ve now revealed that the team needs to also test that “We might be losing customers due to slight performance issues with our current reporting feature”. In this case, the solution might not even require any front-end design work.

5. What does success really look like?

“A smooth product experience” doesn’t cut it. You need to write down S.M.A.R.T goals.

• Specific
• Measurable
• Actionable
• Relevant
• Trackable

Here’s an excellent example of a S.M.A.R.T. goal in our hypothetical UX project:

“Success is achieved when our trial-to-signup conversion rate increases from 2% to 2.50% without affecting churn”.

As you start validating assumptions through user research, you might make the goals increasingly specific as you start to hone in on a solution. If you discover, for instance, that a buggy checkout process is mostly affecting sales completion, you might add a secondary goal that “Success is achieved when our checkout completion rate increases from 45 to 75%”.

Reveal constraints from every corner

Questioning the problem helps us decide if we should design a solution. Formalizing our constraints helps us decide how we should design the solution.

All product design faces these constraints:

• Timing – In today’s world of Lean and Agile, every company wants to ship good products quickly. Find out if you need to release your MVP by a non-negotiable date (e.g. a global tech conference your company spent $58,000 sponsoring)

• Technology – How well can the development team write the code so your system’s response time feels instant? Do your servers support enough bandwidth to feed data quickly in real-time? Does the location of your data centers affect quality of experience in high-revenue regions?

• Medium – What do users already expect from your type of product? What current mental models and UI patterns does your product need for consistency?

• Budget – How much money is the company setting aside for the project, and how much could they stretch in case unforeseen delays or challenges appear? “Wobbler” features might emerge (e.g. features that are just slightly out-of-scope), so it helps to see if any financial cushion is available.

After your questioning exercise, conduct thorough stakeholder interviews so you can further reveal any hidden constraints as early as possible.