Post Image

How To Mix Dogfood Into Product Development

by
Jerry Cao
Jerry Cao

As you build the product, it’s important to test internally and externally to get out of the development bubble. Dogfooding is a quick internal sanity check that ensures the product team stays empathetic to users. On the other hand, beta testing can be more in-depth since it is the last comprehensive user review of your product before you hand it off to be shipped. In this piece, we’ll discuss helpful processes and documents to use for both types of testing.

Eat Your Own Dogfood

Once you’ve iterated your product to the point where it’s usable, testing it with your team is a great low-cost (or no-cost) reality check.

Guide to UX Design Process & Documentation - Dogfood

Since the cost of defects increases the longer they stay in the codebase, this feedback loop — known as dogfooding, alpha testing, or user acceptance testing — might even get you some net gains. The importance of dogfooding might best be understood by what happens when you don’t do it. For example, part of Facebook Home’s failure was that its product team used mostly iPhones while the product was based on Android. As a result, the team left out important widgets, docks, and app folders needed by Android users.

At UXPin, we eat our own dogfood. We use JIRA to keep track of bugs and a highly organized Google Docs spreadsheet for each team member to create a “user diary” of good experiences, annoyances, and times to complete certain tasks. We also use UXPin to prototype new iterations of UXPin (a very meta UX design process).   

Compared to earlier phases, the main difference is that dogfooding may not be done in as controlled a setting. Bug tracking software, user diaries, and alpha test surveys are all great for capturing feedback in this looser environment.

1. Bug Tracking Software

Most of the data you’ll get during dogfooding will likely be bug reports. While we’ve talked about the project tracking capability of JIRA, it truly shines at allowing for easy visibility and collaboration when documenting and squashing bugs.

Guide to UX Design Process & Documentation - JIRA

2. User Diary

As we described in The Guide to Usability Testing, user diaries are great at capturing organic information as someone uses the product freely. Using Google Docs, you can create a simple template and then include open-ended prompts such as:

  • Where were you when using the product?
  • What tasks did you hope to achieve? How long did it take you?
  • Describe something the product did to make your life easier.
  • Include a screenshot of something that frustrated you.

Guide to UX Design Process & Documentation - User Diary

3. Alpha Test Survey

Using a free site like Survey Monkey, you can create a survey to gauge how team members feel about the usability and functionality of certain features and the product as a whole. This sample survey captures both qualitative and quantitative information.

Screen Shot 2014-08-05 at 9.08.06 PM

According to Luke Freiler, CEO of beta testing company Centercode, the goal of an alpha test is to find bugs, crashes, and missing features so tailor your survey questions accordingly. The qualitative and quantitative data from an alpha test will also set the benchmarks for the product and emulate testing done with ~80% of real users.

Now that you’ve fed the dogfood to your own team, it’s time to try it with real users in a beta test.

Feed Your Dogfood to Others

Unlike an alpha test which is designed to improve product quality, a beta test is designed to incorporate customer feedback into the product.

As we described in our Guide to Minimum Viable Products, logging and then acting on customer feedback is one of the lowest-cost ways to prevent feature bloat. Since some bugs may still be discovered since alpha testing won’t cover all edge cases, JIRA can still come in handy for documentation. Similarly, the surveys you ran during internal testing can also be adapted for real users. Below, we’ll look at how other tactics like beta selection criteria, qualification surveys, and user forums can supplement your beta test.

1. Beta Selection Criteria

Before finding applicants, it’s important to prioritize the must-have qualifications, the nice-to-have qualifications, and subjective qualifications for testers. This will help you form a “scorecard” which you can use to filter applicants after they complete your qualification survey. Centercode has a simple selection checklist in their Beta Recruitment Kit to get you started.

Screen Shot 2014-08-06 at 6.10.38 PM

2. Beta Qualification Survey

Whether your beta test is public or private, you will want a send a survey to potential testers to ensure they fit the user profiles outlined in your PRD. Centercode’s Beta Recruitment Kit also contains a quick qualification worksheet survey which you can then translate into a live survey on Survey Monkey. You can also reference this live example from McAfee or the sample below.

In our Guide to UX Design & Process Documentation, we also explain how to create different surveys to improve your product after launching.

 

Guide to UX Design Process & Documentation - User Survey

3. Beta Testing Forums

Using a forum not only fosters earlyvangelism by building a community, it allows your team to actually respond to beta testers. Beta testers are volunteering their time because they believe in the product, so it’s important to make them feel appreciated. Furthermore, a forum will serve as a large-scale user diary of sorts for you to track the experiences of all beta testers.

Screen Shot 2014-08-06 at 6.15.28 PM

Use What Works and Scrap the Rest

Building your product is an ongoing process and the last thing you probably want to do is throw more paperwork into your sprints.

But a certain level of documentation is necessary to keep some order in all the chaos. User requirements coming from product management need to be translated. Dependencies among different technical entities have to be understood. And internal and external testing feedback must be captured to justify expensive-but-necessary changes. In between all this, you’ll need to answer stakeholder questions like “How is everyone staying on the same page?” and “How will we realize our goals in the set time limit?”

The templates and tactics we’ve listed all serve as helpful reference points as you prepare for the ultimate product test — your launch date.

For more smart ways on incorporating documentation into the design process, download the Guide to UX Design & Process Documentation. Expert advice is featured from Aarron Walter, Laura Klein, Ian McAllister, and dozens others. Visual examples are also shown from companies like Vurb, MailChimp, Apple, Google, and many more.

blog

Jerry Cao

by Jerry Cao

Jerry Cao is a content strategist at UXPin where he gets to put his overly active imagination to paper every day. In a past life, he developed content strategies for clients at Brafton and worked in traditional advertising at DDB San Francisco. In his spare time he enjoys playing electric guitar, watching foreign horror films, and expanding his knowledge of random facts. Follow him on Twitter.

Still hungry for the design?

UXPin is a product design platform used by the best designers on the planet. Let your team easily design, collaborate, and present from low-fidelity wireframes to fully-interactive prototypes.

Start your free trial

These e-Books might interest you