To evaluate when and how internal testing should happen (and steer clear of the pitfalls), here\u2019s how to get started.<\/p>\n
![\"Internal](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image07-5.png\")
<\/p>\n
\u201cInternal testing surrogates can give you great feedback on the functionality and sentiment surrounding your designs\u201d<\/em><\/p>\n Internal user testing (dogfooding) is most valuable when there\u2019s greater concern for qualitative feedback and less for validity.<\/p>\n As shown by Slack\u2019s design process in the Real-Life UX Processes e-book<\/a>, dogfooding is all about cost- and time-conscious design insights.<\/p>\n Is your project right for internal testing? Make sure it falls in line with these criteria.<\/p>\n Remember, your goal is to get results quickly. Don\u2019t spend hours on data only your desk drawer will see.<\/p>\n One of the biggest challenges we faced as a team on BostonGlobe.com<\/a> is understanding how readers consume digital news. We needed to present the breadth of our content in an appealing way to maximize recirculation (percentage of users who read more than one article per visit).<\/p>\n For a recent internal test of a new \u201cSports\u201d vertical on Boston Globe.com, we conducted mobile testing on 17 candidates internally sourced from 13 different departments. Participants were asked to complete prompted tasks and open-ended questions. We also asked them to rate the usefulness, effectiveness, and desirability of features.<\/p>\n For this project we needed a way to test functionality around new navigation that uses Red Sox, Patriots, Celtics, and Bruins logos as navigation elements (to drive more circulation between team fronts) and a fly-in menu bottom with additional articles to drive recirculation.<\/em><\/p>\n RITE testing<\/a> (Rapid Iterative Testing and Evaluation) was also used to test alternative designs. We then collected reactions (feature and product level) in the form of a product reaction scale and adjectives, which we visualized in a word cloud.<\/p>\n Mobile sports section features tested and ratings by feature<\/em>\f<\/em><\/p>\n The results of our internal testing (example tables below with ratings for each feature) showed a number of areas for UX improvements.<\/p>\n A fully functional prototype used for testing was built to test the major features and variations in the recirculation button and gauge reaction before product launch.<\/em><\/p>\n The rating table covers the three UX benchmarks: usefulness, effectiveness and in this case. I chose desirability instead of satisfaction because asking participants if something is satisfying sets a sub-standard bar for the quality of the feature or product. Instead, we aim for delight.<\/em><\/p>\n Word cloud used to visualize common words used in qualitative user feedback.<\/em><\/p>\n Findings<\/p>\n Differences in your sample size between tests can skew your results. Don\u2019t trust the data alone to inform design decisions. Proper due diligence includes checking your qualitative data and adjectives for a complete product perspective.<\/em><\/p>\n Feedback word cloud<\/em><\/p>\n Participants found it difficult to recognize the difference between an article they chose to read and a truncated article that served as a landing page (truncated article with recirculation articles at the top of the page). Furthermore, if a reader had chosen the same article as the featured article the hero images were the same so there was some confusion understanding the differences between these pages. If a reader clicked the section logo it would take them to the same article only truncated. Participants explained they expected to see a team landing page which from a UX perspective makes sense because participants didn\u2019t automatically understand that the article was serving as the landing page. To them, it just looked like another article.<\/p>\n Positive to negative word association based on adjectives in user feedback.<\/em><\/p>\n Like many companies that are dealing with time constraints ultimately it was decided to release an MVP to get the product out and see how our readers responded. Once the design was out in the wild our team waited and looked at a two-week post-launch analytics report and a feedback form that was launched at the same time.<\/p>\n What we found was a bit disheartening when we discovered that exit rates increased on sub-section fronts. While the results were not what we had hoped for they still served as an important lesson to test as early as possible. The later you test, the more difficult it becomes to adapt and pivot.<\/p>\n The report validated many of the concerns from our testing results. For example, the recirculation button was clicked on less than 1% of articles and our testing showed that participants didn\u2019t find the recirculation button a valuable feature and many didn\u2019t understand or use positive adjectives to describe their experience. 3\/17 people understood this feature at first glance and 81% or 22\/27 had negative comments about the feature.<\/p>\n Luckily, as we launched iterations of the recirculation feature, qualitative feedback improved alongside the quantitative feedback. The language participants used to describe features started changing. Words like \u201csimple\u201d and \u201ceasy began to spring up which also corresponded with the ratings scales for gathering quantitative feedback.<\/p>\n The secret is maintaining momentum.<\/p>\n Think about testing like an iterative sketch. Your test won\u2019t be perfect \u2013 but remember, the clock is ticking. Test your prototypes with coworkers that possess varying and often overlapping needs\u2014just like your real users.<\/p>\n While the minimum is 5<\/a>, I try to use common sense.<\/p>\n Our team recently reduced the participants from 25 to 15 without loss in the quality of insights. We also recruited across different business segments to give the test as much variability as we needed.<\/p>\n Besides A\/B testing, we run internal tests on nearly all features in our product roadmap. The length of tests depends on the complexity of the problem. We normally run testing in blocks \u2013 for this project, we ran our test over three days.<\/p>\n Now, we typically start testing with lo-fi prototypes during concept exploration. We iterate almost immediately after the first test to prepare for our second round. We may test up to three different iterations in the first round.<\/p>\n For example, we may test a button with an icon with one participant, and one with text on the next participant. If a design is glaringly problematic, simply omit it from the testing as soon as a pattern supports your assumption.<\/p>\n Most of the time, issues will surface after the first few participants. We also found it best to debrief after each session instead of hours later when you\u2019re trying to recall important points from notes or recordings.<\/p>\n In closing, let\u2019s review the benefits of internal testing:<\/p>\n For advice on testing with users, I highly recommend checking out The Guide To Usability Testing<\/a> by UXPin<\/a>.<\/p>\n You\u2019ll find useful tips, as well as guidelines and pitfalls you may encounter along the way.<\/p>\n","protected":false},"excerpt":{"rendered":" A UX case study based on the recent redesign of the Boston Globe website. <\/p>\n","protected":false},"author":116,"featured_media":14843,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,21,6],"tags":[],"class_list":["post-14828","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","category-usability-2","category-ux-design"],"yoast_title":"","yoast_metadesc":"Having a hard time getting access to your customers and time is short? Test with internal users! Discover this agile approach to user feedback!","acf":[],"yoast_head":"\nWhen Does Dogfooding Make Sense?<\/h2>\n
\n
Eating Our Own Dogfood<\/h2>\n
![\"understanding](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image00-15.png\")
<\/p>\n![\"Rapid](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image01-13.png\")
<\/p>\nThe Results Of Our Internal Test<\/h2>\n
Navigation<\/h3>\n
![\"Navigation](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image14.gif\")
<\/p>\nNavigation Ratings<\/h3>\n
![\"Navigation](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image04-1.jpg\")
<\/p>\n![\"Word](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image11.png\")
<\/p>\n\n
Infographics<\/h3>\n
![\"Infographics](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image09-3.png\")
<\/p>\n![\"Differences](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image08.jpg\")
<\/p>\n![\"Feedback](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image13.png\")
<\/p>\nFindings<\/h3>\n
\n
\nfor advertisements.<\/li>\n<\/ul>\nRecirculation Button<\/h3>\n
![\"Recirculation](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image12.jpg\")
<\/p>\n![\"Recirculation](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image02-1.jpg\")
<\/p>\n![\"Recirculation](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image10-2.png\")
<\/p>\n\n
![\"Positive](\"https:\/\/www.uxpin.com\/studio\/wp-content\/uploads\/2016\/08\/image03.jpg\")
<\/p>\nHow We Pivoted<\/h2>\n
How to Run Insightful Internal Testing<\/h2>\n
Minimum number of coworkers to test<\/h3>\n
Frequency of Internal Testing<\/h3>\n
Takeaways<\/h2>\n
\n