Based in the Bay Area with 250+ employees and $161 million in venture capital funding, Sumo Logic serves some of the top enterprises in the world.
The company’s analytics platform visualizes more than 100 petabytes of data per day, helping businesses harness the power of machine data to streamline operations.
In 2015, Sumo Logic hired their first UX team comprised of design leaders, interaction designers, visual designers, and UX architects. One of those designers was Daniel Castro – a 20-year design and engineering veteran in the enterprise space.
Now a Design Director, Daniel spoke with us about the tactics and processes required for Agile UX to succeed in the enterprise.
Sumo Logic’s UX readiness model.
What common mistakes have you seen companies make as they fit the user-centered design into Agile?
The first time I heard of Agile was when I was designing at Verizon. One of the Agile evangelizers who developed the manifesto presented a really detailed deck to the company.
I remember the response from the whole design team was “This whole process is never going to work.” At that time, the concept of a UX “Sprint 0” just didn’t exist. We were just expected to start designing and developing immediately.
So, as with a lot of processes, you need to pick out what applies and what doesn’t. Since the early 2000s, we’ve seen so many different flavors of Agile that sometimes it almost feels like we’ve just created micro-waterfall.
In fact, a developer here at Sumo Logic actually has a sign in his cube that said “Drink coffee, do stupid things faster and with more energy!”. Agile is the coffee for your design process – it is a fantastic tool for efficiency, but it doesn’t deter you from doing dumb things.
The biggest mistake is thinking that Agile will help you move in the right direction. You need to build in a series of checks and balances to ensure you’re building the right thing.
How do you strike that balance between user-centered design and Agile processes?
You take advantage of Agile’s collaborative nature to expose everyone to the value of user-centered design.
User-centered design is naturally iterative, but in reality you only have a few shots at revisions before you frustrate engineers. They’ll get cranky because the requirements keep changing.
To make the most of the iteration sprints you can realistically run, one of our most successful tactics is ensuring everyone understands that each design change is supported by usability research and testing.
In our design process, the first step is to validate whatever problem a product manager presents with customer interviews. Once we’ve spent time digesting the feedback in the form of customer journey maps, our design team clears out their calendar for ~2 days. We email the whole company and advise certain stakeholders to be on-call for feedback.
For those two days, we’ll hold a highly focused “Swarm” session in which we dive deep into the divergent and convergent design around the problem. The Swarm also helps us align with the product vision for the ensuing sprints and the business goals. We might also create additional generative research questions with developers and PMs, which then breaks down to a series of tasks you test with users with alternating sprints of testing and design.
That list of tasks is the contract binding developers and designers together for each sprint. You reach a solid middle ground where developers aren’t pushing back on every change, and designers aren’t lost in blue-sky thinking.
You get more buy-in for unexpected iterations because the context changes completely. Designers aren’t forcing the developer’s hand – the customer is guiding everybody along.
Sumo Logic’s Agile UX process.
How do you approach documentation in the Agile UX process?
You can’t completely throw out your documentation just because you’re Agile. You need to create smarter documentation.
Documentation doesn’t need to live in a document. You can be nimbler, like Slacking your team a screenshot of your whiteboard session. Don’t obsess over the form or format – the singular goal of documentation is to ensure that everyone agrees to the success criteria.
For example, if I need to convey more detail after a whiteboard sketching session, I’ll create a quick UXPin prototype and share with the team instead of marking up a huge product specs document.
Sumo Logic prototype created during a discovery phase for their Unified Logs Metrics product.
In the platform, I can literally just drag and drop objects from the dozens of libraries to convey the concept that our team discussed. Add some notes to the prototype and share with the team, and now everyone has naturally created documentation together that actually helps us make better decisions down the line.
Understand the collaborative spirit of documentation, not the tactics.
Does this natural form of documentation help you mold your product strategy?
For Sumo Logic, the UXPin platform really shines at helping us create collaborative visualizations to help PMs and developers better understand both the horizontal requirements (overall feature set) and vertical requirements (depth of each feature).
Even if the PMs and developers aren’t physically present, we can send them a project link for their comments. They suddenly have clear visibility into how the scenarios play out for each persona, and how it all comes together to form the overall customer experience.
Collaborative prototyping helps the team understand the end-to-end solution and how it breaks down into chunks of work for sprint planning. Very early on, the team can start to understand what Release 1, 2, and 3 looks like and how they all map back to the core vision. Everyone gets a better sense of what they should and shouldn’t build.
The prototype naturally joins the product strategy and design tactics together. Even though your final product might not look anything like the prototype, you’ve created a living representation of the core success criteria. You can’t really see that with paper documentation.
How has prototyping helped you define success criteria in Agile UX?
So, after we’ve held the “Swarm” sessions and conducted user research, we’ll sketch through all the feedback. Once the sketches start to convey flow, we’ll dive into a lo-fi prototype.
The prototype flows are the heart of the design. They inform all the tasks, which of course define success for the product.
With a prototype, you can tell confidently tell a developer “Hey, we tested this design and it’s ready for you”. After the first formal build, you can compare the coded design back to the prototype and list of usability tasks.
Design swarm at Sumo Logic.
For each sprint, how do you build the components without losing sight of the overall strategy?
You need to set success criteria in each sprint on two levels.
Let’s return to the bicycle analogy.
First, you need the horizontal success criteria: the person needs to sit down, hold down the handlebars, pedal, and move from A to B.
Secondly, you need vertical success criteria: the frame design will be aluminum, the pedals are a certain size, etc.
Not only does the team need to understand both levels of success criteria, but you also need to ensure you’re testing at both levels.
You can’t test features in isolation. You need to test the entire end-to-end solution as early as possible. Otherwise, you get lost in vertical paths. You build the most amazing tires but Billy can’t ride the bike because the tires don’t fit on the frame.
Of course, prototyping helps us get that perspective at each step in the process. It’s a living contract for both sets of features. Anyone can have high-level conversations about strategy, or dive deep into individual components.
What Agile principles have you found useful to user-centered design?
Agile gives you a concrete structure for planning ahead. It brings designers back to earth and prevents them from wanting to design forever.
Agile has also democratized the design process so that even sales and customer success can contribute to the product.
We’re able to break up work, encourage open contribution, and move away from the “lone designer” mentality.
On the other side of the spectrum, how do you prevent too much design collaboration (e.g. design by committee)?
That’s always a tough issue.
It starts with the designer’s attitude towards collaboration.
First, they need to see themselves as a facilitator for gathering, shaping, and testing input.
Secondly, consider holding 1:1 sessions with vocal stakeholders. During design reviews, let them air their thoughts, but if they dive too deep into prescriptive advice, tell them you care so much about their ideas that you want to dedicate focused time to discuss.
Sometimes, you’ll find that stakeholders might even reconsider since they realize they need to separate personal opinions from facts. It seems like a paradox, but sometimes being overly open actually helps people better accept dissenting opinions.
For example, we were working with a product manager who wasn’t exposed much to the design team. We had trouble communicating our approach, and he was adamant about a certain feature. We simply responded with “We don’t think it will work, but let’s test it and see”.
Coming out of that usability test, the original idea was invalidated, but we found pieces of it that could work in a different context. So even if the testing validates the designer’s idea, you can’t tell others “I told you so”. You need to convey a spirit of “We’re willing to work with you”.
Finally, designers can try reframing the conversation by saying “Look, we’re trying to make you look amazing. We want to release a product that will make people want to write about your features.” versus “Look, my idea is right because…”
All of a sudden, something clicks and the other stakeholder realizes that the designers are on their side – and sometimes that means saving them from themselves.
Do you follow a component-based design system (e.g. Lego-block approach to design)?
We’re currently in the middle of a reboot project that will help us work even closer along those lines.
Luckily, we’ve been blessed with UI developers who sit inside the UX team. As such, designers and developers can define the major components of a pattern library together.
Once you have that common design language that breaks down to patterns and elements, everyone does less redundant work. Designers can focus on solving large business problems, and developers feel empowered to use vetted components instead of asking designers to verify everything.
We aren’t 100% there yet. But after this reboot project, we’ll be much closer. You can imagine how exciting it will be to just go into a shared environment and grab the exact component to match your context of use.
How would you improve the Agile UX process?
I’m all about expectations, whether they’re big or small.
The best process starts with collaboratively defining the requirements for success. From there, you can chunk out all the pieces into your sprints.
Within each sprint, you also set expectations for each chunk of work. What are the goals? What are the discovery questions? What are the tasks? How will we know customers love this?
You repeat the process, ensuring that everyone on the product team always knows the answers to the four questions above.
Download the 99-page Definitive Guide to Integrating UX and Agile.