The F word of product design research methods: focus groups

Getting a group of users or customers together to provide feedback sounds like a good idea, but is it worth it? (Hint: probably not.)

None of the well-designed products will ever survive the response of a focus group. Well-conceived and -designed products are the result of courage not fear… Good design needs courage not focus groups!

—Massimo Vignelli

In his book, The Vignelli Canon, the legendary designer devotes a page to the topic of marketing. He acknowledges that marketing is “integral to the production process,” but the remainder of the passage delves into his distaste for focus groups. Vignelli distinguishes between two types of marketing: aiming to satisfy either people’s needs or their wants. It’s the latter approach that leads companies down the risk-averse path of simply asking people what they want or whether they like something rather than proceeding boldly based on “vision, courage, and determination.”

Putting aside the noble endeavor of courage in the face of fear for now, let’s examine what some UX gurus have to say about the value of focus groups. In The Use and Misuse of Focus Groups — dated January 1, 1997 — Jakob Nielsen says they can be a “powerful tool” to help you “discover what users want,” but warns against relying on them too much for product validation. Because of the dynamics inherent in a group setting, the session inevitably turns into a product demo. And demos, Nielsen says, are “fundamentally different from actually using the product” because participants are being guided instead of figuring out what to do next on their own.

In her seminal book on human-centered design, Designing for the Digital Age, Kim Goodwin agrees focus groups can be helpful when used appropriately — namely when you’re building something completely new and don’t know your potential audience very well. (Try not to be in this situation too often.) They’re not to be used as a substitute for interviews and usability testing, where you can dive more deeply into individuals’ needs and behavior. It’s this substitution for other methods that leads to the term Goodwin notes some researchers use to refer to focus groups: the F word.

“When they don’t [have the right information], every little design decision becomes a struggle.” —Jared Spool

Since focus groups come with so many pitfalls, what alternatives are there for product discovery and validation? Fortunately, there’s another F-word method. Jared Spool of User Interface Engineering calls field studies the best tool to discover user needs. While they can require more time, cost, and coordination up front, nothing beats them for the insights they can generate and the innovation they can inspire. Here’s a taste of what you can expect from field studies that you can’t get from focus groups:

• See the whole process they actually do (not what they say they do) and hear them talk about it in their words while doing it.

• Experience the complete context they’re in, including the environment, supplemental tools, situational factors, and other things that happen in real life.

• Pick up on unspoken nuances and small details that people don’t even think about and wouldn’t mention if they were just describing what they do.

Spool spends a lot of time coaching design teams and notes that when teams find themselves guessing about their users’ behavior, “every little design decision becomes a struggle.” With observational data based on real people in real situations, you can call a ceasefire on stakeholder opinion battles waged with nothing but airy assumptions.

In my own experience, I’ve seen the suggestion for focus groups crop up once in a while, but I’m typically able to steer the research activities toward more fruitful methods like interviews, usability studies, or some type of observation. For validation, usability testing with individuals will nearly always yield better insights and more actionable results than what you’ll get from a group. More people means more ways to derail a session. Group dynamics are unpredictable, so you’re never really sure how people’s responses are influencing each other.

That said, some input from actual users can be better than none. So if access to a focus group is all you can get, take it, but go into it with a plan for tightly managing the agenda and the flow of the conversation to get as much contextual information as you can. People are better at articulating their actual problems than possible solutions. So rather than try to gauge the feasibility and desirability of specific ideas, spend time with customers having them describe (or better yet, show) what they’re doing today and what their challenges are.

Of course, Vignelli would probably disagree with the assertion that anything good could come from focus groups. Writing in 2010, he recalled that he learned his lesson 50 years earlier: focus groups and good design don’t mix.

---

For more on Massimo Vignelli and his fiery canon, check out this episode:

The Vignelli Canon: Massimo fires shot across bow of vulgar design (#2)