Menu
Qualitative vs. Quantitative UX Research
When I’m presenting our measuring UX and ROI seminar, I sometimes get questions about the differences between quantitative and qualitative research. In UX, we often use qualitative research to gather insights or observations about users and products or services. Popular qualitative UX research methodologies include interviews, qualitative usability testing, and field studies. With qualitative research, our goal is often to discover problems or opportunities in the user experience, to investigate why something is happening or to find ways to fix a problem. For example, maybe I conduct a round of interviews to understand why a feature isn’t as popular as I expect it to be. Maybe through the course of my interviews I learn that our users don’t understand the point of this feature or how they’re supposed to use it. I could use that information with my design team to help us decide whether this feature should be phased out or how we should redesign it. Because our goal is to identify problems and find ways to fix them, it’s okay to test with a small number of users.
Our rule of thumb for qualitative research is five to eight users per user group. To explain why, we can think about a pothole in the street. If you watch five cars drive down the street and each one keeps hitting this pothole, you know it’s a problem. You don’t need to watch 45 more people drive into that pothole before you decide that it should be fixed. When we’re conducting quantitative UX research, instead of gathering insights, we’re gathering numbers that describe some aspect of the user experience. We refer to those numbers as UX metrics. Popular quantitative UX methodologies include, analytics, A/B testing, surveys, and quantitative usability testing. You might wanna use a quantitative methodology if you’re trying to determine the priority or scale of a problem. What proportion of my users are impacted by this particular problem, for example? You might also use a quantitative study, particularly A/B testing if you wanna compare alternative design options and use data to inform your decision.
Finally, quantitative research is also great for benchmarking the UX of a product or a service. Essentially that means tracking your user experience over time to make sure that you’re improving in the ways that you expect. You can also use that data to calculate return on investment. Often with quantitative studies our goal is to compare things. Maybe we’re comparing our current version of our product against our older version. Or our product to our competitor’s product. Or maybe we’re comparing two different alternative designs for a specific layout.
A major benefit of quantitative research is that it gives us the ability to determine whether the difference between two numbers is statistically significant. In other words, to know that that difference is not likely due to random chance. That’s part of the reason that we need larger sample sizes for quantitative studies. If we don’t collect the right amount of data, we may not have a reliable result. There are different recommended minimum sample sizes for each quantitative methodology, so it’s important to check that minimum whenever you are beginning to plan a quantitative study. Qualitative and quantitative research give us different kinds of information about the user experience. The best strategy is to use a variety of methods in your UX practice.
Copyright Disclaimer
Under Title 17 U.S.C. Section 107, allowance is made for “fair use” for purposes such as criticism, comment, news reporting, teaching, scholarship, and research. Fair use is permitted by copyright statute that might otherwise be infringing.
