What is data-driven design?

Data-driven design innovation is a new and important topic. For designers, understanding and learning data-driven design can better help us make design decisions. This article explores how data affects design decisions with examples, explains the difference between data-driven and data-responsive, and introduces a variety of data collection methods.

In this article, we’ll take a deep dive into what data-driven design is and why it drives better user experience, while also understanding why it can help us make more realistic decisions.

As designers, we need to accept and embrace real-world metrics and use their magical power to adjust the way we do things. — Jared Spool

We are currently faced with a huge amount of data from user interactions. Although there are a lot of tools to help us understand the products we design, we are designing products for humans. Therefore, as designers, we need to analyze human behavior and design based on our understanding of human interactions. This will help us make better design decisions based on data.

The data we collect from users can help us make better design decisions. By analyzing and understanding the data, we can design better user experiences. We can make better design decisions based on the results of qualitative and quantitative studies of interaction behavior.

Data helps us explore how users use our products and how designers can optimize the user experience based on data.

The data we collect can help us in four dimensions.

What is data-driven design?

Design is supported by the accumulated data research results. During the design process, design decisions are based on data and user behavior research. Fine-grained user experience design includes evaluation research on the experience itself. Only data can map out valuable evaluation conclusions. User behavior and feedback help product designers verify their findings, assumptions and evaluation results.

The collation and analysis of measurement data can reveal important information. — Peter Drucker.

This data can help product teams understand their target users, discover user pain points, discover new trends, support data-driven design, and ensure the correct direction of the team. User data can directly drive the improvement of business results. Improving user experience through data-driven design has been proven to be a practical method.

Why Data Driven Design?

When we designers design products, we survey, research, and observe our users. The only thing that can verify our assumptions in the product design process is data. If we conduct research without any data-driven decisions and make decisions based on intuition or experience, we are likely to waste money on ineffective or worthless design changes.

The performance of a product can be improved by using, verifying and analyzing user data, and it can also be intuitively reflected in the user data. Even the best designers in the world cannot predict user needs. Data-driven design is a process of learning from users, which ensures that user problems are solved. Meeting user needs is the first condition for product success, and all this requires data to drive design iterations.

Data-driven and data-responsive design

The above terms come from the book Designing with Data by Rochelle King, Elizabeth Churchill, and Caitlin Tan. This book helps us understand and clarify the different terms related to data-driven design and contrasts it with data-responsive design and data-aware design.

Data-driven and data-responsive are two different approaches. They are both based on data, and every decision made by the product team is evaluated and driven by data. The focus of data-driven is on data-assisted product optimization and efficiency improvement. Data-responsive gives us a broader development direction in the use of data. We can handle more than just quantitative data. A/B testing of different experiences or structured usability testing may fail in data-responsive.

Data awareness allows us to understand the breadth and limitations of data collection. We can judge which method is most appropriate for different problems. A data-aware team may find that decisions based on stakeholder workshops, user interviews, or even A/B testing research results are equally valuable.

Research from MIT’s Center for Digital Business shows that “companies that are in the top third of their industry in terms of making data-driven decisions are, on average, 5% more productive and 6% more profitable than their competitors.”

We found that increasing the number of things we measured or increasing the fidelity of the measurements didn't actually improve the accuracy of the results. The data didn't change dramatically based on good or bad performance. It just revealed a deeper level of complexity - good or bad performance involves more things. So the data is really just a metric, and we still need to rely on intuition. We still have to make judgments about the importance of causes. — Jon Wiley (Director of Immersive Design at Google)

How is the data collected?

There are many ways to collect qualitative and quantitative data. Many UX practitioners think that data is numbers, but this is a misunderstanding and a fallacy. In order to use data to drive design, we need qualitative and quantitative data. Quantitative data will tell you what actions users take when using our products. Qualitative data will tell you why they do so, and more importantly - how they feel about the overall experience. So, we need to collect both types of data when making design decisions.

1. Quantitative Data Collection Methods

The data types of quantitative data are numbers, people, things, time, and place. Quantitative data can show degree, but not cause. We can get a lot of quantitative data about website or application usage from Google Analytics, Google Tag Manager, Google Optimize and other measurement tools (such as Hotjar, Crazy Egg, Optimizely, Usertesting).

A/B Testing

A/B testing is also known as split testing. Hubspot defines A/B testing as:

In (an) experiment, you 'split' multiple test groups, test a series of variations, and determine which one performs better. In other words, you might show version A to half of your testers and version B to the other half.

When doing A/B testing, it is important to make sure that you only change one variable at a time and that the control group and the experimental group have the same number of people. You can show version A to half of our testers and version B to the other half. The main goal of the test is to compare different variables under the same conditions.

Analysis

Through parsing, we can know who came to our website, how they got there, how long they stayed there, and what they clicked. This type of data can effectively sort out a lot of valuable indicators, such as the average session duration of users, exit rate, etc. If you want to improve the conversion rate of your application or website, it is recommended to start parsing from pages with high user traffic, because they allow you to collect valuable data faster.

We can also use eye tracking tools, such as heatmaps. Heatmaps use eye tracking technology to understand where users focus on the screen. When heatmaps from multiple users show similar patterns, it indicates that the website or application is worth updating the content module or design iteration.

Research

User experience surveys are an important source of quantitative and qualitative data in user experience research.

A good survey requires carefully designed questions that are not leading and have a clear purpose. We should try to limit the number of questions (no more than 10-15) to prevent users from abandoning the survey questionnaire halfway.

2. Qualitative Data Collection Methods

Qualitative data explains why and how things happened. Why do different groups of users take different actions? Why do different pieces of content keep users on the site for different lengths of time? Qualitative data provides a perspective that helps us understand not only what happened, but also why and how things happened. We often collect qualitative data through user personas, journey maps, or empathy maps.

With clear qualitative data, we can create better user experiences and serve users more effectively.

User Journey/Flowchart

Creating a user flow model is a very helpful way to understand how users interact with your product. The information you gather from the user flow map can help you identify potential weak spots and lay the foundation for further research such as A/B testing or user interviews.

Competitive product analysis

This is an additional method to find out the weaknesses of competing products. Competitive product analysis is to find out the strengths, weaknesses or areas for improvement of similar products by researching competing products.

When conducting a competitive analysis, one must be careful. Simply imitating competitors is not an effective solution. Instead, it is better to use competitive analysis as a means of gaining inspiration. By understanding the competitors, we can learn from their strengths and overcome their weaknesses.

User Interviews

User interviews are an effective way to collect qualitative data from users. They can better identify the core issues of users through open-ended and closed-ended questions. At the same time, we should also limit the number of interviewees. Telephone or face-to-face interviews can obtain more in-depth data.

Social media and user feedback

Social media reflects users’ expectations of the experience. By understanding users’ common dissatisfaction, we can prioritize product needs. Understanding users’ comments and feedback on products can also provide more relevant information.

The method of collecting data needs to change according to the content and needs of the project. We may not need to study the data or do any calculations to effectively use data to drive design decisions. But as a product designer, when designing user experience, we all need to use existing tools to iterate and evaluate the design. Therefore, we should not just make blind decisions based on data, but be informed about the decisions we make.

No matter what our goal is, data-driven design can help improve product performance, increase conversion rates, and meet user needs. Through data-driven design, our designs can get a better return on investment. This also helps to increase the overall usage and reuse rate of the product.