Gather your data

You identified the datasets that will help you understand the current state of your big why. Now, you need to go get the data. There are three possible scenarios for this:

1. You already have access to the data;
2. You know where the data lives but you need access to it; or
3. You don’t know if the data exists, or where to find it.

The first two scenarios are relatively straightforward: you either already have the datasets you need, or you know who to ask. So let’s focus on the third scenario, when you don’t have the datasets that you need.

Finding data that you don’t have

You built your measurement tool from the top-down, by identifying your big why, breaking it down into indicators, then designating the datasets that describe the indicators. At this point, you might look at the datasets you identified and realize you don’t actually have the ones you need. You may not know where to look for them, or even whether they exist at all.

This is a tough problem, but you can figure it out. You have three options: chase, correlate, or create.

• The chase option: Try to track down your designated dataset. This means figuring out how and where to look for the data. Start with the desk research section of the discovery operations guide. There is also a brief refresher below on how to do this.
• The correlate option: Try to find datasets that are thematically similar to your designated dataset, that could stand in for the missing dataset. You can also look for a dataset that’s diametrically opposite and can act as a foil of your original designation. We’ll use a version of the Six Thinking Hats activity¹ to do this.
• The create option: In this create option, you’ll have to go back out of your current work and start from the discovery phase. You might even need to entirely reframe the thinking that backs your project from first principles. We see this situation in the loop between the measure phase and the discovery phase in the HCD bubble diagram. The discovery concept and discovery operations guides can help you get started with this work.

Chase

Chasing a dataset means you’ll have to do research to find it. After learning about desk research in the discovery phase guide, ask yourself the “W” questions:

1. Who might own the data I need? Are there organizations or individuals I should seek out?
2. What are the general categories in which this data lives? Is there a parent to this dataset? Could I search for bigger ideas and then try to winnow down to the dataset I need?
3. When is it likely that this data was collected? Is this a new idea, or an older one? If it’s an older one, could the jargon have shifted? Should I use more outmoded terms in my search?
4. Where could this dataset be held? What’s the nature of this data? Is it likely to be found in bibliographies or in citations in published papers? Could it be classified and held in secure locations?
5. Why would someone collect this data? Are there general motivations I can search for that might lead me to it?

Correlate

Six Thinking Hats is a creative analysis framework in which participants wear six different colored hats that represent different ways of thinking about a problem or topic. The six ways of thinking are:

1. White hat: Facts, objectivity
2. Blue hat: Structured, strategic, and long-term thinking
3. Green hat: Creativity and innovation
4. Yellow hat: Optimism and looking at the positive possibilities
5. Red hat: Subjectivity and instinct
6. Black hat: Caution and risk management; can be a foil for the yellow hat.

The Six Thinking Hats isn’t quite the right fit for correlating datasets, but a modification of the activity is. To start identifying correlated datasets, ask yourself the questions below. When you find one or more that speak(s) to your indicator as strongly as your original designee did, you can stop your search.

1. What are facts about this indicator?
2. Why did you choose this indicator and not another one?
3. What are positives or strengths you see in this indicator?
4. What are the risks or weaknesses you see in this indicator?

Each of your answers can point to different, possible datasets, including ones that might not support the indicator you crafted. If you find that there are many datasets that oppose or point in slightly different directions than your indicator’s themes, question why that is. Loop back to the previous section and evaluate your indicators: do they really drive towards a balanced big why? Should you edit your indicator(s)?

If, however, you find that one of the strengths of your indicator is held by a dataset that you already have access to or know where to find, document that correlation and continue on.

Create

You might have reached the conclusion that the data you want doesn’t exist yet. In that case, you might have to go out and create a new measurement by looping back to the Discovery phase. You can use surveys to help you transition into discovery mode.

Surveys as a hint for where to look

When you send out a survey, you may receive quantitative, quantitative but subjective, or qualitative answers – depending on your questions. But either way, you receive a rather limited amount of information. For example, if using the popular Likert scale to ask about product use, you could get returned data that looks like this:

• 521 reported that they never use the product
• 358 reported that they rarely use the product
• 234 reported that they sometimes use the product
• 129 reported that they often use the product
• 48 reported that they very often use the product.

While this information is helpful to know what’s happening, it does not explain why. Without knowing the why behind the what, it can be hard to know what changes to make to a future iteration.

Complementing survey data with other methods

One thing you do know is that a good number of people rarely or never use the product – and that is telling you who you should try to learn more from. With that information, you can start problem framing your new discovery work and collect the data you need.

One thing to note about both the chase and correlate options—they take time. You’re deep into complexity now. You’ve built an elegantly balanced measurement tool, and you’re understandably in a hurry to use it! But be confident that you designated the dataset(s) that best support your indicator(s); don’t break the balance just because you’ve been at this for a while. Don’t abandon the solid foundation to your compiled indicator simply because finding a dataset is hard. Allocate some time to this, but timebox it by using a stop rule.

Stop rules

Stop rules are predetermined times at which you will stop your work and simply move ahead with what you have. This is much like when you’re taking a test, and when time is up, you simply stop working. Because they impose a hard stop to work, stop rules help us avoid getting caught up in indecisiveness or mental churn. Only deploy them in situations where you have a good grasp of the knowledge required to make a decision, or in cases where emotional or physical fatigue could impair decision-making.²

Your stop rule for your chase or correlate processes should be determined by how much time you have to devote to the task. Be realistic: by this time, you should know the landscape of your intervention pretty well. If you simply cannot find reliable data and you feel the decision to stop is defensible, given the time and effort you have put in, stop work, document your decision in your project notes, and either loop back to the discovery phase for further research, or revisit your indicator. You might need to change it given the data you do have access to. This changes the part of the big why that your compiled indicator will illuminate, but that doesn’t necessarily make your measurement wrong; it makes it different. If you revise or change your indicator wholesale: Document! Document! Document! Making it possible to follow your logical decisions is the best gift you can give yourself and your colleagues in this space.

Up Next