Define the big why
How to appropriately scope your measurement
Understand the space in which you’re working
In building a compiled indicator, you start by identifying the big why that you work in. In government, the big why is often vast: it can be anything from veteran health to space exploration to supply chain management and beyond. In statistics literature, this is known as the “theoretical framework”1 of your compiled indicator. For our purposes, we’ll be more informal and call it the “big why.” The big why isn’t a literal statement about what you do every day; it’s the general realm or category in which you work. This big why is why your intervention was designed and deployed in the first place.
Getting the right “big why”
The big why that you’re seeking is not the largest category of your work, but it’s not the most immediate, daily description of your work, either. Because the goal of HCD measurement is to understand the impact of your intervention, the big why should be big enough to encompass your intervention in its wide context, but not so large that the effects of your intervention would be lost, like a drop in the ocean. It needs to be neither the overall mission of your agency, nor the individual mission of your team.
Is your big why too big?
One way to know if a big why is too big is if it’s essentially just a vague, simple description of what an entire agency or major agency branch does. For example, if your description of your big why is “veteran health” or “housing,” those are simply too large and vague to enable you to build a defensible, replicable, and verifiable compiled indicator for them. In fact, they describe entire agencies: the Department of Veterans Affairs and the Department of Housing and Urban Development. You’re thinking too big, and need to break your big why down.
Is it too small?
On the other end of the spectrum, you’re thinking too small if your big why literally describes what your specific business unit or team does. Examples of these might be “teaching human-centered design to federal employees,” or “establishing principles for survey writing.” While these problems are interesting and complicated in themselves, their answers should be described by your group’s strategic plans, and do not need data or team input. In this case, you need to expand your big why out.
You can use activities like the Wall of Problems and Why is that important? to help you find the right scale for your measurement tool.
Activity: Why is that important?
This process is similar to the Five Whys process from the Discovery phase. While it’s a useful method in discovery research, you can also use it on yourself or with your team to help you back away from the literal, day-to-day problem-solving you do, and see the bigger problem space in which you work.
Set the stage
Ideally, sit down with another teammate (or, if you’re alone, talk out loud throughout this process). One person will be the interviewer, asking the why questions, while the other person will answer to the best of their ability.
Gather materials
Have a notebook, phone, or computer handy to jot down notes, or record or transcribe the session.
Take action
Activity rules:
- The interviewer can only ask “Why is that important?” and let the interviewee answer.
- The interviewee will try their best to answer without resorting to “Because my boss told me to” or “because that’s the law”-type answers.
- After at least 5 rounds of questions, either the interviewer or interviewee can try to wrap the conversation up in a big why statement. One way to do this is to roll up the important portions of most or all of the answers the interviewee has provided into a logical statement starting with the phrase “So the big why here is…”. See an example below.
Example activity from the Office of Patient Advocates, Veterans Health Administration, Department of Veterans Affairs:
Interviewer: Why is your intervention important? Interviewee: Because it allows veterans to make non-clinical reports in medical centers. Interviewer: Why is that important? Interviewee: Because sometimes veterans need to register and resolve non-clinical concerns, complaints, or compliments in the medical centers. Interviewer: Why are non-clinical complaints important?
Interviewee: Because the veteran experience with VHA is more than just clinical. Interviewer: Why is it important that the Office of Patient Advocates represents that non-clinical experience? Interviewee: Because then the Office can help VHA serve the whole veteran. Interviewer: Why is serving the whole veteran important? Interviewee: Because the whole veteran is what VHA and VA support. Interviewer: So the big why here is that: The Office of Patient Advocates supports the veteran experience in VHA and VA to serve the whole veteran through registering and resolving non-clinical veteran concerns? Interviewee: Correct
Activity: Wall of Problems
Set the stage
You’ll need a whiteboard. For the digital version, get access to whiteboarding software. For analog, find a big open wall that’s clear of pictures, bookshelves, etc. You can also use a big expanse of windows. It’s best to do this activity with at least one other person so you can talk it out, but you can also do it solo.
Gather materials
For digital, all you need is the whiteboarding software. For analog, gather index cards or sticky notes, markers, and some masking tape.
Take action
-
Write down your first idea(s) for your big why. Remember: this should be a picture in which your intervention’s impact can be detected, but it shouldn’t simply describe your intervention’s activities.
-
Draw a line across the center of your workspace.Digital: Use the line tool in your software. Analog: Use the masking tape.
-
Label the space above this line something bigger than you’re going to measure to find impact. Something like “entire agency mission” could work. Label the space below the line something smaller than your intervention itself. Something like “daily/weekly/quarterly goals” could work. Name the middle line something that’s in between the vast upper part and the tactical lower part. Something like “general category of work” or “department mission”.
-
This general category of work or department-level mission is the scale at which you can build a compiled indicator that can be described in a few indicators, which themselves can be described in a few datasets. Since you’re just beginning to build your measurement tool, finding this balance point will require you to use your best judgment and the expertise you have in this complex problem space. You don’t have to be perfectly right the first time, but by going through this activity, you’ll find that you have more information about the right scale than perhaps you thought you did.
-
Consider the space above and below the middle line. Where does your version of the big why land? Is it near the top/bottom of the screen, or the ceiling/floor? Is it actually near the middle line? Place your version of the big why where you think it fits best.
-
Now, rewrite your big why to either expand it out if it’s too small, or scale it down if it’s too big. If the problem is too big, ask yourself: What do I mean by each word in my big why? What vague words (e.g., change, experience, strategy) can I better define to scale my big why down?
If the picture is too small, ask yourself:What am I really talking about in this big why? Why are concrete terms like “improve” or “implement” or “decrease” important? Further defining these terms will help you scale the picture up.
-
Place your new big why statement on the wall. Does it still have very vague or very concrete terms? If so, place it above or below the center line.
-
Repeat the actions above until you can define the terms in your big why easily, in simple, single sentences; then you’re probably just about right in your scale. Place your big why on the centerline and congratulate yourself for naming your big why.
Footnotes
-
OECD/European Union/EC-JRC (2008), Handbook on Constructing Composite Indicators: Methodology and User Guide, OECD Publishing, Paris,22 Aug 2008. 17. ↩