Establish and use compiled indicators

What is an indicator?

Once you’ve named your “big why” (as discussed in the previous section), it’s time to establish the indicators that will tell you whether and how well your intervention is having the effect it was designed to have. Terms you might have heard instead of “indicator” include: dimension, angle, facet, variable, component, or signal. All these terms express the same idea: that the big why is made up of lots of smaller parts.

Since most human-centered projects are complex and multi-faceted, you cannot rely on just one indicator to give you an accurate picture of what’s happening. Instead, you’ll look across a set of different indicators to understand and present the most accurate possible view of the effectiveness of your intervention.

What makes a good, useful set of indicators?

1. They should combine to give you as much of the big why as is feasible.
2. They should exist in balance with each other; if one takes, another one usually gives.

Example: Global Happiness Index

As an example of this balance, take the indicators of purchasing power parity and generosity in the Global Happiness Index,¹ a composite indicator of happiness in countries around the world and reported upon annually in the Global Happiness Report. Purchasing power parity (PPP) is a metric establishing how much people can buy with their money. Generosity is defined by contributing to charitable causes. Generosity has to do with spending money, but the product you “purchase” through your contribution is intrinsic, not material, which contributes to balancing the consumptive skew of the PPP calculation. Combining indicators like purchases and generosity together present a more accurate view of what makes up human happiness than a single indicator alone ever could.

Method 1: Rule of Eight

A central rule of building composite indicators is to keep the number of indicators you establish to a manageable, navigable number. If you have too many indicators, it becomes unwieldy to manage them, and you cannot reliably track movement in your composite down through indicators and into datasets.

It should take you no more than an hour to brief an audience on your indicators. Such a briefing breakdown might go like this:

• Gather the audience and make introductions - 10 minutes
• Provide descriptions and explanation, answer a few questions - 40 minutes (5 minutes per indicator)
• Share conclusions and wrap up - 10 minutes

That’s a lot of information to present in such a short time. That’s why we recommend using eight or fewer indicators in your composite, and you could responsibly have fewer. It depends on the complexity of your big why, and the amount and density of your underlying datasets. If you find yourself with so much data or so many indicators that the measurement tool is becoming unwieldy, consider breaking your measurement work into two different but related measurement projects, with indicators that you can use side by side. That way, you can give each dataset and indicator the attention it needs to ensure that your work is defensible, replicable, and verifiable.

Two indicator examples

The World Happiness Report

The big why that Gallup’s World Happiness Report takes on is to assess the happiness of people in countries around the world.² Because it’s very complex to actually understand the happiness of a whole country, they look at relevant indicators that are related to someone’s happiness to try to gain a big why of overall happiness. To do this, they use the following indicators, which they call variables (2024):

1. Gross domestic product (GDP) per capita
2. Social support
3. Healthy life expectancy
4. Freedom
5. Generosity
6. Corruption

The EDX Index

Another example is from GSA’s Enterprise Digital Experience (EDX) initiative. This initiative was launched to “improve the digital experience for government customers.”³ To accomplish this, the team fostered an ecosystem of products, services, and systems, and constructed a compiled indicator called the EDX Index, to measure the overall effectiveness of their work. The indicators used to measure this work (see the footnotes for the supporting/related policy and law) include:

1. Accessibility⁴
2. Performance/search engine optimization (SEO)[⁵]
3. Required links⁶
4. User behavior⁷
5. U.S. Web Design System (USWDS) implementation⁸
6. Customer-centricity,⁹ which is its own compilation. Elements include:
   1. Website stated purpose
   2. Stated audience
   3. Repeatable customer feedback mechanism
   4. Ability to action
   5. Ability to measure effectiveness

In both the World Happiness Report and the EDX Index, some of the indicators, like GDP and Performance/SEO, can be measured directly, while others, like freedom and customer-centricity, cannot. Still others, like healthy life expectancy and accessibility, can be measured somewhat directly, but not fully.

That mismatch between measurement methods in indicators is okay. Because big whys are multi-faceted, there will always be a variety of directly and indirectly measurable items in your set of indicators. But taken together, these indicators can show whether the current state of the big why in question is on the road towards success or failure.

Activities to establish indicators

Here are some activities to help you establish indicators.

Activity (part 1): Define those terms!

Every organization has a vocabulary of words and phrases they use to describe their work. Sometimes these words are crystal clear, and sometimes they are clear as mud. In this first activity, you will identify both clear and unclear key terms, and rewrite the unclear terms until they make sense.

Set the stage

For a digital version of this activity, you need access to whiteboarding software. For analog, a pen and paper or actual whiteboard will work.

Gather materials

Gather your organization’s most recent strategic plan. Gather reports and other public papers from other organizations also working in the same space as your big why.

Take action

1. Identify the outline, section headings, or goal statements in each strategic plan or report, and copy them down.
2. Circle each vague or meaning-packed term or phrase that might be hinting at an indicator.
   • Examples from the World Happiness Report include “GDP” and “social support.”
   • Examples from the EDX Index include “drive GSA’s use of design thinking” and “continue our role as connectors.”
3. For each term or phrase, rewrite it in plain language that can be understood as a stand-alone concept. You might need to try several different ways of writing it.
   • If it helps, consider which phrasing is still vague. How can you add precision to help an outside audience understand? Where can you explain jargon or take out acronyms?
4. Now, circle vague or meaning-packed terms or phrases in each of your rewrites.
5. Repeat steps 1-4 until you’ve got a collection of terms and phrases that you find straightforward and outline most plainly the dimensions of your big why.

Activity (part 2): Create your collection of indicators

Armed with your list of words and phrases that clearly and succinctly describe the dimensions of your big why, you’re now ready to create your collection of indicators.

Set the stage

In a different part of the whiteboard or on a new sheet of paper, write down your big why in the center.

Gather materials

Copy over your collection of indicator terms or phrases.

Take action

1. Begin placing your indicator terms and phrases near the big why. Which ones can you directly connect to the big why? Which ones seem farther away? Discard those that seem farther away.
2. Keep doing this until you have no more than eight indicator terms or phrases with a clear connection to your big why.
3. Consider them alongside each other. Ideally, you want to be able to place them around the big why in a complete circle, so that each one is unique, but they all balance.

If you find that some indicators overlap, or are all clustered around the same topic area of the big why, go back to step one and consider whether any of the indicators you discarded might be included instead.

Repeat steps 1-3 until you’ve found a collection of indicators that are unique, directly related to the big why, and seem to surround it in a balanced way.

Remember: This collection of indicators is a “…construction (that) owes more to the craftsmanship of the modeller (sic) than to universally accepted scientific rules for encoding…”¹⁰ , so concentrate on making a readable, balanced, and logical set of indicators, as opposed to trying to find the “perfect” set. It’s better to measure something than nothing, because measuring will get you some information, however imperfect, and you can tweak your compilation as you learn more.

Up Next