Data storytelling with compiled indicators

Data storytelling with compiled indicators

Understand the value of evidence-based stories

When we use compiled indicators to understand how a human-centered project is working in the world, we’re crafting an evidence-based story. This is, in fact, “data storytelling,” a term you might have read about. Data storytelling is not some mysterious, specialist endeavor, but simply the use of data to understand a complex picture; in our case, a picture of impact and value.

Constructing a measurement instrument

Once you have the data you’re going to use to understand your human-centered project, you have a choice to make: you can either compile all the data into a single numerical value, producing a true compiled indicator, or you can keep some or all of the data separate, creating a dashboard of data for your stakeholders.

The upside to combining is that a single number is easy to communicate to senior leaders, your peers, and the general public. The downside is that it can obscure the data and methods you used to make your conclusions.

The upside to the dashboarding approach is that it keeps the data somewhat transparent, while the downside is that it’s harder to communicate your ultimate findings, and conversations might focus on your methodology, rather than your findings. As you can imagine, whether or not to combine data into a single number is a hot topic with statisticians around the world. Some folks advocate for aggregation, as it simplifies the complex information into a single, easy-to-consume number. Others oppose this aggregation, since that ease of consumption comes at the cost of transparency.¹ Like the OECD Handbook that we reference throughout this guide, we do not seek to resolve these perspectives; instead, we seek to give you the information you need to make informed decisions.

Recommendations on expression

For the generalist, we recommend keeping things simple; straightforward averages and weights will get you far in this work, and you can leave the more sophisticated calculations, like uncertainty analysis, to evaluation and data scientists. Always pair your combined indicator with an outline of your indicators and underlying data. Allowing a peek under the hood to explain the complexity underlying your indicator will help to avoid the perception that you might have hidden some data risks that could undermine your central argument.

We’ll go into detail about how to build your compiled indicator, plus simple storytelling aids, in the HCD measurement operations guide.

When to go to the experts

As a generalist, you can use combined indicators to both understand, and precisely communicate, the impact of your interventions. When you’re just getting started, you can assemble your composite index from this guide. As your work matures, however, you might reach a point where you have so much data that simple weights or averages plus storytelling aids might not be sufficient. For these situations, let’s examine some conditions under which you might consider sourcing additional support from evaluation, statistical, or data science experts.

For example, you might need support if you want to evolve compiled indicators into true composites to help determine major policy or legal changes, use the data for forecasting, or present findings for academic consumption. Because such compilations are computationally advanced, and reasonable applications of the data might result in major change, consider partnering with a professional group specializing in this type of research.

Another condition under which you might seek additional support is if the work you’re doing includes sensitive, human-subjects research. In this work, your compiled indicator carries the risk of flattening, occluding, or risking the privacy or rights of the humans in your study. Under this condition, we recommend reaching out to ethics and/or human-subjects research specialists, as well as statistical or data science specialists.

The exciting part of this work is that we’re seeking understanding. This might seem lofty, but it’s accurate. In the public sector, we don’t often look for large-scale truths such as the meaning of life, or the definition of happiness (consider philosophy if you’re interested in those issues).

Instead, we search to understand as completely as possible what happens when human-made interventions function alongside, and in contact with, other human-made designed things, and in combination with the random pressure and influence of time and context changes.

A compiled indicator is a wonderful way to succinctly use evidence to understand complex truth. They are a robust and tested best practice for understanding complex, human situations.

Up Next