The Metrics Café: looking at the tension between flexibility and uniformity

Common Approach has recently discovered the Metrics Café, a great resource for thinking about impact metrics. We love what that author, Laura Budzyna, created. Let’s explore this metaphor and “unpack” (pun intended!) why we think “bring your own lunch” is the best way to serve up impact data.

This short post from MIT D-Lab and the associated 16-page PDF guide offers a clear and concise description of how the tension between flexible and uniform approaches to measurement plays out between funders and grantees. At Common Approach, we have many different ways to explain this tension, but Laura Budzyna absolutely nails it. The metaphor of a café is accessible and intuitive. This will undoubtedly help anyone involved in making decisions about what should be measured and how to both frame the problem and review the options available to solve it.

The Metrics Café framing quite accurately captures the systemic nature of the issues that arise when different people and organizations need different things from impact measurement. Funders need to aggregate and compare across a portfolio, while social purpose organizations need to closely manage their own unique work.These often-conflicting needs cannot easily be reconciled. The table on page 13 of their PDF guidebook summarizes four “trade-off” scenarios for choosing indicators:

• “Prix Fixe” offers the least flexibility. Social purpose organizations (SPOs) all report on the same set of standard metrics chosen by the funder.
• “A La Carte” offers some flexibility, with a “menu” of standard metric options created by the funder that SPOs can choose from.
• “Made to Order” increases the flexibility, with funders and SPO grantees working together to determine what will be measured.
• “Bring Your Own Lunch” offers full flexibility; SPO grantees come with their own metrics.

For each, the framing explores who they think should use each approach, the tradeoffs involved, tips for each scenario, and other pros and cons.

The case for “Bring your own lunch”

At Common Approach, we believe that centring the needs of social purpose organizations (SPOs)— namely, grantees and investees—results in the best impact (and impact data measurement) for everyone. We think “bring your own lunch” serves them best.

It’s important to recognize that impact measurement is a practice that is associated with power. “What gets measured, gets managed.” One of our guiding principles is that impact measurement standards should place this power with social purpose organizations and those they serve rather than focusing primarily on the impact measurement needs of foundations, grantmakers and impact investors.

While the ‘Prix Fixe” scenario may appeal to funders with low capacity for customization or technical assistance or to grantees without their own metrics that are looking to benchmark, it is not necessarily “better than nothing.” All measurement has an opportunity cost. Prix Fixe measurement, which dictates a set list of metrics, risks allocating resources to measurement that does not align with the SPO’s strategic goals and that does not create much impact value relative to the effort invested.

But perhaps we can have our proverbial cake and eat it too. In the points below, we explore what might be possible if the Common Approach standards were ingredients in the options offered by the Metrics Café menu.

The row “who should use it” in the guidebook chart assesses the funder’s and grantee’s readiness and capacity for measurement and evaluation. The Common Foundations are a valuable tool for Funders and SPOs to assess their current practices relative to an acceptable minimum standard. This may help them focus their capacity-building efforts where this is needed, reducing the need for “low capacity” options.

Common Foundations can create a shared understanding that the essential practices of impact measurement are being done. For example, if an SPO completes the Common Foundations self-assessment, confirms their impact measurement meets the standard, and shares these results with the funder, it may be easier for funders to put their trust in the SPO’s chosen measurement and evaluation approach, even if it is unfamiliar to the funder.

When it comes to funders’ needs around analyzing the data gathered, we believe the amount of funder resources required for the more flexible scenarios could be “medium” or even “low” if there were better data interoperability. The ability to perform aggregation of data across a portfolio, compare grantees, and have a streamlined data system could be made easier with the Common Impact Data Standard and Common Form in place.

The Metrics Café framing does a great job of exploring and unpacking the tension between flexibility and uniformity caused by the different needs of funders and the SPOs they fund. Our vision is that the Common Approach standards will empower SPOs to align their impact measurement practice with their purpose while still meeting the reporting needs of funders. Rather than making all the metrics the same, our aim is to provide a way to make sense of the difference. This will eliminate the need for less flexible approaches such as Prix Fixe or À La Carte. The overall result will be impact measurement that supports better impact rather than hinders it.

📣 Follow us on LinkedIn and Twitter.

📬 Subscribe to our mailing list below.

📌 We recently launched our newsletter, the Common Approach bulletin! Read the first issue here.

Published Dec 5, 2022

More like this