The Metrics Café: looking at the tension between flexibility and uniformity

Common Approach has recently discovered the Metrics Café, a great resource for thinking about impact metrics. We love what that author, Laura Budzyna, created. Let’s explore this metaphor and “unpack” (pun intended!) why we think “bring your own lunch” is the best way to serve up impact data.

This short post from MIT D-Lab and the associated 16-page PDF guide offers a clear and concise description of how the tension between flexible and uniform approaches to measurement plays out between funders and grantees. At Common Approach, we have many different ways to explain this tension, but Laura Budzyna absolutely nails it. The metaphor of a café is accessible and intuitive. This will undoubtedly help anyone involved in making decisions about what should be measured and how to both frame the problem and review the options available to solve it.

An orange book rests on a cafe table next to a cup of coffee and a plant. On a cover is a line graphic representing data and measurement.

The Metrics Café framing quite accurately captures the systemic nature of the issues that arise when different people and organizations need different things from impact measurement. Funders need to aggregate and compare across a portfolio, while social purpose organizations need to closely manage their own unique work.These often-conflicting needs cannot easily be reconciled. The table on page 13 of their PDF guidebook summarizes four “trade-off” scenarios for choosing indicators:

  • “Prix Fixe” offers the least flexibility. Social purpose organizations (SPOs) all report on the same set of standard metrics chosen by the funder.
  • “A La Carte” offers some flexibility, with a “menu” of standard metric options created by the funder that SPOs can choose from.
  • “Made to Order” increases the flexibility, with funders and SPO grantees working together to determine what will be measured.
  • “Bring Your Own Lunch” offers full flexibility; SPO grantees come with their own metrics.

For each, the framing explores who they think should use each approach, the tradeoffs involved, tips for each scenario, and other pros and cons.

The case for “Bring your own lunch”

At Common Approach, we believe that centring the needs of social purpose organizations (SPOs)— namely, grantees and investees—results in the best impact (and impact data measurement) for everyone. We think “bring your own lunch” serves them best.

It’s important to recognize that impact measurement is a practice that is associated with power. “What gets measured, gets managed.” One of our guiding principles is that impact measurement standards should place this power with social purpose organizations and those they serve rather than focusing primarily on the impact measurement needs of foundations, grantmakers and impact investors.

While the ‘Prix Fixe” scenario may appeal to funders with low capacity for customization or technical assistance or to grantees without their own metrics that are looking to benchmark, it is not necessarily “better than nothing.” All measurement has an opportunity cost. Prix Fixe measurement, which dictates a set list of metrics, risks allocating resources to measurement that does not align with the SPO’s strategic goals and that does not create much impact value relative to the effort invested.

But perhaps we can have our proverbial cake and eat it too. In the points below, we explore what might be possible if the Common Approach standards were ingredients in the options offered by the Metrics Café menu.

The row “who should use it” in the guidebook chart assesses the funder’s and grantee’s readiness and capacity for measurement and evaluation. The Common Foundations are a valuable tool for Funders and SPOs to assess their current practices relative to an acceptable minimum standard. This may help them focus their capacity-building efforts where this is needed, reducing the need for “low capacity” options.

Common Foundations can create a shared understanding that the essential practices of impact measurement are being done. For example, if an SPO completes the Common Foundations self-assessment, confirms their impact measurement meets the standard, and shares these results with the funder, it may be easier for funders to put their trust in the SPO’s chosen measurement and evaluation approach, even if it is unfamiliar to the funder.

When it comes to funders’ needs around analyzing the data gathered, we believe the amount of funder resources required for the more flexible scenarios could be “medium” or even “low” if there were better data interoperability. The ability to perform aggregation of data across a portfolio, compare grantees, and have a streamlined data system could be made easier with the Common Impact Data Standard and Common Form in place.

Funders and social purpose organizations can adopt these standards by choosing a Common Approach aligned software, which will allow them to exchange data even when using different softwares. Our current Pathfinder Pilot project is currently testing this with different networks of social purpose organizations across Canada. If successful, the Pilot will demonstrate a way for SPOs to report to multiple funders without the need to manage separate reporting softwares, tools, or accounts. This will save them time and effort and in theory, will result in more robust and complete impact measurement.

Funders and investors will be able to report on the impact of their portfolios by using the Common Framework, which is still in development. The Common Framework will allow for the various bespoke indicators being used by the various SPOs being funded to be rolled up into useful analysis.

The Metrics Café framing does a great job of exploring and unpacking the tension between flexibility and uniformity caused by the different needs of funders and the SPOs they fund. Our vision is that the Common Approach standards will empower SPOs to align their impact measurement practice with their purpose while still meeting the reporting needs of funders. Rather than making all the metrics the same, our aim is to provide a way to make sense of the difference. This will eliminate the need for less flexible approaches such as Prix Fixe or À La Carte. The overall result will be impact measurement that supports better impact rather than hinders it.

Join the Common Approach community to stay up to date on our efforts to make impact measurement better, and help shape impact measurement standards!

📣 Follow us on LinkedIn and Twitter.

📬 Subscribe to our mailing list below.

📌 We recently launched our newsletter, the Common Approach bulletin! Read the first issue here.

Published Dec 5, 2022

More like this

Common Approach receives a Tipping Point Fund grant

Common Approach receives a Tipping Point Fund grant

Common Approach is thrilled to be receiving a grant from The Tipping Point Fund on Impact Investing! This recognition of Common Approach’s role in improving data, metrics and measurement will support our continued work in developing and promoting impact measurement standards that support flexible, sharable data.

read more
Case study summary: Using the Common Impact Data Standard and Common Framework

Case study summary: Using the Common Impact Data Standard and Common Framework

To illustrate how these two Standards work, we collaborated with Carleton University and Rally Assets on an example using an investment portfolio focused on affordable housing. This case study demonstrates how these Standards can be used to represent impact data, allowing for a nuanced analysis across a portfolio while reducing the burden of reporting.

read more

Get email updates.

The Common Approach is community-driven! Get updates on standards and see upcoming events and webinars. 

Merci!

Inscrivez-vous à notre infolettre

L'Approche commune est propulsée par la communauté! Recevez des mises à jour et des invitations aux webinaires. 

Merci!