Impact evaluation of a portfolio of private investments

Of the Common Approach’s four Standards, the Common Impact Data Standard and the Common Framework are the most technical. They can be tricky to understand at first!

All four Common Approach standards work together to help make one-of-kind accounts interoperable, meaning data is easily shared, aggregated and used across systems.

• Common Foundations help to ensure the quality of the data by setting a minimum standard for impact measurement.
• Common Form helps to ensure that there is good contextual data about the organization measuring its impact.
• Common Impact Data Standard provides a way to label and link data, making it easier to share and interpret, particularly across measurement standards.
• Common Framework is a set of principles and rules that provide guidance on how to roll up dissimilar indicators into a measure of a group’s impact (such as a portfolio or a network of organizations).

The Common Impact Data Standard and the Common Framework do the heavy lifting in making impact data interoperable. To illustrate how they work, we collaborated with Carleton University and Rally Assets on a case study using the example of an investment portfolio focused on affordable housing. You can read Common Approach’s research note on the case study if you’re interested in digging into the details.

At Common Approach, we believe that the world becomes a better place when the voices of people whose lives are most affected are reflected in impact measurements.

To get to that better world, social purpose organizations (SPOs) need to be the ones articulating indicators and metrics: not standard setters, not investors, not grantmakers. Rather than making all the metrics the same, we need a way to make sense of the difference.

Empowering SPOs to select their own impact measurement tools and articulate their own indicators and metrics will result in higher quality data and enable data-informed decision-making. It will save resources—time and money—and, eventually, in theory, lead to better impact.

In the case study, the value of interoperability is explained through the lens of an impact fund manager who needs to measure impact across a portfolio of organizations.

The case study examines a portfolio that invests in affordable housing assets across North America.

The investments include upgrades to existing buildings and investments in new buildings. This means that the impacts relate to both the quality and quantity of affordable housing. The portfolio includes units for rental and units for sale. This means that affordability is understood broadly as including affordable ownership and affordable rent. The investor wants to measure the impact of the portfolio. Specifically, the investor seeks to speak holistically about the change in affordable housing outcomes that the portfolio contributes to.

Each asset holder provided us with the indicators they used to measure impact, resulting in a list of 83 indicators. While all organizations are focused on affordable housing, this is expressed in subtly different ways (# of affordable units; % affordable housing units available). Housing has effects on mental health and the environment; the asset holders measured different specific effects of housing.

For example, the data gathered included these metrics:

• Affordable housing: # of affordable housing units
• Affordable housing: # of affordable units
• Affordable housing: % of affordable housing units available
• Improvements in their mental health: % of tenants say they have hope for the future
• Renewable energy generation: MWh of renewable energy generated
• Reduction of greenhouse gas emissions: tonnes of CO2 equivalent (CO2e) avoided
• Reduction of greenhouse gas emissions: tonnes of CO2 equivalent sunk
• Waste management: tonnes of waste diverted from landfill
• Water efficiency: volume (in cubic meters) of water saved
• This variation, even among very similar organizations, is typical.

The wide variation in outcomes and measures is a phenomenon to be embraced rather than a problem to be solved. Indicators are most effective at informing impact management when they are closely aligned with strategy and operations. Organizations with different strategies and operations should have different indicators in order for them to manage for better outcomes.

To turn 81 dissimilar outcomes and indicators into a portfolio-level impact analysis, the team at Carleton University and Common Approach used the Common Impact Data Standard to represent the data and assigned each item to a theme (see sidebar and full case study for details). Using a flexible framework, the data was then aggregated by theme.

Themes are higher-order ideas or constructs. They may be drawn from existing frameworks such as the UN Sustainable Development Goals (SDGs) or the IRIS+ system. Alternatively, a fund or network could develop its own set of themes.

Using the Common Impact Data Standard, an outcome can be associated with several themes. For example, one outcome might contribute to more than one SDG. Or, an outcome might be associated with an SDG, an IRIS+ theme, and a theme within a custom framework. This is important because it allows social purpose organizations to connect their unique impact measures to the frameworks of their networks, investors, and/or funders.

In the affordable housing case study, the indicators were linked to SDGs and IRIS+ themes. Of the 9 indicators mentioned above, only the first three, those related to affordable housing, connect to SDG 11: Sustainable Cities and Communities. The indicators on greenhouse gas emissions connect to SDG 13: Climate Action.

By contrast, when rolling up to the IRIS+ thematic categories, all these outcomes are within the theme “Real Estate”.

Using software aligned with the Common Impact Data Standard, a portfolio manager should be able to simply click a button to assemble the portfolio of impact by SDGs or IRIS+ or whatever other framework the Investment fund is using. We say “should” only because we rely on our software providers to make the Common Impact Data Standard user-friendly and to generate ways to visualize the data.

The final step is to use the Common Framework to create a “messy roll-up” of the data for the portfolio-level impact report.

By “messy roll-up”, we mean imprecise and incomplete. The things that make it messy are subtleties in the data. In the case study, examples of these subtleties are:

• Several of the Affordable Housing providers identified the outcome “Affordable Housing” but used different indicators: “# of affordable housing units”, “# of affordable units”, and “% of affordable housing units available.”
• The indicator “# of affordable housing units” is used to refer to ownership as well as rental accommodation.
• The indicators have different units: counts of people, counts of houses, % of available housing and % of spending.

The SDG 11 roll-up looks like this:

The IRIS+ theme looks like this:

These roll-ups show that the same data can be used to speak about the impact of a fund according to two different frameworks—in this case, the SDGs and IRIS+. It illustrates how dissimilar indicators can be aggregated. The purpose of a messy roll-up is to give readers a sense of the portfolio impacts.

The affordable housing case study demonstrates how the Common Impact Data Standard and Common Framework can be used to represent impact data, allowing someone like an impact fund manager to undertake a nuanced analysis across a portfolio.

When used with impact management software, the Common Impact Data Standard makes it easier for social purpose organizations (SPOs) to enter and share data. Not only is there a place for everything and flexibility to record the data they have, but also the ability to share between softwares. The ability to share information between softwares means that the Common Impact Data Standard will reduce SPOs’ reporting burden.

By grouping data based on the Common Impact Data Standard, an impact fund manager can create a thematic roll-up that allows for a messy roll-up into themes. Using this flexible framework, impact fund managers can articulate the impact of a portfolio of dissimilar indicators, thereby reducing the reporting burden.

Portfolio-level actors can take the following actions now to support better, more relevant impact measurement:

• Most importantly, support and encourage investees to use impact measurement tools and measures that suit their impact management needs.
• Nudge investees on their journey by inviting them to complete the self-assessment.
• Use an impact measurement software that is aligned with the Common Impact Data Standard.
• Use a thematic framework to make sense of one-of-a-kind impact accounts rather than impose uniform measures on your investees.

Want to learn more? See Common Approach’s project log on Research Gate or Rally Asset’s report on this work.