Case Study Summary
Impact Evaluation of a Portfolio of Private Investments
Of the Common Approach’s four Standards, the Common Impact Data Standard and the Common Framework are the most technical. They can be tricky to understand at first!
All four Common Approach standards work together to help make one-of-kind accounts interoperable, meaning data is easily shared, aggregated and used across systems.
The Common Foundations help to ensure the quality of the data by setting a minimum standard for impact measurement.- The Common Form helps to ensure that there is good contextual data about the organization measuring its impact.
- The Common Impact Data Standard provides a way to label and link data, making it easier to share and interpret, particularly across measurement standards.
- The Common Framework (still in development) will be a set of principles and rules that provide guidance on how to roll up dissimilar indicators into a measure of a group’s impact (such as a portfolio or a network of organizations).
The Common Impact Data Standard and the Common Framework do the heavy lifting in making impact data interoperable. To illustrate how they work, we collaborated with Carleton University and Rally Assets on a case study using the example of an investment portfolio focused on affordable housing. You can read Common Approach’s research note on the case study if you’re interested in digging into the details.
Why is interoperability important?
At Common Approach, we believe that the world becomes a better place when the voices of people whose lives are most affected are reflected in impact measurements.
To get to that better world, social purpose organizations (SPOs) need to be the ones articulating indicators and metrics: not standard setters, not investors, not grantmakers. Rather than making all the metrics the same, we need a way to make sense of the difference.
Empowering SPOs to select their own impact measurement tools and articulate their own indicators and metrics will result in higher quality data and enable data-informed decision-making. It will save resources—time and money—and, eventually, in theory, lead to better impact.
In the case study, the value of interoperability is explained through the lens of an impact fund manager who needs to measure impact across a portfolio of organizations.
The impact data
The case study examines a portfolio that invests in affordable housing assets across North America.
The investments include upgrades to existing buildings and investments in new buildings. This means that the impacts relate to both the quality and quantity of affordable housing. The portfolio includes units for rental and units for sale. This means that affordability is understood broadly as including affordable ownership and affordable rent. The investor wants to measure the impact of the portfolio. Specifically, the investor seeks to speak holistically about the change in affordable housing outcomes that the portfolio contributes to.
Each asset holder provided us with the indicators they used to measure impact, resulting in a list of 83 indicators. While all organizations are focused on affordable housing, this is expressed in subtly different ways (# of affordable units; % affordable housing units available). Housing has effects on mental health and the environment; the asset holders measured different specific effects of housing.
For example, the data gathered included these metrics:
- Affordable housing: # of affordable housing units
- Affordable housing: # of affordable units
- Affordable housing: % of affordable housing units available
- Improvements in their mental health: % of tenants say they have hope for the future
- Renewable energy generation: MWh of renewable energy generated
- Reduction of greenhouse gas emissions: tonnes of CO2 equivalent (CO2e) avoided
- Reduction of greenhouse gas emissions: tonnes of CO2 equivalent sunk
- Waste management: tonnes of waste diverted from landfill
- Water efficiency: volume (in cubic meters) of water saved
This variation, even among very similar organizations, is typical.
The wide variation in outcomes and measures is a phenomenon to be embraced rather than a problem to be solved. Indicators are most effective at informing impact management when they are closely aligned with strategy and operations. Organizations with different strategies and operations should have different indicators in order for them to manage for better outcomes.
Analyzing the data
To turn 81 dissimilar outcomes and indicators into a portfolio-level impact analysis, the team at Carleton University and Common Approach used the Common Impact Data Standard to represent the data and assigned each item to a theme (see sidebar and full case study for details). Using a flexible framework, the data was then aggregated by theme.
Themes are higher-order ideas or constructs. They may be drawn from existing frameworks such as the UN Sustainable Development Goals (SDGs) or the IRIS+ system. Alternatively, a fund or network could develop its own set of themes.
Using the Common Impact Data Standard, an outcome can be associated with several themes. For example, one outcome might contribute to more than one SDG. Or, an outcome might be associated with an SDG, an IRIS+ theme, and a theme within a custom framework. This is important because it allows social purpose organizations to connect their unique impact measures to the frameworks of their networks, investors, and/or funders.
In the affordable housing case study, the indicators were linked to SDGs and IRIS+ themes. Of the 9 indicators mentioned above, only the first three, those related to affordable housing, connect to SDG 11: Sustainable Cities and Communities. The indicators on greenhouse gas emissions connect to SDG 13: Climate Action.
By contrast, when rolling up to the IRIS+ thematic categories, all these outcomes are within the theme “Real Estate”.
Above: These flow charts illustrate how the same information can be used with different themes from different frameworks. (Left: SDG 11; Right: IRIS+ theme “Real Estate”)
Using software aligned with the Common Impact Data Standard, a portfolio manager should be able to simply click a button to assemble the portfolio of impact by SDGs or IRIS+ or whatever other framework the Investment fund is using. We say “should” only because we rely on our software providers to make the Common Impact Data Standard user-friendly and to generate ways to visualize the data.
Impact at the portfolio level
The final step is to use the Common Framework to create a “messy roll-up” of the data for the portfolio-level impact report.
By “messy roll-up”, we mean imprecise and incomplete. The things that make it messy are subtleties in the data. In the case study, examples of these subtleties are:
- Several of the Affordable Housing providers identified the outcome “Affordable Housing” but used different indicators: “# of affordable housing units”, “# of affordable units”, and “% of affordable housing units available.”
- The indicator “# of affordable housing units” is used to refer to ownership as well as rental accommodation.
- The indicators have different units: counts of people, counts of houses, % of available housing and % of spending.
The SDG 11 roll-up looks like this:
The IRIS+ theme looks like this:
The Common Impact Data Standard is a way of labelling and organizing data. It is a collection of classes and properties and the relationships between them. It creates “a place for everything,” meaning all possible impact data. There are hundreds of classes and properties in the Common Impact Data Standard. This is one of many good reasons why the Common Impact Data Standard works best when it is implemented by impact measurement softwares rather than spreadsheets!
In the context of this report, the title Common Framework is a bit of a misnomer. What the report shows is a flexible framework designed around one portfolio. That isn’t a Common Framework. It will be Common when there is a principle-based approach to help all portfolios accomplish the roll-up in similar ways. But before we can create a Common Framework, we need need to learn from the fund-level flexible frameworks. The Common Framework will emerge slowly as investors and other portfolio-level actors (foundations, networks etc.) and Common Approach gain experience with these flexible frameworks and use them to create guidelines for how and how not to roll up indicators.
These roll-ups show that the same data can be used to speak about the impact of a fund according to two different frameworks—in this case, the SDGs and IRIS+. It illustrates how dissimilar indicators can be aggregated. The purpose of a messy roll-up is to give readers a sense of the portfolio impacts.
What we learned
The affordable housing case study demonstrates how the Common Impact Data Standard and Common Framework can be used to represent impact data, allowing someone like an impact fund manager to undertake a nuanced analysis across a portfolio.
When used with impact management software, the Common Impact Data Standard makes it easier for social purpose organizations (SPOs) to enter and share data. Not only is there a place for everything and flexibility to record the data they have, but also the ability to share between softwares. The ability to share information between softwares means that the Common Impact Data Standard will reduce SPOs’ reporting burden.
By grouping data based on the Common Impact Data Standard, an impact fund manager can create a thematic roll-up that allows for a messy roll-up into themes. Using this flexible framework, impact fund managers can articulate the impact of a portfolio of dissimilar indicators, thereby reducing the reporting burden.
Portfolio-level actors can take the following actions now to support better, more relevant impact measurement:
- Most importantly, support and encourage investees to use impact measurement tools and measures that suit their impact management needs.
- Nudge investees on their journey by inviting them to complete the self-assessment.
- Use an impact measurement software that is aligned with the Common Impact Data Standard.
- Use a thematic framework to make sense of one-of-a-kind impact accounts rather than impose uniform measures on your investees.
Want to learn more? See Common Approach’s project log on Research Gate or Rally Asset’s report on this work.
Join the Common Approach community to stay up to date on our efforts to make impact measurement better, and help shape impact measurement standards!
📣 Follow us on LinkedIn, YouTube and Instagram.
📬 Subscribe to our mailing list below.
📌 Check out the latest Common Approach bulletin!
Published on Aug 18, 2022
More like this
Public review of the Common Impact Data Standard
We often quip that standards are communities, not documents. To live up to this community-driven mandate, we ask the users of our standards and the broader public for feedback on existing and proposed draft updates to the standards.
Linking Impact Data
Understanding the impact of impact investments is a challenge for asset managers. An impact data ontology is a crucial part of the digital infrastructure that will be needed to improve impact measurement in the years ahead.
Choosing the right measurement software
We came into the Pathfinder Pilot understanding that social purpose organizations frequently face challenges with the software they use for impact measurement and reporting. We would like to share what we are learning about how SPOs wish to use software to support their impact measurement practices and the attributes that make software most useful to them.
