Limits of Impact Measurement: Care

by Kate Ruff

Impact is about change. It is defined as change. Impact is a change in social or environmental outcomes as a result of an organization’s activity. Impact measurement measures – or uses data to estimate – change.

Change is at the heart of how we understand social-purpose work. At the Common Approach, we talk about the five essential practices for how to do impact measurement, the first one being to plan your change.  The tools that many organizations use to do this—logic model, theory of change, outcomes map—are all about change. The terms widely used in these change-based tools, terms like output and activity, come from manufacturing. The metaphor suggests that people can be processed through a program and delivered at the other end changed.

Most social-purpose organizations strive for change, but not all. For those social purpose organizations that don’t strive for change, impact measurement can require bizarre contortions that may not be worthwhile.

In my research, I draw on a literature about ethics of care to talk about a group of charities and other social purpose organizations that are not focused on change. A focus on care means attending to things like being present in a way that expresses connection. There are some obvious examples of caregiving charities – like hospices and long-term care facilities. There are also less intuitive ones like one that helps youth by pairing them with caring mentors. For an organization like that, one might say that care is an activity that leads to outcomes for the youth like higher self-esteem, better performance in school and higher graduation rates. That is a very impact-focused view. Even if care did not lead to those outcomes, the care is valuable and important as an end in itself.

Impact measurement is not very effective at helping managers, funders and other stakeholders understand care.  Because impact measurement (or outcomes measurement or results-based measurement) is such a thing right now, care-based organizations have contorted themselves to fit the impact model. 

One contortion is to frame care as an activity or process that produces change, as in the youth mentorship example above. An organization that might initially have described itself as first-and-foremost about care changes itself to be first and foremost about self-esteem, school performance and graduation rates. Illustrated in an impact model, like theory of change, care is demoted from the end in itself to a means to an end. Care risks being presented as worthwhile only insofar as other outcomes are produced. This entirely misses the point of care.

Another contortion is to imagine a world without care. Impact is measured as the difference in outcomes between an imaginary world without care and the actual world in which the charity offers care. A supportive housing charity that I worked with measured their impact on things like hospital stays and police encounters by comparing their cared-for population to an imaginary (statistically informed) world where no care is given to the same population. Year after year the same several dozen people are imagined as being newly rescued from homelessness, even though they have been stable, and cared for, for decades. This is exactly how impact measurement gurus recommend that the impact of a caring charity be assessed. This contortion misrepresents sustained care as newly initiated care (imagined as newly initiated each year). It finds worth in newly initiated care but cannot at all speak to the worth of sustained, ongoing, consistent, human, compassionate care. 

At the Common Approach, we believe that the main benefit of undertaking impact measurement is for improved impact management: helping organizations get better at doing good. The tools of impact measurement are not great at helping organizations that are focused on care get better at delivering care. Worse, the tools can encourage organizations to take their eyes off care and focus on auxiliary outcomes like graduation rates and reduced hospital stays.

 If you know of great tools for measuring and improving care, please be in touch! 

More about the limits of impact measurement:

• Consequentialism
• Colonialism

Published on Sept 12, 2021

More like this