Research brief: Accountability for social impact

There are some research papers that I keep coming back to. This is one of them. It states the reality of so many social enterprises very well. It catalogues the challenges that social enterprises face when measuring impact (see Figure 1 below). And it finds that despite this long list of challenges, social enterprises continue to strive to find ways to measure their impact. In short, this research paper finds that social enterprises make impact measurement work for them by using bits of different tools and a mishmash of data, all the while holding a very clear-headed view of the limits of impact measurement.

To me, this research paper is further evidence that it is a fool’s errand to require social enterprises to adopt rigid, standardized impact measurement methods. While not stated in the paper, my takeaway is that what is most needed to improve impact measurement for social enterprises is an easy way for them to share the data they do have and to support that data sharing in a manner that equips funders and investors to similarly create their own patchwork or “bricolage,” as the authors say, so that funders, too, can make their own impact measurement work.

Summary of the research paper

Source: Molecke, G., & Pinkse, J. (2017). Accountability for social impact: A bricolage perspective on impact measurement in social enterprises. Journal of Business Venturing, 32(5), 550–568.

Literature Review: Social impact measurement: a contested management practice

In their literature review, Molecke & Pinkse describe what academic literature has already documented regarding the “frictions inherent in social impact measurement between multiple stakeholders involved in a social enterprise with regard to their understanding of the nature of social impact and how it can be measured” (p. 552):

1. Lack of definition clarity around impact, particularly as distinguished from outputs and outcomes. Sometimes outcome and impact are conflated as one idea.
2. The difficulties of measuring impact and attributing impact to a specific actor. In reality, programs don’t unfold the way causal models, logic models or theories of change imply.
3. The difficulty of translating “rich, experiential information into simple, parsimonious measures of social impact” (p 552). The authors connect this concept to the mobility of the measurement such that accounts that “capture the experiential richness, variance and flexibility of the social entrepreneurs’ interpretations”… are not “easily transferable and interpretable for funders” (p 552).
4. There are many tools and approaches. Funders require measurement but rarely prescribe a specific formal methodology. Different people prefer different methods for different reasons.

Theory: A bricolage approach to social impact measurement

The authors use the theoretical term bricolage which emphasizes that “entrepreneurs do not simply accept the limitations set upon them, but instead show critical agency in enacting their environment” (p 552). Material bricolage refers to the “process through which people use and combine the various resources they have “at hand” as a means to find workable—if typically imperfect—approaches to a wide range of problems and opportunities’ (Baker, 2007 p 697)” (p 553.) Ideational bricolage “refers to the process by which organizations recombine elements of older myths to create new myths serving new functions (Baker, 2007 p 679)��� (p 554).

Method

The authors interviewed 22 social enterprises whose primary mission was to aid the extremely poor in developing countries and who all used market-based funding mechanisms to start and grow their enterprises. Examples of the interviewed enterprises were those selling straw bale housing in Pakistan, chocolate in Belize and solar base stations in Tanzania.

The authors transcribed interviews and coded statements about impact measurement (first-level categories). They then abstracted those codes into second-order themes and grouped them into what they call “aggregate dimensions” (see Figure 1).

Findings

The authors found two things. The first is that social entrepreneurs use four main critiques to delegitimize formal impact measurement approaches. As shown in Figure 1, those four critiques are i) social impact measures as incomplete, ii) social impact measures as irrelevant, iii) social impact measures as immeasurable and iv) social impact measures as imprudent.

Importantly, the authors found that despite these critiques, social entrepreneurs continue to do and believe in impact measurement. “Interestingly, our data did not suggest that social entrepreneurs used the four delegitimizing critiques to avoid social impact measurement. The social entrepreneurs largely agreed with the purpose and goals behind impact measurement and expressed a strong belief that measuring impact in a formal manner was important. … what we initially mistook for a lack of methodology among social entrepreneurs was actually a competing approach to evaluation that forsook strict adherence to formal methodologies [and instead] emphasized the value of opportunistically collecting at-hand experiential data and ideas that might prove useful.”

Discussion

The authors suggest that the four critiques of formal social impact measures are used to create space for the bricolage. “These lines of critique were in part used to serve social entrepreneurs prior to the choice and implementation of a methodology… in attempts to fend off formal methodologies, saving them from practices which they feared would waste effort and resources. We also observed these critiques being used during and after the evaluation to justify adjustments, to abandon impact measurement practices, or to manage evaluation results that failed to reflect their own understanding of social impact.”

Implications for Common Approach to Impact Measurement

The goal of Common Approach is to center the impact measurement needs of social purpose organizations within the impact measurement ecosystem. Part of how we do this is to help purpose-driven organizations better mobilize the data that they already have and use. We believe that impact measurement is high-quality only when it is also useful. We believe impact measurement is useful when organizations are free to use this “bricolage” or mishmash approach; when they can make do with what is at hand. This “making do” is not a cop-out. Like Molecke & Pinkse’s (2017) findings, Common Approach observes that social purpose organizations are keen to make their impact measurement better and that they do so when they can.

We are aware, however, that when organizations are free to undertake this bricolage approach, their funders (investors and grantmakers) are ill-equipped to speak to the impact of their portfolio-level impacts. As concerns grow regarding social finance greenwashing and impact washing, it is in our—society’s—best interest to hold social finance and private foundations accountable. That means that the funders and investors need their own impact measurement; they need to do their own bricolage.

Common Approach’s four flexible standards were created to solve this problem: how can impact measurement systems center the social purpose organizations while also giving funders and investors the tools they need to assess portfolio-level impacts? The Common Impact Data Standard makes it easier for organizations to share the data that they do have. Combined with the Common Form, these Common Approach standards offer structured, flexible data formats that give investors and funders the freedom to do their own bricolage. The Common Framework aspires to articulate the bricolage (we are still working on this!). The Common Foundations sets a minimum standard to ensure that the bricolage at least surpasses some minimum floor for data quality.