Ep. 03 – Measuring Impact


We all know that good intentions are not enough. A sexy idea that makes a lot of buzz and attracts all kinds of support is no guarantee for real sustainable change. So how can organizations ensure that they really are achieving their mission? Are they humble enough, and ambitious enough, to adjust their trajectory if needed?

There are methodologies and tools that can help organizations to deal with these issues. Social Impact Measurement is a rapidly growing field, and the mission of this episode is to explore it. The subject can become a bit technical – beware – but it dives deep in the soul of social entrepreneurship.

With the participation of:

Marguerite Evenaar, member of se.lab and founder of Evenaar & Partners.

Raúl Contreras, businessman, economist and social entrepreneur, founder of Nittua (a Spanish platform working on an inclusive economic ecosystem) and an Ashoka fellow.

Maxime Pekkip, financial manager at CRESUS, a French NGO working on the prevention of overindebtedness.

LINKS, REFERENCES, FURTHER DISCOVERIES

Read here about the trouble with Scared Straigh, a juvenile delinquency prevention program in the U.S.

Learn about carbon offset programs on this episode of NPR’s Planet Money Podcast.

Listen to Nobel laureate James Heckman talk about Facts, Evidence and the state of Econometrics on this episode of Econtalk Podcast.

Norman, Wayne and MacDonald, Chris (2004) “Getting to the Bottom of the Triple Bottom Line”, Business Ethics Quarterly.

Allen Paulos, John (1988) Innumeracy: Mathematical Illiteracy and Its Consequences.

In this episode, I stayed at the level of the organization in my quest to understand what social impact measurement is about. CRESUS is using measurement tools, NITTUA is developing new tools, my colleague Marguerite is coaching organizations in the use of those tools. But the underlying idea that value is not being properly accounted for can also be approached from a macroeconomic perspective. A lot of thinkers are proposing alternative or complementary indicators of wealth, for instance. Gross domestic product – GDP – doesn’t tell the whole story about a country’s wealth. In 2008 the French government put together a Commission on the Measurement of Economic Performance and Social Progress, lead by economists Joseph Stigliz, Amartya Sen and Jean-Paul Fitoussi. Their mission was to look for alternatives to GDP. On its report one year later, the commission recommended broadening the scope of economic performance to include measures of quality of life, inequalities and well-being, not forgetting sustainability and environmental conditions. Based of these recommendations, the OECD launched its Better Life Index. It’s an interactive tool that you can even customize in order to compare countries according to their performance on the indicators that you consider most important. It may not be the ultimate index of well-being – there’s enough criticism about the nature and weight of its indicators – but it’s a step forward in this challenging task of showing a mirror to ourselves.

In the context of impact measurement on a macro level, I could also have evoked the discussion about the effects of foreign aid on developing countries. Dambisa Moyo’s first book, Dead Aid: Why Aid Is Not Working and How There is Another Way for Africa (2009) argues that government-to-government foreign aid has harmed Africa and should be phased out. The mainstream vision, though, maintains that aid works, but it’s simply not enough – we need to give more. In response to this major question about the efficacy of the world’s actions to do good where is it most needed, some new trends emerged, like Effective Altruism that promotes screening and evaluation of organizations according to their efficacy and invites all donors to be proactive in the choice of the organizations they donate to.

The work of French economist Esther Duflo fits in this mindset. She is co-directing the M.I.T.’s Poverty Lab and strongly advocates the use of randomized control trials to assess the efficacy of programs.