BLOG

Change vs. Impact

How can changes in the lives of microfinance clients best be measured? How does that differ from tracking impact? Last month I participated in a panel that gathered three MFI practitioners, one social investor, and one researcher from Innovations for Poverty Action (IPA) to find answers to that question.

Anticipating some sparring between the “camps,” I sat between those proposing monitoring tools and non-experimental research as a way to track change, and those supporting impact studies with randomized controlled trials (RCTs) as the way to attribute impact, for example, from a loan or savings product.

Tracking or studying change and measuring impact are fundamentally different things, and there’s a growing schism in microfinance between these two camps. On the one hand, there is a movement composed of social investors, MFIs, management consultants, and researchers that support monitoring and change studies. For this group monitoring indicators, such as those offered by IRIS or MIX, are sufficient: research sans randomization may give a “good enough” orientation of what microfinance really does for clients. This group also supports monitoring outputs, making diagnostics on how processes affect outputs, and non-invasive academic research that does not need control groups and helps managers to make decisions. They want rigor: but the approach is pragmatic, rather than purist. A list of the various monitoring tools can be found at the Social Impact practice of McKinsey.

The impact supporters, on the other hand, say that impact evaluation with RCTs is all that will actually measure and answer the impact question: there’s simply no other way. And a significant amount of money and brainpower is going into it. There are currently over 300 RCTs completed or ongoing in the research portfolios of the main RCT researchers: the World Bank , J-Pal, or IPA. “Tell me what you exactly want to know and we will measure it and give you answers,” the IPA researcher told the practitioners.

At the heart of this discussion are different interpretations of what impact means and radically different drivers.

The one thing we all agree on is that marketing change or monitoring and evaluation as impact studies is misleading, and simply wrong. The very name “change studies”—coined by an MFI—may reflect a growing awareness and understanding of what monitoring can—and can’t—do, which is good news. But the gulf between the practitioner and the research worlds needs bridging if we’re really going to make progress on such an important discussion.

Comments

06 September 2012 Submitted by Christine Horansky (not verified)

Hi Meritxell, great post! A very interesting discussion. If we are to “measure” poverty reduction it begs the difficult question of how to measure poverty itself in the first place. Deprivation takes so many forms. I myself am a fan of Yunus’s poverty escape checklist, which he uses with Grameen’s microfinance clients and includes questions such as, Are your children in school?

Take care,
Chrissy

06 September 2012 Submitted by Dr V.Rengarajan (not verified)

Be it for a change or impact in the poverty sector, we have tools. But all these methodologies facilitate to look at the change or impact from supply side perspectives. It is also more useful how the change or impact is looked into by the poor themselves with the given inputfrom the demaqnd side.In this respect ‘social audit ‘ assumes importance as the poor themselves reveal the change or impact as they experienced without much biases

Add new comment

CAPTCHA