Why Can’t We Answer the Question: Does Aid Work?

This blog series and a preceding workshop organized by CGAP aimed to explore what aid effectiveness means in the context of financial inclusion.

Both Homi Kharas and Owen Barder pointed out that despite large amounts of aid over a significant period of time, we know very little about what aid actually achieves. Whether you find it extraordinary or trivial, we still cannot answer the simple question of whether aid works or not.

Owen Barder dissects the heart of the question effectively when he writes: “At the heart of this disagreement is not a dispute about the impact of aid but about what we mean when we ask whether ‘aid works.’”

Let’s explore what we mean when we ask whether aid works.

Does the global aid system work?                                                                                                                                                                                                                                      
We have to differentiate whether we are looking at the global aid system or aid to support a specific sector. Mayada El-Zoghbi highlighted the parallel track of global aid effectiveness initiatives, such as the Paris Declaration, and sector-specific initiatives, such as the SmartAid for Microfinance Index. If we are asking “does the global aid system work” we will look at very different things and might come to a different conclusion than when we take a technical perspective and look at what is needed in a specific sector. And unfortunately, improving effectiveness of the global aid system does not necessarily go hand-in-hand with what is needed at a sector-level.

Did the project reach its objectives?                                                                                                                                                                                                                                  
Homi Kharas summarizes one of the typical challenges for aid agencies: “Every aid agency faces this dilemma. They might do an evaluation at the project level to provide information on whether they are doing things right, but they have a great deal of difficulty knowing whether they are doing the right things to achieve the objectives they have set out for themselves.”

Donors can use a broad range of evaluation tools to measure whether “aid works.” Depending on what they want to find out (are we doing things right or are we doing the right things), different tools are needed. While project evaluations help understand why in a particular case a project succeeded or failed, the findings can hardly be generalized. Portfolio reviews that look at a significant segment of a donor’s portfolio (e.g. all microfinance projects) help draw lessons on what works more broadly. Also, they help answer the question whether a donor is actually “doing the right things” to reach its overall development objectives.

Are aid agencies doing a good job?                                                                                                                                                                                                                                                  
A different, but no less relevant, question is whether aid agencies themselves are doing a good job. As Homi Kharas pointed out: “But donors want to know about the quality of aid agencies. Bureaucrats have to make decisions about which multilateral agencies to fund. Donor country citizens want to know whether their government is spending aid wisely and as effectively as possible. Aid agency managers want to know how to do better.”

Nobody would argue that strong aid agencies are a guarantee for successful projects on the ground, since many other factors influence whether a project reaches its objectives or not (e.g. external factors such as global crises and conflicts, to mention the most extreme ones). That is why some effectiveness initiatives focus on improving the effectiveness of aid agencies, e.g. the Quality of Official Development Assistance Index and the SmartAid for Microfinance Index.

Donor agencies themselves put in place internal accountability systems to measure and show the taxpayer whether they are doing a good job. Heather Clark explored in her post why this still seems to be a major challenge.

While these indices and accountability tools address an important piece in the puzzle, they obviously answer only one aspect of “does aid work.” And as Homi said: “Indices like QuODA are useful if they are used.”

Do we spend aid money on the right things?                                                                                                                                                                                                                        
Many articles in the aid effectiveness literature focus on how the money is spent. While discussions about aid allocation tend to turn political or even ideological, aid agencies have to make decisions about how to spend limited amounts of aid. But maybe the question is not about whether to support public health or private sector development, for example. Owen Barder asked a much more relevant question: “If we want to see aid used effectively, we should demand that it is used for these purposes for which it has a unique contribution to make.”

Tilman Ehrbeck answers this question for financial inclusion: “The role for public (aid) money is pushing the frontiers of financial access through demonstration projects, and public goods, such as infrastructure and the kind of global knowledge exchange that is arguably where the microfinance community has been most effective in organizing itself over the past few decades.”

Readers who expected a definitive opinion on whether aid works will be disappointed. However, I hope we’ve been able to share some of the key issues we have to address to make it work better.

I would like to give the last word to Jonny, one of the commentators in this blog series: “The failures of the past, even if they heavily outweigh the successes (of which, I am sure, there have been many), should not deter us from striving to continue to improve, and continue to help those who need it most.”

This post is the last in a special series aimed to help us reflect on broader aid effectiveness initiatives, CGAP’s own work on effectiveness, and what this all means for access to finance today. Please share your comments.