Getting The Proof To The Top Of The Pile

Policymakers are swamped. They have a wide range of interest groups talking to them all the time, they have large numbers of papers and emails and phone calls to deal with every day.  So if you approach them with a brilliant evaluation that is fifty pages long, complete with graphs and tables and lots of Greek equations, it will go straight to the bottom of the stack. And stay there.

Researchers are doing brilliant work in impact evaluation, and using methods that create what they call “proof of concept.” In layman’s terms that means whether an idea works. They use an arcane language to discuss this, including terms such as “counterfactual” (what would be the case if you didn’t try the new idea).  And that language gets in the way of policymakers and practitioners understanding and using the results in sensible ways.

Let’s take as an example the discussions in the recently held Asian Development Bank/IPA Impact and Policy Conference in Bangkok. Two very thoughtful papers were presented in the area of improving financial capability (code for “learning to manage money better”), one on basic education on compound interest, and one on the impact of different financial education methods on groups of children.  Because the audience was mixed, the researchers kept the messages simple and the slides down to only one or two with the requisite analytic results tables (what my daughter calls the “crazy tables” because everyone’s eyes go crazy when they are presented in a PowerPoint.)

The messages from the researchers were clear:

• If rural families in China get simple, clear information on the effects of compound interest on rural pension schemes right before enrollment, they decide to invest more. Families who are paying into the scheme understand why contributing more money now might really help them later.
• In school, if you teach a group of children about savings and social issues, and you teach another group of kids just about savings, they both save. And they both are more careful about money.  The ones who learn about savings but don’t learn about social benefits are more careful about earning money.

Policy makers in the People’s Republic of China (where I work) loved this, not least because the first study was done there. They took away the following messages from the presentations:

• A short, focused training session can really help rural pensions grow. Find out a way to do this.
• Kids save and think more carefully about money if you give them this information in school, and set up chances to save at school. Set up this system.

I bet they are talking about a pilot to do more of this, right now.  Legitimate researchers, clear methods acceptable to peers, understandable results and clear ways forward to benefit others: this is the way to make a difference.

To get their lessons to the top of the pile, the lessons for researchers can be simply summarized:

1. Deliver clear, simple messages on what works, how, and what practitioners can do to make it work for others. Get your 12 year old to check if it is simple enough.
2. Keep the jargon and the equations for other researchers.
3. Talk to policymakers and practitioners about their problems and help them think through how they would find out if a proposed solution can work.

--- The author is the Director, Public Management, Finance and Trade Division of the Central and West Asia Department of the Asian Development Bank. The blog is written in her personal capacity and she is solely responsible for all content and views.