BLOG

Outcome Indicators in Financial Inclusion - Are They Up to the Challenge?

The financial inclusion sector is at a pivotal moment, with outcome indicators taking center stage. For years, stakeholders have largely focused on tracking and measuring access to and usage of financial services. But now the spotlight has shifted: what benefits are financial services actually providing? The growing demand for outcome indicators reflects a deeper interest in understanding whether and how financial services translate into improvements in people’s lives. Are individuals and micro and small enterprises (MSEs) leveraging these services to build resilience, create jobs, and fuel sustainable growth? Are financial services advancing women’s economic empowerment? This deeper interest in the outcomes of financial inclusion is turning the spotlight on how we can measure them.

Many investors, donors, policymakers, and financial service providers are looking to indicators to help answer one or both of two high-level questions:

  1. What progress is being made towards specific outcomes in broad populations, and how does it vary geographically and across demographic groups? 
  2. What contributions are specific financial service interventions making toward outcomes? 

Can indicators rise to these challenges? Given their high cost in time and money, it's crucial to clearly define their capabilities and limitations.

1. Tracking progress nationally and globally 

Leveraging demand-side surveys

Traditionally, progress in financial inclusion has been tracked through periodic demand-side surveys like Global Findex and FinScope. While these surveys were originally designed to measure and track uptake and use of financial services, the growing recognition that such uptake and use doesn’t always equate to improvements in people’s lives has prompted a shift. Surveys are now beginning to incorporate outcome-focused questions, such as the ones in Global Findex about funding emergency spending – a critical dimension of financial health

Several national surveys have extended their coverage of financial health beyond measuring resilience to also include, for example, the ability to seize opportunities. But can they go further and include indicators of longer-term development outcomes such as poverty reduction? While appealing, this approach warrants caution. Beyond practical concerns about survey length and complexity, significant methodological challenges arise. The deeper surveys delve into outcomes, the more other factors – social, cultural, and economic – influence these outcomes, making it harder to isolate the impact of financial services. 

Leveraging supply-side data 

Global and national surveys occur only every three years or so, leaving policymakers and funders eager for more frequent data and insights into national trends. A promising solution lies in supply-side customer data sets held by regulatory authorities. 
Certain supply-side indicators, such as credit repayment patterns, have proven useful as proxies for dimensions of financial health. Payment system data has great potential for insights into the economic dimension of people’s lives. 

Linking demand- and supply-side survey data – particularly through customer identifiers – is particularly promising and can go a long way in mitigating the shortcomings of each. There’s even potential to connect financial data with personal records from social protection, housing, or employment systems, or to integrate geospatial data for a richer understanding of the customer environment. 

2. Identifying the contributions of interventions

What can indicators tell us about the contributions of financial services to outcomes? Global and national surveys allow for the correlation of outcomes with the use of financial services. But of course, correlation doesn’t imply causality, as confounding factors and uncertain directionality can cloud the picture. As a result, such analyses have been approached cautiously. 

Measuring outcomes from specific interventions is a different story. Some donors and research institutions continue to use rigorous evaluative studies, but these approaches are rarely sustainable. They are costly and require specialized expertise, which financial service providers (FSPs) often lack on a continuous basis. Consequently, while donors continue to deploy rigorous evaluative studies, interest among FSPs and impact investors has recently doubled down on regular measurement through indicators. 

Using indicators to measure impact in the context of specific interventions has several advantages over their use in national and global surveys:

  • Indicators can be tailored to reflect the specific outcome objectives of the intervention(s).
  • Demand-side data can be linked to provider administrative data via customer identifiers, offering deeper insights into how specific populations are impacted and by what factors. 
  • Data analytics can be conducted at shorter intervals, allowing for timely monitoring and decision-making. 
  • Longitudinal surveys targeting identified individuals can capture changes in outcomes, providing a clearer picture of progress for specific customers. 

Despite these advantages, a key challenge persists. Indicators alone, even in these more favorable environments, cannot reveal financial services’ exact contribution or how they have made an impact. While indicators can point to potential contributions, they can’t provide a full picture. To gain greater confidence and actionable insights, these pointers must be explored through deeper analyses that allow us to unpack the “black box” of how impact happens.  

New kids on the block

Until recently, stakeholders had these choices of methodology. However, there are new kids on the block. Machine learning (ML) and other artificial intelligence (AI) mechanisms are now helping to mine data for deeper insights. 

For example, generative AI can process large amounts of qualitative data from multiple sources, identifying recurring themes through natural language processing. Meanwhile, ML can uncover patterns in quantitative data, offering high confidence in the relationship and directionality between intervention outcomes, and providing predictive insights in specific contexts. 

While current AI techniques cannot yet explain the underlying reasons behind certain patterns – and fall short of establishing causality – they bring us closer to that level of understanding by analyzing how variables interact. 

What is CGAP doing about this?

CGAP’s Financial Inclusion 2.0 initiative has several irons in the outcome evidence fire. Through our Impact Pathfinder, we are synthesizing existing evidence about the outcomes of financial inclusion. Further, in collaboration with a handful of pioneering financial service partners, including the Central Bank of Brazil, we are exploring how ML can provide insights into how credit and digital payments can benefit customers in different local contexts. Additionally, we aim to push the boundaries on the use of indicators for outcome insights through the use of ML. 

Expectations of outcome indicators in financial inclusion are running high. We need to be realistic about what they can do and where their limitations lie. There are exciting developments that suggest their shoulders are broader than we thought. But we may need to wait a little longer to be sure.

Resources

Blog

The GPFI’s 2024 G20 Policy Note on Financial Well-Being offers a clear framework for linking financial inclusion to meaningful outcomes. It highlights how financial policies, consumer protection, and digital innovation can drive progress, shaping lives far beyond access to services.
Blog

Global financial health measurement varies, reflecting the need for widely applicable guidelines to make sure financial health measurement provides relevant insights. We outline three principles to help refine approaches and improve insights.

Add new comment

CAPTCHA