SmartAid FAQs
August 2, 2010
What is the SmartAid for Microfinance Index?
Quality, not just quantity of aid matters. Acknowledging this premise, the heads of leading development institutions endorsed the creation of an index that would measure the effectiveness of their work in microfinance. The SmartAid for Microfinance Index (SmartAid) is CGAP’s effort to fulfill that commitment.
Measuring the effectiveness of microfinance funding is not easy, given the long chain from funding to impact on the ground. SmartAid focuses on the first part of this chain, arguably the one that funders can most directly influence: their own management system.
The premise of SmartAid is simple: strong internal management systems lead to better projects on the ground. The Index purposefully focuses on the quality of funders’ systems and does not evaluate the on-the-ground performance of funders’ microfinance portfolios. Other tools (e.g., portfolio reviews) exist for that purpose.
How does SmartAid evaluate effectiveness?
SmartAid builds on a framework of five elements of effectiveness that have emerged from eight years of CGAP’s work on aid effectiveness in collaboration with many partners:
- Strategic clarity measures whether the funder has an agency-wide vision for microfinance and whether its policy and strategy are aligned with good practice and based on its capabilities and constraints.
- Staff capacity measures whether the funder has staff with relevant expertise to ensure the quality of design, implementation, and monitoring of programs and investments
- Accountability for results measures whether the funder has systems in place that ensure the adequate monitoring and performance-based management of microfinance programs and investments.
- Knowledge management measures whether the funder has systems and resources to create, disseminate, and incorporate learning from its own and others’ experience
- Appropriate instruments measures whether the funder has instruments that are used in a flexible manner and are adapted to market needs.
These five elements were translated into a set of well-defined indicators that, taken together, provide a comprehensive picture of a funder’s internal management systems.
What does SmartAid offer participating funders?
SmartAid is a path to greater effectiveness. It is a service for donors and public investors who seek to better understand how internal management systems, processes, and incentives affect their work in microfinance. All participating funders receive a report that includes quantitative scores for all indicators, a detailed description of strengths and weaknesses, and concrete recommendations.
From the submission of the documents to the discussion of the report, the SmartAid team works closely together with the funder. The process includes opportunities for in-depth discussions between staff and management and brings together different units that work on microfinance. Many participating institutions have found the process of preparing the submission useful for taking stock of internal processes and policies.
Many funders said that SmartAid helped them accelerate change processes that were already underway, but needed additional impetus. As an independent external assessment, SmartAid might further highlight issues that have been overlooked internally. It helps focus management attention on key strengths and weaknesses and provides a framework for prioritizing changes that need to be made to improve effectiveness.
The Index can also help funders understand where they stand relative to other microfinance donors and investors. This helps funders connect to others that have complementary strengths and facilitates collaboration.
By opening their doors for an external review, funders participating in SmartAid also demonstrate their commitment to transparency and accountability. This sends a strong signal to all who care about development and potentially contributes to improve aid management beyond microfinance.
For concrete examples of how SmartAid helped accelerate change processes, see the SmartAid Change Stories.
What indicators does SmartAid use and how are they defined?
SmartAid builds on over five years of collaboration between CGAP and a broad range of microfinance funders. The index is the most recent tool developed as part of CGAP’s aid effectiveness initiative, which started in 2002 with the Microfinance Donor Peer Reviews and the country-level effectiveness and accountability reviews.
From this work, five elements emerged as critical for the effectiveness of funders’ internal management systems: strategic clarity, staff capacity, accountability for results, knowledge management and appropriate instruments. With the help of CGAP’s members, academic advisors including the Center for Global Development, and microfinance experts, these five elements of effectiveness were translated into a set of well-defined indicators that constitute the SmartAid Index.
SmartAid for Microfinance Index Indicators
|
| Strategic Clarity |
1. Funder has a policy and strategy that addresses microfinance, is in line with good practice, and is based on its capabilities and constraints |
15 points |
| Staff Capacity |
2. Funder has quality assurance systems in place to support microfinance projects and investments |
15 points |
| 3. Funder has the staff capacity required to deliver on its microfinance strategy |
10 points |
| Accountability |
4. Funder has a system in place that identifies all microfinance programs and components |
10 points |
| 5. Funder monitors and analyzes performance indicators for microfinance projects and investments |
10 points |
| 6. Funder incorporates performance-based elements in standard agreements with partners |
10 points |
| 7. Funder regularly reviews the performance of its microfinance portfolio |
10 points |
| Knowledge Management |
8. Funder has systems and resources for active knowledge management for microfinance |
10 points |
| Appropriate Instruments |
9. Funder has appropriate instrument(s) to support the development of local financial markets |
10 points |
| MAXIMUM SCORE |
100 points |
Indicators represent in between 10 and 15% of the total score of 100 points. Different weights were assigned to the five elements of effectiveness, giving due prominence to those that make the most difference.
As the microfinance industry evolves, funders need to keep up to date with good practice, adapt to a changing environment and address upcoming challenges. To reflect this, the SmartAid indicators might be adapted slightly over time.
Who scores the participating funders?
SmartAid is based on an assessment of documents reported by the funder. Participating funders submit documents for each of the SmartAid indicators. The SmartAid Submission Guide describes the indicators in detail and provides examples of documents to submit. The submissions are then reviewed by a review board of four microfinance specialists who have extensive experience with a broad range of funders.
How is the score determined?
 Each review board member independently scores agencies against all indicators; final scores are then agreed on in a review board meeting.
Participating funders receive a score from 1 to 5 for each indicator. A score of 1 stands for inadequate performance, a score of 5 means that the funder provides an example of good practice for this indicator.
The scores are weighted and added up to a total of 100 points. SmartAid differentiates between five performance categories: inadequate, weak, partially adequate, good, and very good.
Can scores be compared between the 2007 and 2009 round of SmartAid?
SmartAid was developed over a 10-month period and piloted with seven agencies in 2007. Funders participating in the pilot round and an external assessment of SmartAid provided many lessons and recommendations for further refinement which have been incorporated in the 2009 round of the Index.
To increase the relevance, credibility and rigor of SmartAid, the original indicators were reviewed against the following criteria:
- Does the indicator measure a practice that affects the effectiveness of a funder’s systems
- Would most agencies have written documentation available that demonstrates their policy/ state of practice with regard to the indicator?
- Can the indicator be objectively measured, given the Index’s reliance on self-reported documentation?
- Is the indicator relevant for a broad range of agencies?
Based on these criteria, the original 20 indicators were streamlined into a set of nine. Given these changes, the results from the pilot round and the 2009 round of SmartAid cannot be compared.
Going forward, the indicators will be adapted as good practice evolves, however the scoring system and weighting across indicators will retain consistency so that funders will be able to track their performance over time.
Where can I find the SmartAid reports?
Reports from the 2009 round of SmartAid are available here.
Given the changes that have been made to the methodology between the pilot round and the 2009 round of SmartAid, the scores from these two rounds are not comparable. For this reason, the reports from the 2007 round are no longer available on our website. Please contact cgapbetteraid@worldbank.org if you are interested in the 2007 SmartAid reports.
Can one standard apply to so many different types of funders?
The SmartAid indicators are based on five key elements that are critical for the effectiveness of a funder’s internal management systems. These indicators capture the “spirit” of what good performance may look like but are not prescriptive in themselves. For example, good knowledge management practices and systems may look very different in a highly centralized agency as compared to a highly decentralized agency. Two agencies with vastly different systems might score similarly on this indicator as the Review Board scores what is appropriate for the institution given its size, level of centralization, and mission.
Although SmartAid has been developed specifically for microfinance, many SmartAid participants said the framework, in an adapted form, could even be valuable for other sectors of development.
Microfinance is a very small part of our agency’s portfolio. Does it make sense to participate in SmartAid?
SmartAid is open for all kinds of funders; there is no limit in terms of portfolio size.
Compared to other sectors of development, microfinance programs do not absorb large amounts of funding. So a small part of your agency’s portfolio could still be a considerable percentage of total funding for microfinance. The sixteen funders that have already participated in SmartAid together provide more than 50% of total funding committed to microfinance.
When considering a participation in SmartAid, funders should ask themselves whether an independent external assessment could be helpful for them to improve their systems. Preparing documents for SmartAid is a task that requires the necessary human resources. The process is most helpful when supported by top management and when a broad range of staff from headquarters and country offices take part in the discussions.
How can I prepare my agency for participation in SmartAid?
Funders that are not ready for an independent external assessment of their systems can do a self-assessment based on SmartAid. The self-assessment builds on the same five elements of effectiveness as the SmartAid Index and consists of a staff survey, focus group meetings and the discussion of the findings. CGAP is available to support the process, and the methodology can be adapted for different funders’ needs. If you are interested in learning more about the self-assessment, please contact cgapbetteraid@worldbank.org.
When does the next round start? And how can my agency sign up?
The SmartAid for Microfinance Index is open for submissions every two years. The next submission period starts in September 2010. If your agency is interested in participating, please contact cgapbetteraid@worldbank.org for more information.
Are the results from SmartAid public?
Participating funders receive a report that includes quantitative scores for all indicators, a detailed analysis of their strengths and weaknesses, and concrete recommendations on how to improve. Public disclosure of the report is not mandatory, but encouraged. By sharing the reports with peers and with the public, funders improve transparency about their work and show a strong commitment to accountability. If public, SmartAid reports also help funders connect to others that have complementary strengths, thus facilitating collaboration.
Reports from the 2009 round of SmartAid are available here.
Has SmartAid lead to change within participating agencies?
As an external assessment tool, SmartAid draws management attention to issues that may have been overlooked and helps identify priorities for change. As a result of their experience with the Index, funders are making significant improvements in their systems. Some participants reoriented their strategic focus, others upgraded staff skills, or revamped monitoring and evaluation systems. Some funders that participated in 2009 started making changes even before their reports were finalized.
To read more about how different agencies used SmartAid as a tool for change, see the SmartAid Change Stories.
|
