CGAP photo by Lorena Velasco via Communication for Development Ltd.
Research & Analysis
Publication

Russia Country Case: International Confederation of Consumer Societies (KonfOP)

Mystery shopping has allowed KonfOP, the Russia-based International Confederation of Consumer Societies, to yield a wealth of findings and observations. Mystery shopping revealed important instances of noncompliance with consumer protection regulations, consumer discrimination (including gender discrimination), and abuse. Some findings have led to changes in regulations pertaining to financial consumer protection.

Type of market monitoring tool: Mystery Shopping
Sub-type: Product-focused mystery shopping


Background

FACTSHEET
Country: Russia
Implementor: International Confederation of Consumer Societies (KonfOP)
Sector: Financial services
Tool: Product-focused market monitoring based on mystery shopping
What is the tool used for? To collect and analyze information about how well financial consumers are informed of their rights in the field of financial services, violations of these rights, and consumer protection practices. The goal of data collection and analysis is to develop recommendations for state authorities to ultimately improve consumer protection.
Third parties: • Ministry of Finance of the Russian Federation • The World Bank • Federal Service for Surveillance on Consumer Rights Protection and Human Wellbeing
Estimated cost: US$110,000–150,000 per independent “monitoring wave.” Each wave covers an average of six months, 20 regions of Russia, 25 banks, 25 insurance organizations, and 25 microfinance institutions (MFIs). It also covers the costs of the involvement of international experts, report preparation and distribution, and presentation of report results.
Year(s) of usage:2014–2020 (13 monitoring waves)
Keywords: Consumer experience, consumer issues, credit, deposit insurance, disclosure requirements, insurance, investments, loans, mortgage, mystery shopping, Russia, savings
  • About the implementor. Russia-based KonfOP was founded in 1992. The consumer advocacy organization currently has 36 members, all leading consumer organizations from six countries: Belarus, Georgia, Kazakhstan, Russia, Tajikistan, and Ukraine. KonfOP promotes legislative and regulatory changes that result in a safer and healthier consumer environment. A key focus area is the protection of the rights of financial consumers.
  • Market monitoring role. Since 2014, KonfOP has regularly published reports on monitoring the state of activities that protect the rights and interests of consumers of various financial products in Russia. Between 2014 and 2020, KonfOP published reports on 13 independent monitoring waves that covered various consumer loans, payday loans, microloans, mortgage loans, and savings and insurance products.

Purpose and incentives

  • What is the tool used for? The purpose of KonfOP’s mystery shopping and other market monitoring activities is to collect and analyze information about how well financial consumers are informed of their rights in the field of financial services, the violation of these rights, and consumer protection practices. The goal is to develop recommendations for state authorities in order to improve consumer protection.
  • Incentives for tool development. These mystery shopping activities were implemented under the KonfOP project, “Conducting an independent monitoring (public inspection) in the field of protecting the rights of consumers of financial services” (2013–2020) which, in turn, was part of the Russia Ministry of Finance and the World Bank’s joint Financial Education and Financial Literacy (FEFL) Project. FEFL’s objectives were (i) to improve the financial literacy of Russian citizens—especially school-age and college students, and active and potential low- and middle-income users of financial services, and (ii) to strengthen the foundations for improving consumer protection in financial services. The project included a Fund of Good Ideas to support initiatives by nongovernment organizations to strengthen financial consumer protection.

Technical methodology and data ecosystem

  • Overall approach. Each monitoring wave comprised several research activities, including:
    • Background research on judicial and international practices of the protection of the rights of financial services consumers.
    • Desk research on usage, regulations, surveys, publications, press articles, etc., related to the product under study.
    • Interactions with financial institutions, market regulators, experts in the Federal Service for Surveillance of Consumer Rights Protection and Human Wellbeing (the general consumer protection agency or CPA), as well as consumers. 
    • Study of financial institution websites that describe the products under monitoring, following a set of questions to gather key background information (see Table 1).
    • Mystery shopping—the key tool of monitoring waves.
  • Mystery shopping. Mystery shopping activities were conducted by phone and in person. Mystery shopping hypotheses and questionnaires were developed based on the background research and the study of financial institution practices with respect to the product being monitored (see Table 2). Responses to each question serve as indicators that either confirm or refute the hypotheses developed. Each mystery shopper must first pass a sociodemographic capability test. They were then trained on their profiles and behavioral scenarios, and receive specific instructions on testing additional hypotheses (see Table 3). 
  • Sampling. Financial institutions were selected based on their size and presence in the respective product markets monitored in each wave. For example, for the 2020 monitoring wave of savings products, KonfOP selected the 27 largest banks (covering over 80 percent of the deposit market) and the ten largest microfinance companies (MFIs authorized to attract investments). Selection was based on data from the Central Bank of Russia and independent rankings of the RA Expert think tank. The research covered 19 regions.
Sample research questions for studying financial institution websites: Monitoring wave of savings products (2020)
Source: Report on the 13th wave of independent monitoring, The State of Protection of Consumer Rights and Interests in the Market of Savings Services in Russia (KonfOP April 2020, content in Russian).

 

Sample research hypotheses: Monitoring wave of savings products (2020)
Source: Report on the 13th wave of independent monitoring, The State of Protection of Consumer Rights and Interests in the Market of Savings Services in Russia (KonfOP April 2020, content in Russian).

 

Example of mystery shopper profile and behavior scenarios: Monitoring wave of savings products (2020
Source: Report on the 13th wave of independent monitoring, The State of Protection of Consumer Rights and Interests in the Market of Savings Services in Russia (KonfOP April 2020, content in Russian)

Staff, expertise, and other requirements 

  • Staff requirements. The following staff members were engaged to implement the full set of research activities for each monitoring wave:
    • Regional coordinators with experience in marketing or sociological research in the regions selected for monitoring.
    • Website researchers with experience collecting and processing data for marketing and sociological research.
    • Mystery shoppers with the following demographic profile: age 25+, higher education, at least two years of work experience. To better assign the different profiles, potential mystery shoppers were also asked to share the following personal data: information about education and employment, marital status and number of dependents, household income and expense data.

Benefits and impact

  • Mystery shopping allows the mystery shopper to directly record the information a consumer receives when communicating with a particular financial services provider (FSP). The tool makes it impossible for an FSP to hide its practices or present itself in a better light since the FSP believes it is dealing with an ordinary customer.
  • Mystery shopping allows on-the-spot evaluation of the actual visual presentation of financial products since the mystery shopper checks which promotional materials are available at FSP branches.
  • The tool assesses and compares an FSP’s practices in several regions to draw conclusions on the consistency of its customer treatment.
  • KonfOP’s product monitoring waves yielded a wealth of findings and observations (including consumer discrimination and abuse) previously not identified by supervisors. For example, in the most recent monitoring wave of savings products in 2020, KonfOP had not specifically focused on gender disaggregated information. However, the mystery shopping tool identified cases of women being discriminated against so it has proven to be a tool supervisors can use for that purpose.
  • Mystery shopping has allowed the discovery of important instances of noncompliance with consumer protection regulations and of malpractice. Some findings have led to legislative and regulatory changes pertaining to financial consumer protection. For example, the Central Bank of Russia banned discrimination in the insurance industry, requiring MFIs to provide Key Fact Statements to clients. Laws on consumer credit and microfinance activities have also been amended to introduced limits on MFI charges and interest rates. (See Annex.)

Limitations and implementation challenges

  • Mystery shopper training is key to the success of mystery shopping as insufficient preparation may lead to distorted results.
  • Mystery shopping exercises need to be regularly repeated to monitor changes in the behavior of market players.
  • The regulator cannot use the results of third-party mystery shopping against specific institutions, yet results can inform policy and supervisory decisions and changes.
  • A research budget is necessary for comprehensive monitoring. With the expiration of the FEFL project, KonfOP will look for ways to continue its market monitoring activities on a smaller scale.

Future plans for the tool

  • In early 2021, KonfOP began looking into the possibility of holding a new “light” wave of monitoring due to the expiration of the FEFL project and associated funding for the exercise. The light wave will include desk research of FSP websites, plus communication with FSP staff via phone and online through chats and apps. KonfOP is currently searching for a partner to continue its market monitoring work.

Learnings

The use of the mystery shopping tool proved to be an effective means of market monitoring. In particular:

  • Mystery shopping can capture nuances of FSP behavior that otherwise may be difficult to identify. For example, the 2020 savings market monitoring discovered that some FSPs advertised risky investment products not protected by deposit insurance on the back of customer queue tickets.
  • Mystery shopping proved to be an important tool to identify discriminatory practices against women and vulnerable groups.
  • Mystery shopper training is the key to the exercise’s success. Mystery shoppers need to be able to clearly state the purpose of their visit, and do so in cases where the conversation follows a particular scenario or takes an unexpected turn (e.g., an FSP staff member refuses to provide answers). Mystery shoppers must be capable of casual conversation, possess strong interviewing skills, be able to record or remember conversations, and have knowledge of the financial market.

Annex: Main findings and results of product monitoring waves

Savings Wave (2020)

  • The most recent monitoring wave discovered that only three out of 27 banks visited in 19 regions consistently offered deposits when mystery shoppers requested them, regardless of the location of the bank office. In eight banks, various offices of the same bank had different practices; in others, bank staff offered either hybrid products or investments that were not protected by the deposit insurance scheme.
  • With respect to disclosure of risks to banking customers, 19 banks did not inform mystery shoppers that the return is not guaranteed for non-deposit products; 20 did not inform them that the deposit insurance scheme did not apply to non-deposit products; 15 did not inform them that part of their investment would be lost in the case of early withdrawal; and 14 did not inform them of any of these risks.
  • Regarding investments offered by ten microfinance companies, four refused to provide information by phone, instead referring shoppers to the company’s website; only one partially informed mystery shoppers about the risks of retail investments. Websites of both banks and microfinance companies contained incomplete and often misleading information about non-deposit products.
  • Based on the results of the study, KonfOP proposed regulatory changes to prescribe standardized disclosure of information on deposit and non-deposit products—both by staff of financial institutions as well as on websites.
  • In January 2021, the State Duma proposed a bill prohibiting the sale of complex investment products to novice investors. Several initiatives were put forward to strengthen the protection of savers and unqualified retail investors, including amendments to the law on the securities market; a Central Bank of Russia recommendation discouraging insurance companies from selling insurance products with an investment component to individuals; and a Central Bank of Russia information letter to all MFIs requiring elaboration of Key Facts Statements for key financial products.

Source: Report on the 13th wave of independent monitoring, The State of Protection of Consumer Rights and Interests in the Market of Savings Services in Russia (KonfOP April 2020, content in Russian)

Insurance Wave (2017 and 2018) 

  • In 2017, KonfOP discovered that insurance companies abused certain groups of customers when they purchased insurance policies to secure loans such as mortgages. The consumer advocacy organization found significant discrimination against pregnant women, people with HIV, cancer patients, and people with disabilities.
  • After KonfOP presented its mystery shopping results, the Central Bank of Russia held meetings with insurance companies to discuss the problematic findings. In May 2019, the Central Bank issued basic standards that ban discrimination.
  • KonfOP further monitored the situation in 2018 and improvements were found—even before basic standards came into force.

Sources: Report on the 13th wave of independent monitoring, The State of Protection of Consumer Rights and Interests in the Market of Savings Services in Russia (KonfOP April 2020, content in Russian)
Elevating the Collective Consumer Voice in Financial Regulation (CGAP 2021)

Payday Loans Wave (2018) 

  • KonfOP research discovered that most payday loan consumers belong to lower-income groups: 91 percent considered their financial situation to be average or below average, while 20 percent admitted that their income was sufficient only to buy food.
  • Over 50 percent of payday loan borrowers did not know the annual interest rate of their loan and did not understand key conditions of their loan agreement. They also had never compared payday loan conditions with other loan products.
  • Mystery shopping among payday lenders (registered as MFIs) showed that over half did not inform consumers about annual interest rates and the vast majority did not check consumer creditworthiness (only one of ten MFIs requested information other than a personal ID). All MFIs offered payday loans at 500 percent per annum or higher.
  • All MFIs gave loans to consumers who had defaulted on loans from other sources, as well as to vulnerable, low-income groups such as pensioners and unemployed people.
  • Based on increased public attention to the issue of irresponsible lending practices by payday lenders, policy makers initiated legislative changes. In December 2018, the laws on consumer credit and microfinance activities were amended to introduce a number of limits to the interest rate and total amount of loan interest and penalties that MFIs could charge.

Sources: Report on the ninth wave of independent monitoring, Analytical Report on Payday Loans (KonfOP 2018, content in Russian)
Report on the ninth wave of independent monitoring, The State of Protection of Borrower Rights and Interests in the Market of Credit Services in Russia (KonfOP 2018, content in Russian)

Back to top ↑