Why do internal evaluations matter? – what we learned from CARE’s evaluation of an emergency cash-first response in Zimbabwe

by 16th Jun 2017
A participant in the cash transfer programme in Masvingo district, Zimbabwe A participant in the cash transfer programme in Masvingo district, Zimbabwe

CARE and other INGOs are increasingly exploring cash transfer modalities both for emergency response and other multi-component interventions. Yet public and political pressures to demonstrate results are also increasing – and are leading implementing agencies to set up comprehensive monitoring systems and rigorous evaluation cycles.

To respond to the demand for evidence, CARE decided to invest a relatively small amount of internal resources (£10,000) to complement findings from an OPM-led qualitative evaluation of this programme with unprecedented size in Zimbabwe.

The representative sets of information from monitoring and an additional round of data collection at endline offer a great opportunity for CARE to demonstrate the scale of cash-first response and the usefulness of linking datasets to trends. The objective is to both maximize internal learning and to propose alternative quantitative approaches that can relax strict experimental design requirements and still provide compelling evidence of change across all target areas.

Introduction to DFID-funded cash-first response

CARE International in collaboration with World Vision International in Zimbabwe implemented the DFID-funded project ‘Emergency Cash-First Response to Drought-Affected Communities in the Southern Provinces of Zimbabwe’ from August 2015 to April 2017. The objective of the project was to enhance the food security and reduce negative coping strategies of vulnerable and drought-affected households in four provinces. The project’s specific outcomes were to ensure that beneficiaries were able to cope with food shocks and met their basic food needs during the 2015/16 and 2017/17 agricultural periods.

The first phase of the project ended in February 2016 but after the second season of failed rains, the project continued into a second phase, with transfers being delivered from July 2016 until March 2017. It began by supporting 67,200 households in the first phase and increased to 73,736 by March 2017; affecting over 418,000 people (average household size is 5.7 individuals). The project transferred an estimated $40.9m (£25.7m in the second phase) to 73,736 recipients through mobile money, reaching households that had been selected through community-based targeting in drought-affected areas. A monthly transfer was initially $5 per each household member and increased to $7 in August 2016, with households on average receiving $554.68 (total budget/direct recipients) through 17 payments from February 2015 to April 2017.

Results from CARE-led final evaluation

The CARE-led evaluation focuses on the second period of the programme from 1 April 2016 to 1 April 2017. The starting point of this study is the results from the midline and all evidence collected subsequently. By considering all limitations of the available datasets, we can comfortably identify some strong trends from information that has been collected recurrently. Cash transfers in target areas have significantly boosted food security, nutrition and the abilities of communities to cope with shocks.

cash transfer evaluation table

Operational learnings from evaluating the cash transfer programme

Alongside these remarkable results, the external evaluation from OPM also confirmed the suitability and efficiency of the programme thanks to: the use of adequate mobile networks for transfers, prompt accountability/feedback systems, comprehensive monitoring loops, and tight interactions with DFID. All of these factors combined allowed for the generation of large datasets on a monthly basis and for this quantitative study to achieve a broad-spectrum analysis of trends.

Given the new spaces institutional donors are exploring in the cash transfer debate, monitoring systems aggregating batches of data from both mobile operators and individual recipients are going to be the game changers in evidencing results. For this cash transfer, the scale of monitoring data is sufficient to identify indicative trends but by boosting its consistency, longitudinality and relevancy (to reduce survey fatigue) the evaluation cycle can validate causal claims at the population level.

Strategic lessons from evaluation findings

Key lessons from this study are:

  • In a context of liquidity crisis and severe climate shocks, mobile platforms play an essential role to accelerate a shift to a cashless economy which can offer a business opportunity for merchants and revitalise demand/supply. Yet, evidence of market integration, entrepreneurship and financial inclusion at the recipient level remains weak since the predominant effect of this programme links to food security/nutrition and the reduction of currency-exposure risks in the food supply chain thanks to DFID support to the Grain Market Facility.
  • Once a cash transfer programme is already in place and relationships with mobile network operators established, it is easy to provide and justify additional transfers, as shown by the addition of a multi-purpose grant delivered in October 2016.
  • The transfer value of $7 per person was appropriate for enabling people to access basic food needs and was well received by over 85% of target recipients. To expand the nature and breadth of change, the amount could have been tailored based on other criteria such as: gender-specific vulnerabilities, household non-food needs and severity of climate shocks by specific metrics of rural livelihood loss.
  • The partnership model between DFID, implementing NGOs and mobile network operators is at the heart of a functioning cash transfer model that is responsive to shocks and can leverage monitoring evidence to address the consequences of recurrent climate shocks.
Nicola Giordano

Nicola was formerly the Monitoring, Evaluation, Accountability and Learning (MEAL) Specialist within the Programme Management Team at CARE International UK. His main background is in the analysis of large-scale research and impact assessment studies. In his role at CARE International UK, he was responsible for providing technical and strategic support across CARE's portfolio to ensure that changes led by development interventions are logically framed, well-captured, triangulated and shared at different levels.

Nicola's main areas of interests are: data analytics applied to the third sector; digital means to generate evidence; and inclusive project design that can assure feedback loops between implementing partners and targeted communities. Before joining CARE, Nicola worked for a number of governmental, non-governmental and private organisations in the development sector at the international and national level.