Payment by results: How to deliver results-based financing through monitoring, evaluation and use of evidence

by 14th Feb 2017
 CARE staff interviewing a recipient to ensure transparency in distributing cash payments to displaced people returning to Chad CARE staff interviewing a recipient to ensure transparency in distributing cash payments to displaced people returning to Chad

Payment upon demonstrating a set of agreed results is an emerging contractual modality that many donors are choosing to pursue. DFID together with USAID are leading this discourse and applying this funding approach to an increasing number of thematic areas, especially in health and education. So what does putting evidence at the centre of payment mechanisms mean for monitoring, evaluation and learning?

Shift from short-term to strategic programming

Under payment by results, the condition for payment is in practice agreed upon a set of indicators that typically represent outputs and selected outcomes. At this time it is reasonable to assume that donors will increasingly focus on longer-term and structural changes, both in development and humanitarian sectors.

This shift of focus from short-termism to more strategic programming will require rigorous evidence, both in learning from the context, risks and experience from past programmes to design new ones that can generate evidence of outcomes; and in continuous monitoring and evaluation, in order to demonstrate results and so to secure payment.

Challenges of the shift to a results-based approach

The principle of putting evidence at the centre of payment flows is a bold and complex approach since some areas of change or attribution are hard to measure with a level of rigour that is satisfactory. For instance:

  1. How to account for spill-over effects between groups of recipients and non-recipients
  2. What is the percentage of recipients’ retention in an intervention?
  3. What are the conditions associated with cash flows, risk-sharing and pre-financing requirements
  4. How to create incentive structures for implementing partners to reduce ‘easy picking’ of results at design of a project.

Challenges also mean opportunities

Beside the existing bias of statistical validation to measure project achievements, there is an on-going opening from donors to negotiate on alternative ways to monitor and evaluate results. The CARE-led Girls’ Education Challenge programme in Somalia represents an example of change of condition from the UK Department for International Development (DFID) as it agreed to stop the payment conditionality based on the tracking of control groups (individuals receiving a different or no intervention) in view of the contextual barriers in following two distinct clusters of girls. Only the recipient group has been tracked for the whole duration of the project and evidence indicated a series of significant improvements in their condition. In selected countries like Niger for BRACED and GEC countries (Somalia, Zimbabwe, Afghanistan), the recipient groups have been tracked for the whole duration of the project and evidence indicated a series of significant improvements in their condition.

This example shows that an appreciation of the context allows for an open discussion with key donors on realistic targets and incentives to highlight observable change. Mastering this language opens a gateway to partnerships and it represents a vehicle for further funding.

Payment by results requires investment in M&E

It is likely that donors will continue to favour greater commercialisation and results-based financing of suppliers; the inclusion of the private sector to further trade interests is a clear indication of this trend. In addition, political and media pressures are reinforcing the issue of transparency of results. For these reasons, payment based on evidence boosts the incentive for programme managers to focus on the use of data to demonstrate results as a top priority.

Therefore, even if payment by results (PBR) has an inherent limitation in striking a balance between adaptive response and the ability to forecast and quantify results, it also compels development partners to boost resources in systems to generate real-time evidence on what delivers value for money. However, the effort to extract evidence should not overwhelm any partner but directly address the areas of measurable change that have been agreed and are under constant review with a key donor.

What are the implications for monitoring, evaluation, accountability and learning (MEAL)?

In a PBR scenario, the business case to select the best indicators and trackable group of respondents is stronger than ever. An open discussion with donors and implementing partners on what is feasible and its costs requires a certain dose of conviction when advancing valid alternatives to experimental designs (experiments which test for specific outcomes). CARE’s conviction in this respect would be based on its long-standing presence in multiple countries and thought leadership in participatory methodologies (CVCA).

In support of this dialogue, there are also MEAL trends that confirm the need to trace contribution instead of measuring attribution when reporting on results, particularly if related to social transformations or modifications of regulatory frameworks.

Using evidence to inform activities

The agreement with donors on how to generate and use evidence to inform activities can lead to a renewed management approach. For instance, if a particular training showed more effectiveness once repeated, the project lead should discuss with the donor about how to reallocate resources to reinforce a specific investment in the set of skills that proved to cause significant improvements. In fact, a constant review of evidence and assumptions might even sustain some degree of standardisation in tackling particular conditions of change and of feedback (eg complaints mechanisms in Core Humanitarian Standards).

In other words, the fluidity of many contexts implies that a set of assumptions and risks (internal and external) need to be well-appraised at design and constantly tracked along the whole cycle of the intervention. If a new government comes to power, will it remain accountable to policy reforms agreed previously thanks to the engagement of donors and their partners? If a pre-condition is favourable at design, how would a sudden change in power dynamics either boost or restrict the scale of interventions in domains like social activism or minorities’ protection?

Interventions can then embed triggers to adapt to possible contextual variations. Multiple courses of actions need to be identified and considered at inception to outline scenarios in case these shifts occur. Thus, MEAL vests a critical role in adjusting project objectives based on the evolving circumstances.

Involving target populations

Evidence of changes and the perceived quality/relevancy of activities by beneficiaries require a link to contextual fluctuations, particularly if of political nature. Therefore, a political economy analysis is a necessary requirement at the design of a project and during the inception phase.

The public from donor countries expects that sustainability of development interventions is borne by recipients’ capacity to resource the continuation of funded initiatives, preferably via market-based solutions. To deal with this demand and result-based prerequisite, solid and well-resourced MEAL frameworks can validate in real-time if a sequence of activities is triggering a structural change and the roles of multiple stakeholders in that process.

The question remains if a four or five year horizon is ever enough to bring about a transformative change. Nonetheless, the ability of programme managers to collaborate closely with MEAL advisors signifies the foundation for learning from these contracts under which extracting, synthesising and sharing data from the field becomes a participatory task.

How is CARE responding to these challenges and opportunities?

The ability of CARE International UK to manage a fund or to deliver technical assistance within the context of PBR-like arrangements is emerging due to contracts established on output and outcome targets in the domain of girls’ education and resilience building.

The current steps CARE is now considering are an overall improvement in processes to handle similar contractual arrangements and a way to track systematically all project experiences that contain examples of adaptive response and strong MEAL frameworks. Together, these actions will support a management toolkit and skills growth strategies to enable teams involved in programmes to:

  1. articulate indicators clearly
  2. link inputs to long-term changes
  3. resource MEAL sufficiently via large monitoring teams
  4. focus on updating context-based assumptions and the theory of change.

All these actions are the way forward to respond to PBR requirements if coupled with a progressively deeper understanding of areas of intervention from previous programmes by CARE or other partners. What donors seem inclined to favour in PBR is the identification of what has worked in the past within a defined geography and its replication or scale-up based on market opportunities and local circumstances/risks affecting target groups.

Learning from past projects on how management choices delivered the most cost-effective activities linked to a theory of change can position CIUK as a reliable supplier related to our CARE 2020 outcomes. In the last financial year alone (2016), over 1,000 initiatives have been implemented by CARE and its partners, so it is reasonable to expect that plenty of examples can inform the debate on “measurability”, but most importantly they can consolidate recognition of how past achievements were attained or exceeded.

CARE’s impressive portfolio remains the starting point for future steps and to distil a system that can respond and even influence the very definition of payment by result and outcome areas. The wealth of evidence that is within reach is surely the next big opportunity for CARE and partners to engage donors in a deeper understanding of where flexible programming and testable assumptions meets conditions for payment and validation of a programme theory.

Nicola Giordano

Nicola was formerly the Monitoring, Evaluation, Accountability and Learning (MEAL) Specialist within the Programme Management Team at CARE International UK. His main background is in the analysis of large-scale research and impact assessment studies. In his role at CARE International UK, he was responsible for providing technical and strategic support across CARE's portfolio to ensure that changes led by development interventions are logically framed, well-captured, triangulated and shared at different levels.

Nicola's main areas of interests are: data analytics applied to the third sector; digital means to generate evidence; and inclusive project design that can assure feedback loops between implementing partners and targeted communities. Before joining CARE, Nicola worked for a number of governmental, non-governmental and private organisations in the development sector at the international and national level.