The role of evidence in programme quality: a cross-functional ally for adaptive management

by 15th May 2017
A meeting with the target group of a climate change adaptation project in Chhattisgarh, India A meeting with the target group of a climate change adaptation project in Chhattisgarh, India

One of the biggest challenges in achieving programme quality is to link Monitoring, Evaluation, Accountability, and Learning (MEAL) systems to project management practices. Here are some proven ways to achieve this at project design, baseline, implementation and endline stages.

There are some encouraging trends in MEAL practices. Technological advances in data collection and synthesis methodologies alongside an increasing uptake of adaptive programme approaches are leading many INGOs to rethink the role of MEAL across organisational structures. Key donor priorities are also helping to drive a paradigm shift in the way agencies generate and use evidence to adapt their interventions, take management decisions, and ultimately, demonstrate the value of aid resources to different audiences.

CARE as a confederation is making substantive advances in this regard by sharpening technical standards across all our Outcome Areas and providing operational frameworks that set important benchmarks in the sector for gender, governance, resilience and Value for Money. Download a PDF of our MEAL Core Standards Management Manual.

At the same time, we are establishing a strategy to diffuse MEAL capacity among programme managers, funding coordinators and contract/finance officers in order to create an adaptive system that enables multiple stakeholders to benefit from trends of evidence. Our aim is to integrate MEAL across as many organisational functions as possible. In fact, linking team roles to specific sets of data makes everyone accountable and is a crucial step towards a final synthesis of project learnings.

This abstract concept translates into accessible, logical steps along the whole project cycle from design to endline based on the nature of a specific contract. The proposed four sequential standards and their sub-steps represent CARE International UK programme quality controls and enable us to trace and use evidence throughout the entire project cycle.

1. At design stage: Retrieve-Reflect-Document

The context, risks and MEAL capacities are appraised and weighted at design

  • Retrieve previous experiences of CARE and aid organisations in a given context and previous tracking strategies for all the key risks and assumptions for a proposed intervention.
  • Reflect on gender and governance considerations in relation to the context, power dynamics and barriers preventing the transformation of unequal relations. Specific information is necessary to identify conditions and entry points for a project to be relevant and impactful.
  • Document weighted contextual forces upon submission and track them during implementation after negotiating adequate resources for dedicated MEAL capacity with relevant donors.

2. At baseline stage: Locate-Define

Targeting and sampling methodologies are reliable and enable tracking of a theory of change

  • Locate the target group and define its composition. Therefore, the whole evaluation cycle needs to start under this premise at baseline, especially to meet the explicit requirements of tracking information that are pertinent to a theory of change.
  • Define: 1) what are the key characteristics of target participants, 2) its distribution across target areas, 3) its levels of disaggregation, and 4) its traceability (particularly for nomadic and mobile/internally displaced groups). In cases where CARE and other international aid organisations are building on previous projects, the identification and quantification of targets will need to include specifying results that have been already attained and reducing double counting through better identification systems.

3. At implementation stage: Address-Demonstrate-Review

A monitoring system is in place, functioning and generating digital outcome data

  • Address the critical gap across projects in terms of limited access, diffusion and use of monitoring information – which tends to be collected to meet reporting requirements instead of informing management response.
  • Demonstrate how key indicators, contextual forces and project assumptions are valid and tracked throughout the project monitoring cycle. At this end, a functioning monitoring system is adaptable to contextual forces and generates relevant information for management actions.
  • Review management response in order to ensure the project is increasingly responding to output and outcome changes. The pathway to change led by projects can be demonstrated through adequate tools design and triangulation of monitoring data that directly relates to SDGs/outcomes, their targets and inputs.

4. At endline stage: Appreciate-Learn

Endline data generates representative evidence of change to be used for learning

  • Appreciate that an evaluation is the quintessential thread between sums of actions and conclusive validation of an expected change. In most cases, it represents a synthesis point where expenditures are intersected with evidence of “how” and “why” a change (eg to social norms) happened and the level of our contribution.
  • Learn what “representative” means: how changes within the sample of a whole target population can be generalised for the largest number of project participants. It is a powerful concept that gains great traction with donors and private sector partners, therefore programme teams need to recognise themselves as gate-keepers of valuable evidence that needs to be requested and shared within organisations and with donors as much as possible.

Linking past evidence with new learning

Standards to link past evidence with learnings from new findings: a combination of programme management styles and technological innovations

The current initiative undertaken in CARE International UK aims to bring management practices closer to programme quality and the use of compelling evidence. The key for success is a team of committed professionals who increasingly prioritise dialogue with local counterparts on how evidence supports both reporting and management actions to maximise results.

In that direction, our MEAL Core Standards Management Manual outlines steps and tools for programme management teams to enable them to identify key sources and quickly appraise the strength and validity of a MEAL cycle in relation to a project’s causal claim, methodological approach and outreach targets. The objective remains to harness organisational practice by turning the vision of integrated evidence into the basis for operational frameworks to drive adaptive project management practices.

The inception of this process is usually driven by influencing practice by re-focusing dialogue with local counterparts, research/learning partners and donors. Once priorities and key data are identified and established there are significant opportunities to apply IT solutions that can be tailored based on the management needs of each organisation. For example, real-time monitoring data can today be collected and visualised from the field within a short turn-around time. Programme management functions now have an unprecedented ability to assess how impact happens from evidence collected across difficult terrains. Digitisation also offers the opportunity to link and tag financial expenditures to specific indicators of change or VfM metrics on a consistent basis.

Along with influencing management practices by adopting similar MEAL standards, the benefits from the potential for data integration are clear to see. Advanced database solutions rooted in management practices and shaped in an audience/bandwidth-sensitive way can lead the way. Yet the vast range of options to shape data reinforces the need to ensure its quality at source and management skills to make the best use of it in maximising long-term changes.

Nicola Giordano

Nicola was formerly the Monitoring, Evaluation, Accountability and Learning (MEAL) Specialist within the Programme Management Team at CARE International UK. His main background is in the analysis of large-scale research and impact assessment studies. In his role at CARE International UK, he was responsible for providing technical and strategic support across CARE's portfolio to ensure that changes led by development interventions are logically framed, well-captured, triangulated and shared at different levels.

Nicola's main areas of interests are: data analytics applied to the third sector; digital means to generate evidence; and inclusive project design that can assure feedback loops between implementing partners and targeted communities. Before joining CARE, Nicola worked for a number of governmental, non-governmental and private organisations in the development sector at the international and national level.