Monitoring, evaluation and learning: Adaptive management to achieve impact results

By Team: Authors 18th Jan 2017
An information point for recipients of cash transfers in Zimbabwe An information point for recipients of cash transfers in Zimbabwe

One of the trendiest buzzwords in the development and humanitarian sector at the moment is “adaptive management”, which carries heavy weight in focusing on MEAL practices while remaining neutral to political forces and the increased commercial pressures on aid spending. But what does adaptive management mean in practice and what are the key considerations to bear in mind in relation to programme design and implementation?

1: Recognise the importance of using evidence when taking management decisions

One of the greatest benefits from institutional donors’ increasing focus on adaptive management is a widening recognition of the need to learn, iterate and adapt. Particularly when a programme presents multiple components or when the context is volatile, project managers are supposed to enable rapid changes as soon as evidence points out sub-optimal processes, lower results or changing forces in the context of intervention.

The ability to act based on evidence varies significantly depending on the willingness and expertise of programme managers and MEAL focal points to work together and review multiple kinds of information (financial, programmatic, operational) at various stages throughout a project life cycle. The current trend indicates that input financing is not enough for donors anymore; demonstrating long-term changes is the condition for funding.

2: Design a management approach that is established on adaptive practices

A recent study commissioned by BOND indicated there are two dimensions to consider when choosing between a more rigid or flexible programme approach. In fact, there are two kinds of information needed at the design stage of a new project: knowledge of what causes change and knowledge of the context where project activities will be implemented.

By addressing these two important questions, it will be easier to understand the kinds of data needed to inform how decisions are taken. For example: do we know enough about the target groups and their needs? Do we have enough equipment to improve a certain health/food issue in the areas with the most acute exposure to it? Are we addressing a climate shock or a long-term drought, or do climate extremes alternate?

BOND graphic 

As the table above shows (taken from BOND, Adaptive management: What it means for CSOs, page 11), the more we know about the context and what causes results, the more linear the implementation will be. However, in many cases the issue the project seeks to address is caused by unpredictable shocks that require fast response, or by complex development issues that require a lot of learning to address effectively, like social or policy change. In some cases, for example, selected health outcomes would be more measurable than social empowerment as scientific methods are applicable to the treatment that is typically fully controllable (eg vaccines).

All these variations inform the management style and the speed needed to make decisions, either to adapt to evolving risks or to consolidate results. In both cases, data and adequate methods to inform the programme manager are necessary.

3: Design is the beginning of success when it sets realistic outcomes and targets

For the reasons outlined above, the choice of suitable and feasible outcomes is necessary at the design stage. Since outcomes and impacts represent long-term changes – empowering women financially and socially, building resilience to climate change, reducing chronic food insecurity, etc – the ability to forecast and attribute these kinds of behavioural or social transformations can be complicated without iterative learning about the context and solutions to the issue under consideration. Linear thinking does not always apply to long-term changes – for example, social risks among target communities need to be taken into account when new behaviours are the expected result of a programme.

4: Adapt to donors by expanding monitoring from short-term to long-term changes

Deciding how transformational an intervention could be – which for CARE means addressing the CARE 2020 outcome areas – also requires a full understanding of direction of travel in delivering aid as set by influential donors.

The table below, taken from a DFID technical guidance document on designing and delivering payment by results programmes, illustrates a shift from 100% funding upfront to 100% funding on delivery. This change from input financing to output/outcome-based payments has profound implications on the modality to use aid. In fact, constant evidence review becomes more critical to justify management decisions when implementing project activities that aspire to be transformational and with a large outreach.

DFID direction of travel graphic 

Managers are supposed to start considering rigorous use of mixed methods (qualitative and quantitative evidence) to inform quick responses to changes in contextual forces or assumptions on how long-term changes can be achieved. The Girl Education Challenge Fund, under which CARE delivers programmes in several countries, is an exemplary case of how DFID wishes to proceed in delivering long-term and strategic funds.  

Traditional ways of measuring change through evaluation surveys and loose monitoring systems are inadequate when seeking to demonstrate long-term changes. A culture of proof needs significant organisational shifts to prioritise skills growth in generating and analysing evidence of structural changes and their scale in intervention areas.

A possible approach could be strategy testing, focused on monitoring the interests and incentives driving key actors and linking them to efforts to promote critical reforms. Outcome changes can be achieved only by considering how to navigate an intricate landscape of different understandings from multiple stakeholders on how long-term transformation should unfold. Therefore, a plan of activities cannot be rigid and needs constant feedback and reviews from all key partners involved in implementation.

5: Develop negotiation strategies with stakeholders and funders

One of the first questions that could come up is: why do all these complexities need to be embedded in contracts when there are clear needs in a context of new or chronic shocks? The reason is that a transition from reactive to forward-looking aid entails greater risks. It is hard to predict and forecast in numbers long-term changes even though political representatives and the public are requesting greater scale – in depth and breadth – of results.

There is greater agreement around situations of immediate response, to a natural shock for example, but this branches out in many directions when solutions need to apply to chronic marginalisation and less tangible needs, like empowerment and legal rights. However, in figuring out what works, there is also scope and space to negotiate, since donors themselves recognise the need for constant learning in volatile and unpredictable contexts. They are open to resourcing ways to access adequate evidence and inform all their stakeholders.

CARE, along with other development partners, can legitimately request adequate funding for delivering credible monitoring, evaluation, accountability and learning systems. How to frame and budget for them needs to be based on the ultimate goal, in order to boost data quality and the ability to turn complex evidence into simple and compelling messages that can inform management and the public.

The negotiation will be finally successful when evidence on results from programmes is generated and used to model costs and predict targets. The story of change will then become a narrative with multiple facets but based on well-appraised facts from the field, where change happens in measurable form.

Nicola Giordano

Nicola is the Monitoring, Evaluation, Accountability and Learning (MEAL) Specialist within the Programme Management Team at CARE International UK. His main background is in the analysis of large-scale research and impact assessment studies. Currently, he is responsible for providing technical and strategic support across CARE's portfolio to ensure that changes led by development interventions are logically framed, well-captured, triangulated and shared at different levels.

Nicola's main areas of interests are: data analytics applied to the third sector; digital means to generate evidence; and inclusive project design that can assure feedback loops between implementing partners and targeted communities. Before joining CARE, Nicola worked for a number of governmental, non-governmental and private organisations in the development sector at the international and national level.

Email: giordano@careinternational.org