How to measure Value for Money in delivering aid

by 20th Apr 2017
Residents of Torit, Eastern Equatoria, South Sudan wait in line to receive relief items Residents of Torit, Eastern Equatoria, South Sudan wait in line to receive relief items

Launched by DFID, Value for Money (VfM) is now a widely accepted standard in the sector. For CARE, quality assurance is determined by evidence of how monetary resources translate into long-lasting changes. It is understood as a key approach during proposal development and implementation to maximise the integration of Monitoring, Evaluation, Accountability and Learning (MEAL) systems with project management practices.

Value for Money: Basis for data integration and adaptive programmes

In favourable conditions, a well-developed MEAL system includes a multitude of information on progress towards targets upon which performance is assessed and payment delivered. A dependable value for money framework encourages implementing partners to constantly review costs and targets in an integrated way to respond to the evolution of particular indicators and to justify the value of activities in view of long-term results (improved health, income, food security, etc).

An initial context/political/social analysis sets out the major assumptions behind any interventions by establishing the breadth and depth of critical needs of vulnerable groups across specific locations. At implementation, the execution of a VfM framework entails a comparison in time and space of costs and results across operations while promoting data-driven project management. The following sections provide insights and simplified steps of how it could be done.

Economy: Getting the best value inputs

Economy is a function of quality of inputs in their selection, delivery and feedback upon receipt. A credible VfM framework enables tracking of the whole input cycle from procurement of goods/services to their delivery by considering feedbacks, quality, timelines of delivery, suppliers’ reliability, etc.

Given the importance of identifying key and strategic costs, their sequence and how they feed into output targets, an effective VfM framework should disaggregate costs in detail. The most important distinction to be made is between the unit cost intended as the net price of an input and the delivery cost as the amount of resources required to deliver it (eg transport). The definition of what is delivery and net depends on the input and it is critical at the design stage to demonstrate understanding of the context, its cost drivers, and what it takes to achieve activities through evolving timelines.

The next level for economy is contained in the word “benchmarking”: the comparison of unit costs across similar activities that had been implemented in the same area by either CARE or other development partners. The following example explains possible steps to measure it.

 

Generating benchmarks of key costs might not be easy if what is proposed is new in the intervention area or lacks a proxy value. If any data in regards to unit cost has been collected by local counterparts during past programmes and can be retrieved, it would be best to use it.

Efficiency: Maximising the outputs for a given level of inputs

Efficiency tracks how variations of inputs determined by cost drivers and risks affect the delivery of output targets. Monitoring the depth and breadth of outreach per output area requires extensive integration of workplan and budget tools to oversee the cost per project participant over time. With this aim, key questions to consider are:

  • How many activities and outputs do you expect to deliver for a given level of resources?
  • Have you managed to deliver in a timely manner, in line with output indicators and their expected targets while responding to contextual changes?

To answer, the efficiency principle translates into operational tools to calculate cost-efficiency on a regular basis and to describe the trends of cost incidence on output targets. The following example of farmers’ access to drought-resistant seeds presents the steps to produce a ratio that can be compared over time across the same or similar projects.

VfM chart 2

Previous programme experiences should provide enough data to cement cost-efficiency trends and compare them. In the case of indirect participants, the outreach value of specific activities should be estimated in a representative way. For example, a radio message received by local communities on climate information is the population average of listeners to a station programme at a certain time.

Effectiveness: Ensuring the outputs deliver the desired outcomes

Effectiveness can be defined as “the extent to which outputs are converted into outcomes and impacts”. For measuring it, two pieces of information ought to be collected:

  • How many project participants reported an improvement for each outcome area?
  • How much change have they experienced across each outcome indicator and location?

A clear strategy to deliver relevant inputs to target participants in line with their needs, and assumptions-based scenarios for outcome targets, need a strong link with a theory of change. In fact, outcome values are hard to predict and the context analysis informs the logic based on previous programming experiences that can strengthen the assumptions behind cumulative yearly changes.

The choice of indicator to measure cost-effectiveness is the next step. The way this ratio is generated relies on a clear idea of outcome-level milestones, how they are sequenced and linked to costs. When assumptions of how benefits translate into monetary value are clear, a cost/benefit analysis can be conducted; otherwise a simple cost-effectiveness calculation is sufficient.

VfM chart 3

Equity: Ensuring the benefits are distributed fairly to the most vulnerable groups

Projects are supposed to reach the most vulnerable groups through robust targeting criteria and feedback mechanisms that generate evidence on critical needs. Yet, a simple disaggregation of targets might not be enough since a clear justification on proportion of specific types of project participants is necessary when demonstrating longer-term and structural changes. Here is a proposed set of steps to qualify and measure target distribution for one example of equity criteria (nutrition).

VfM chart 4

Looking at one specific area (nutrition), equity should be linked to an outcome change addressing a critical vulnerability that informed the targeting strategy (eg food insecure households) at design. By demonstrating what outcome change the project has achieved and a baseline value outlining the degree of initial vulnerability, CARE is able to link results to a dimension of equity.

Overall, this last dimension remains the most significant. CARE and other agencies need to emphasise the importance of targeting the most vulnerable groups, determined by the community themselves. Value for money can only be complete when a strong equity proposition is embedded and delivered along with indicators that can show how input translates into meaningful changes.

Even if the measurement of change remains a complex endeavour and requires innovative methods, Value for Money frameworks create incentives to improve the practice of interlinking data systems and functions that typically remain in silos – a necessary move to better trace and demonstrate results.

Nicola Giordano

Nicola was formerly the Monitoring, Evaluation, Accountability and Learning (MEAL) Specialist within the Programme Management Team at CARE International UK. His main background is in the analysis of large-scale research and impact assessment studies. In his role at CARE International UK, he was responsible for providing technical and strategic support across CARE's portfolio to ensure that changes led by development interventions are logically framed, well-captured, triangulated and shared at different levels.

Nicola's main areas of interests are: data analytics applied to the third sector; digital means to generate evidence; and inclusive project design that can assure feedback loops between implementing partners and targeted communities. Before joining CARE, Nicola worked for a number of governmental, non-governmental and private organisations in the development sector at the international and national level.