Gilbert’s key points
First, what were Gilbert’s key points? His blog was a reaction to the recent discussion paper ‘How International Non-Government Organisations are Doing Development Differently’. He argued that despite emphasising the importance of development being locally led, the DDD community of practice and its advocacy has not been led by development practitioners and reformers in the global South. Without this, the DDD community cannot have a complete understanding of the practical application and implications of this agenda in the South, or how the narrative will be received. Gilbert also argued that the DDD agenda is problematic because its aspirations are unclear and, in particular, does not appear to have clarified whose vision of politics and development it aligns with.
Gilbert is organising a series of roundtable meetings in Nairobi from May onwards to discuss this further – see the end of this blog for details.
A discussion (see below) between CARE’s Tam O’Neil, Senior Gender Advisor (former Research Fellow at Overseas Development Institute in the Politics and Governance Team, one of the convenors of DDD) and Rebecca Haines, Senior Governance Advisor – Asia and MENA (who has worked for several multi-mandate INGOs, the World Bank and has managed a variety of donor funding, including from DFID and USAID, in programme implementation countries for over eight years) led to four suggestions for how the DDD agenda can better listen to and engage Southern civil society:
- Creating a DDD narrative that is a good fit for field level practitioners, including listening to how they describe their adaptation needs and problems.
- Actively testing the assumptions underpinning the DDD agenda done with a diverse range of practitioners in the global South.
- Acknowledging how the current political economy of how aid is distributed makes it difficult for NGOs and CSOs in the global South to engage and align with the DDD agenda.
- Changing public expectations, to allow for more honest conversations about how the ways in which donors currently fund aid can undermine risk-taking by NGOs and other development practitioners.
Tam and Rebecca’s exchange, similar to Jake Allen’s recent discussion of old dog, old tricks, further explores how southern civil society perceives Adaptive Development and reflects on the practical challenges and barriers faced by NGOs and their field teams when engaging with the DDD agenda.
Rebecca: The issues raised by Gilbert are really important! The DDD narrative (and the Thinking and Working Politically agenda for that matter) are indeed still very Northern-led and often don’t register clearly with most of the colleagues I engage with in our country offices and/or among our partners. It’s problematic to have large investments in niche communities of practice based in the global North, that quite often don’t appear to be actively testing their language, framing, and assumptions with a diverse range of practitioners in the South (and adapting accordingly!). And while these communities of practice are challenging the sector to improve in some useful ways, they’ve gained a momentum that is approaching orthodoxy without robustly including the views of people outside of quite an elite circle.
My feeling is that one of the reasons that many INGOs have been slow to engage is because the DDD narrative actually doesn’t always feel like a good fit for us – for the reasons Gilbert raised, and also because adaptation happens in so many small daily undocumented ways on the ground that the sudden interest in codifying adaptive development sometimes causes our field programmer colleagues to scratch their heads. There are still useful reminders within the DDD narrative that should push us to do better, but I do feel the narrative resonates more with donors and think tanks than with field level practitioners. In that sense, I don’t really agree with Duncan Green’s characterisation that INGOs are ‘finally catching up’ in this discussion – it feels to me more like this discussion has always felt a bit out of sync for us, but that we now feel compelled to engage because of the optics around these issues at the moment.
Tam: The point about who the DDD agenda speaks to is an interesting and relevant one. Even within the DDD community of practice people have argued that the importance of adaptation for development and its practice is not new. The response from others is usually ‘we know but we want to change the way that donors think and fund development and move them away from universalist paradigms and approaches like the Washington Consensus or Good Governance discourse.’
Rebecca: I do agree we still need changes in the way donors fund development – my personal experience with how difficult it was to lobby (read: beg) donors to allow for mid-programme logframe changes aligns with this. When I worked in a country office in Afghanistan, donor representatives did (in some cases) struggle to understand why design elements and assumptions defined years previously may not still be valid, in a rapidly changing conflict context. And even if the in-country donor representative was convinced, her/his ability to convince their superiors back home was sometimes limited. In several cases I can recall, the in-country donor representative on the ground literally recommended that we kind of collude together in ‘bending’ our reporting to still fit a three-year-old logframe, to avoid the interminable bureaucracy of getting changes approved. These things certainly need to change, but some of it is about more agile systems (while some is indeed about mindsets).
Tam: To play devil’s advocate, and think of the challenge that DDD advocates might respond to that with, if NGOs are routinely doing this, why do we still (often?) get programmes recreated that have not worked well (at all or in the absence of other complementary interventions) or why do we still have programmes that make claims about higher level impact that are clearly unrealistic in scope or ambition?
Rebecca: I would say there is a whole political economy around the cases (and there are many) of re-created programmes that haven’t worked well the first time. I’ve certainly seen it happen, and it often had to do with incentives – either the pressure on the NGO to ‘sell’ a method they had invested a lot of time and reputation in, the pressure to compete with a crowded NGO field and peers that are also ‘selling’ pretty hard, the pressure on the donor’s lower level in-country representative to argue that their portfolio is high-performing, the pressure on a donor agency to sell results to parliament and protect their budget, and the list could go on. And beyond incentives, replicating a model because a country office doesn’t have time or resources to design a new one, replicating a model because the way evidence has been documented is sloppy, etc.
Basically, I think there is a whole range of reasons why people at every level don’t admit or recognise that something isn’t working, and don’t make the necessary changes to improve in a timely fashion. It might have to do with not knowing how to adapt well, but there are a lot of drivers beyond that (which I think are more culpable). I’m not sure the DDD platform really diagnoses those problems well. To the extent to which the DDD agenda is focused on fostering a climate of honesty around results (with fewer penalties around failure), I think that’s useful; however, again I think this is heavily donor-facing (and maybe needs to even be public-facing).
However, in the cases where theory of change claims are far too ambitious – I agree that this is a large problem. But I think this has to do with a skills gap, which is quite predictable since very few of our colleagues ever get training on theory of change development. Even those who studied development most probably graduated at a time when everyone was still being taught to build logframes (as I did), and most of our workforce have never been trained on logframes either. Theories of change are, after all, still a Northern-driven development community structure, albeit a useful one (in my view).
Furthermore, (and likely more significantly), there often also feels like there is systemic pressure to ‘dream big’ and articulate an ambitious vision – I’ve certainly sometimes felt that donors (and head offices of INGOs) reward the most ‘inspiring-sounding’ pitches, rather than the most realistic. On the donor side, this is partly because their systems typically rotate people quickly from post to post, so they are often ill-equipped to evaluate what is realistic in a given context. In short, theories of change are just as much marketing tools as anything else, in a competitive proposal development process; if we therefore sometimes find them unrealistic, it should really be no surprise.
Tam: To play devil’s advocate again, there’s a difference between practitioners responding to changes in the external environment (eg, a drought interrupts or changes programme activities) or even changing course because activities are not working well (a form of adaptation) and when they set out without a clear idea of what the causal pathways might be or establishing different possible routes to the sought-after change, and then trying them out in a purposeful way (that is, using experimentation and adaptation). Are NGOs routinely doing the latter?
Rebecca: This is a really important point – I do think the practice of testing pathways is weak in most single programmes (except for the lucky few which were given the latitude to be designed that way explicitly). But I also think this happens much more when one backs up and looks at a whole portfolio over a 10 year period, which is probably a more realistic way of seeing testing and adaptation – every long-standing organisation has a ‘story’ of how they’ve tested and adapted over long periods of time (CARE Bangladesh can give you a more than 20-year story on supporting local governments to be more responsive to rural poor households, for example). But we can absolutely do this in more intentional and systematic ways. However, I would go back to my previous refrain in saying this is fairly donor-facing – you’re simply not allowed to design most programmes without presenting your selected causal pathway. Of the 10 or so concept notes or proposals I’ve given inputs for so far in 2018, I have yet to see a single format that creates space (in the narrative or in the budget) for unclear causal pathways. Of course there are very notable exceptions to this, but they represent a very small portion of funding.
Tam: In thinking about when and how an adaptive approach should be used it seems useful also to disaggregate types of NGOs and their activities more. For example, I feel like all this applies much less to more advocacy/activist activities than the larger development organisations and implementers. Small NGOs and activist groups may be trying out different tactics in practice but we should not expect them to divert scarce resources to documenting this. However, we should expect larger NGOs providing accompaniment/supportive type roles and development organisations designing and implementing programmes, whether for profit or not, to be doing more purposeful experimentation and certainly be questioned if they are repeatedly not achieving outcomes, eg in the health or education sectors. But, as I’ve not managed or implemented programmes, this may be a poor distinction or unfair on colleagues working on these programmes!
Rebecca: Yes, I agree that our expectations on documentation should vary. In terms of whether smaller CSOs are likely to be more agile – maybe, I’d have to think about this more. Examples that both confirm and contradict spring to mind – local NGOs that have been remarkably agile, but also local civil society orgs that are very traditional and don’t like to try new things. And we also need to remember that many small local NGOs do only direct service delivery, and often do this in the absence of being familiar with global evidence around what works (more a traditional local charity model, often based on religious or moral imperatives, but not very professionalised or connected to others in a community of practitioners). While there are many impressive examples of very locally-connected activist human rights CSOs, women’s rights movements, etc, by the numbers, the former description is still probably the majority of local NGOs worldwide. Also small budgets can make you risk averse as your margin of error is narrower, and doing one thing that doesn’t work can fold a small CSO. So the ‘small bets’/fail forward narrative does to some extent come from a position of privilege (the privilege to take risks).
Tam: What’s your view on the (lack of) documentation issue? From the perspective of doing research and evaluation at ODI it was frustrating because it seemed that it leads to such waste. Without documentation, learning across programmes (different ones or phases of a programme) depends on people's tacit learning. From the outside, the failure to build on tacit learning seemed an important factor in programmes repeating problems, not achieving outcomes or even potentially doing harm.
Rebecca: I definitely think the lack of documentation is extreme and a big problem. And agree that it hampers good research and evaluation terribly. I personally think some of the tools that outcome mapping uses (the programmer’s journals and things like that) are interesting options for improving on this – if I was back in the role of a programme manager in the future, I would want to give that a try with my team. That way we gather information along the way, while hopefully avoiding the habit of sending someone to write a case study at the end with little to go on, which I think often glosses over so many of the interesting twists and turns along the way (mostly because no one remembers them by that time).
Tam: And is the question of documenting or not (and how much design is informed by evidence and analysis) mostly about time/money available for these activities, or are there other disincentives?
Rebecca: I think time and money are big issues, but donors and INGOs have also distorted documentation by pushing so hard to get people to write ‘success stories’. There’s a real tension between our instincts when it comes to documentation – is this about honest learning, or is this about selling? Programme staff get mixed messages constantly, to the point where they are almost always primed to be in ‘selling’ mode when external actors are around. That’s a pretty big sectoral problem, and I would argue that it’s again donor-driven. (Reporting formats still regularly ask for ‘success stories’.) There’s a pretty significant cultural shift that is needed, in terms of asking people to document the good, the bad, and the ugly, and this cultural shift would require a patience from the public that is currently on the decline.
We also can’t underestimate the risks that come with candid documentation – many colleagues are strongly restricted regarding what they can say and certainly what they can write down, especially if it involves naming local actors or explaining the political reasons that some programmes might not achieve their planned results. But there are probably ways to mitigate these risks.
This blog was co-authored by Rebecca Haines, Senior Governance Advisor - Asia & MENA, Tam ONeil, Senior Gender Advisor, and Hayley Capp, Governance Learning Officer, CARE International UK.