However, these issues were the right challenges to focus our conversation with Emily Janoch (Director of Knowledge Management and Learning). The podcast, and this resulting blog, discuss how the Tipping Point Initiative’s cluster RCT simultaneously drives our ability to influence the discourse on child, early and forced marriage (CEFM) while presenting real challenges to our feminist principles of gender transformative programming and CARE’s mission and values. As Tipping Point is just one piece of this partnership, I strongly suggest you read a complement to this post, authored by Cari Jo Clark from Emory University, the co-principal investigator for Tipping Point’s research in Nepal.
Tipping Point’s journey
Tipping Point is a multi-country initiative addressing CEFM by focusing on its root causes. We see CEFM as an act of violence in that it allows the appropriation of mainly girls’ unpaid household labor; the refusal to permit girls control over their sexuality and reproduction; and a tolerance of girls’ vulnerability to gender-based violence. Tipping Point Phase 2 is a structured and integrated approach to address the underlying causes of CEFM – we focus on girls’ individual agency related to sexual and reproductive health, life skills, and economic empowerment while simultaneously bolstering and transforming the relationships around them and shifting the discriminatory norms that act as barriers to girls’ individual and collective agency.
To accompany a more focused Phase 2 package, we sought to widen our sphere of influence by: (1) developing robust evidence on the added impact of gender norms programming for CEFM prevention and (2) contributing to the evidence base on effective, holistic, and replicable packages to address root causes and adolescent girls’ rights-focused programming to prevent CEFM.
Why we chose a Randomized Control Trial
Other methodologies, such as contribution tracing and qualitative comparative analyses, have been discussed and vetted in the international development and evaluation discourse for impact evaluations due to their contributions on understanding “how” and “why” change occurs. However, experimental designs – and RCTs in particular – remain lauded as the gold standard to understand “what” works. This standard has largely been defined by funders and by researchers conducting meta-analyses as they generate and rely on summaries of the evidence base on particular issues and interventions to set their agendas (for example see the DFID How To Note on Strength of Evidence). For better or worse, donors and researchers constitute highly-prized voices driving philanthropic and institutional policy on girls’ rights and CEFM, in particular. Thus Tipping Point’s choice: we decided on the RCT methodology paired with multiple qualitative evaluation methods to uncover the “why” in addition to finding out what works, speaking to these audiences and our peers, which is in service of the long-term goal of Tipping Point to influence the discourse on child, early and forced marriage (CEFM).
Why are RCTs so hard on programs?
With over four years of community-level programming under our belts, a large-scale research study was a new and significant undertaking for our country-office and global teams. RCTs come with a slew of methodological demands to maintain the integrity of the research – these are particularly present in the sampling and data-collection phases of the research before programming begins. For instance, even though CARE Nepal put in tremendous effort and a large amount of time to identify all potential sites for implementation, it is nearly impossible to gather such in-depth information quickly or at all – not to mention while balancing feasibility, logistics and need as CARE does in non-study settings. Thus, our research partners completed a random selection of geographic clusters within that list of potential sites, a census of households in selected clusters, and then a randomized selection of eligible participants within enumerated households. Each of these steps prioritizes alignment with RCT standards, and for good reasons: external validity is greater, claims of generalizability are more credible, and the research is more compelling to change policy and practice. When you compare these steps to how communities and participants would be selected in a non-study setting, we experienced complications for the delivery of program activities and the teams’ relationship with communities. For CARE, an organization that focuses efforts on the most vulnerable and hardest-to-reach communities, this process presented not only new types of communities – such as semi-urban contexts – but also socioeconomic differences with the rural communities experiencing extreme poverty normally designed for and engaged with in the previous phase of the project.
Adapting to new and contextually specific communities is part of our work, but logistical concerns outside of the program’s control also were prominent. Efforts to ensure the random selection of participants strongly influenced group formation – while we weren’t able to prioritize communities most in need or participants clustered around an accessible community center, in some ways this stretched us to test our model in more difficult circumstances. In one community, this process meant that half of the randomly selected girls would be unable to participate due to both a river and jungle in the middle of the cluster that were deemed unsafe to cross, completely blocking participation. Ideally, we would have eliminated this community from the list of potential sites if all stakeholders were given enough time for that discussion and negotiation. For most communities, this process meant much longer distances to travel in order to participate. Such distances may be feasible for adults with access to a bicycle or greater mobility, but for girls participating in weekly sessions, the heavy demands of school and household chores can quickly dampen their own enthusiasm and their parents’ approval to participate.
After jumping through these hurdles, teams were excited to implement the Phase 2 package, which integrates social norms in each component but with particular emphasis on girls-centered movement building and engaging decision-makers and key stakeholders. Social norms change requires programs to go beyond individual- and family-level change and transformation to community-level shifts – many times facilitated through organized diffusion. The catch: by purposely choosing stakeholders that hold influence and power to enact change above community-level, we are potentially contaminating nearby control areas.
Making it work – and finding extra “wins” in the process
More than ever, the project team is flexing our adaptive management muscles to make the program and research a success. We’ve worked with community leaders to bolster participation, adapted the location of meetings and session length to address girls’ time and mobility constraints, and created allies at the community-level to communicate the program’s objectives as well as structure and rationale for the accompanying research to explain how and why we randomly select participants. Also, CARE selected the sites for qualitative data collection to ensure a balanced representation of the most significant socio-economic influencers for CEFM.
While the need for “fresh” communities previously caused tension with CARE’s Program Approach to comprehensively addressing the needs of communities, the LEAD program in particular has moved to enhance social norms change approaches in other program areas while we implement Phase 2.
Finally, the Tipping Point’s Implementation Fidelity Standards were developed in response to the need to track implementation progress and adaptations needed across two countries within our learning outcomes and against the study’s objectives. The added bonus: we now have detailed documentation for adaptation and scale processes for other teams and contexts.
Capitalizing on the opportunity in front of us
Investing in research has obvious returns; however, due consideration of the challenges and limitations is essential to avoid tipping the balance away from communities’ needs. Grounding the partnership between researchers and programs in a mutual understanding of CARE’s approaches starts by prioritizing program design – in coordination with researchers – at the start of research design.
Programs can work to center communities’ needs during implementation by seeking an understanding of the balance between “gold standards” for research and adaptive management – noting that this balance is ill-defined. Striking this balance requires communication and coordination between all implementers and researchers to strike a shared agreement on where this balance lies. For instance, designing our engagement with community stakeholders prioritized the program quality and potential for diffusion for social norms change, and later it was communicated to research partners who offered solutions for the research to account for this risk of contamination with control areas.
Another benefit from using a RCT in Phase 2 comes from our ability to build on Phase 1, which offered critical learning and evaluation tools before undertaking the RCT. Phase 1 was novel in its deep formative research and then highly participatory evaluation methodologies – providing a more comprehensive picture of the challenge we were taking on with Phase 2 programming and how our approaches already were working. This formative effort was foundational to decreasing the number of surprises or the need for large course-corrects. This experience underscores that qualitative research can be rigorous and is vitally important – therefore, any RCT should be accompanied by rigorous and well-integrated qualitative methods.
If programs can be realistic about the costs – including the up-front investment of staff time, they can make informed decisions about whether and how to take on a RCT. Such studies take up significant time and money – be really clear on why you’re doing an RCT and what you’re going to use it for. Furthermore, for organizations like CARE, we need to understand the ethical implications of how we treat control communities. Each project needs to think about their working relationships with those communities, and in turn, the implications for field-level staff, especially when those communities will not directly benefit from the intervention during the research period.
Finally, RCTs may be the least participatory method for evaluations – so we believe it’s important to reconcile this disjuncture from centering girls’ voices and experiences via both research and program-based solutions. Phase 2’s girls-centered movement building, adapted from the EMpower Learning Communities approach, uses girl-led monitoring methods. This method is paired with monitoring activities that highlight the voice and experiences of CARE and partner staff to understand their own pathway to gender transformative change in their personal and professional lives. In Phase 3, we hope to fulfil our feminist principles to treat girls and their communities not as the producers of knowledge that benefits others, but as active participants, fulfilling our mission and upholding our values while seizing the opportunity that RCTs bring through proactive research and learning agendas.
Don’t miss our research partner’s take on this partnership, the struggles and opportunities they’re working through alongside our teams, and what INGOs like CARE can do better when undertaking research!
Anne Sprinkel is the Tipping Point Initiative Director.
Dipendra Raj Sharma is Team Leader of CARE Nepal’s LEAD Program. He has worked in the development sector for more than two decades, and for the past three years with the Tipping Point initiatives. His roles are to manage the project cycle, mobilisation of resources, and supporting partners for effective implementation of sessions and activities.
