The collection, analysis, and use of evidence to validate or inform the quality of an intervention, or its component elements, is core both to the work of Geneva Global and to the guidance and support we provide to clients and partners alike. Quite simply, if we say we’re going to do something, we feel it is critical to demonstrate that the strategies we planned to do this have indeed happened and that they succeeded. Specifically, we must check that we achieved the expected results, or better.
Geneva Global’s commitment to evidence comprises a common sequence of four basic steps.
- It starts with a clear and precise Theory of Change.
This is a simple statement of what change or outcome one expects and of the intervention to achieve that outcome.
- Then, we identify precise SMART goals.
For each dimension, it is essential to identify clear and measurable goals to appraise the fidelity of implementation and the attainment of the expected outcomes.
- Next is the timely collection of data for each key performance indicator (KPI).
Often the most challenging step, this can require a variety of program actors, ranging from an intervention’s managers to its beneficiaries, and occurs repeatedly.
- Last are the analysis, sharing, and use of the data.
This step gives meaning to the overall endeavor: the time, effort, and resources that go into the collection of evidence must lead to discussions, decisions, and actions that serve to validate and strengthen the concerned intervention and heighten its impacts.
The multiple benefits of evidence are exemplified in Geneva Global’s experience with the Speed School program in Ethiopia. Speed School is an award-winning accelerated education initiative that delivers the first three grades of primary education to out-of-school children aged 9 to 14 with the aim of bringing them into (or back into) the formal primary school.
In an independent study conducted by the University of Sussex and Hawassa University, “Tracking the Progress of Speed School Students: 2011 – 17” (available here), researchers found that Speed School has enabled over 150,000 children who had previously abandoned, or were abandoned by, the education system to return to their studies in conventional government classrooms and to perform there successfully. This is nearly 95% of the total children who enrolled in Speed School classes over the program’s first eight years.
Speed School, which is implemented with about 20 local civil society organizations—“Implementing Partners”—on behalf of the Legatum Foundation (originally), Luminos Fund, and two anonymous clients, is carried out with the ultimate goal of serving the government in pursuing its education access and quality goals nationwide.
What We Learned
The Sussex/Hawassa study basically asks whether the learning that students acquire in Speed School classes translates into sustained learning and progress in the official school system. The answer provided is a solid “yes.” Not only have children transitioned from Speed School classes to conventional school classes, but they are excelling there.
Compared to their classmates who have done all their schooling in conventional government classrooms, the former Speed School students:
- score better on language and mathematics tests
- are routinely among the best in their classes
- drop out less frequently
- are class leaders
- have more ambitious professional and personal aspirations
- exhibit greater learning and other personal competencies
Since the program’s launch in 2011, Geneva Global has featured monitoring and evaluation as a critical element of its strategy. In addition to a comprehensive internal data collection protocol, the program’s key funder at the time, Legatum, contracted with University of Sussex to design and conduct a robust research agenda to evaluate independently the Speed School model, its implementation, and, especially, its effects, working with Hawassa University. Over the years, Geneva Global has also engaged local education authorities and another Ethiopian university, Mekelle University, to conduct additional external research and evaluation.
Overall, the accumulated evidence has combined to give confidence that Speed School is appropriate to the Ethiopian context and is yielding the desired effects. More practically, it has also served to inform many significant decisions and actions, allowing Geneva Global, its Implementing Partners, and the government to continue to strengthen the model, its impacts, and its potential in two major ways:
One, it provides valuable technical insights. The routine flow of evidence, both from internal and external sources, has provided valuable data and analysis to motivate and guide substantive improvements to the model and its component methods. It has informed as well how the implementing partners use these and how Geneva Global trains and supports partners in their use. Lastly, it continues to point to the need (and opportunities) to add new components to the overall model.
Two, it serves to attract others to emulate and support the program. From an advocacy angle, the existence of relevant, robust evidence has stimulated greater government interest and collaboration, spurring central and regional authorities alike to launch a process with Geneva Global to transfer full responsibility for the program to the government over the next few years. In fact, three regions have already launched 110 of their own classes this year, representing 14% of the current total. The ability to demonstrate the impact of Speed School with concrete evidence has also brought new international funders to support the initiative; and domestic funders are also showing interest.
Why Monitoring & Evaluation is Essential to Program Success
Evidence-based decision-making has driven education and pretty much every other development sector for many years now. This movement is captured in A World that Counts, a publication of the United Nations that aims to launch a “data revolution for sustainable development,” asserting that:
Data are the lifeblood of decision-making and the raw material for accountability. Without high-quality data providing the right information on the right things at the right time, designing, monitoring and evaluating effective policies becomes almost impossible.
Just in the past month, the Global Partnership for Education shared an article from the Khmer Times with the headline “Better data, better Education.” Therein is the assertion that:
By accurately collecting and analyzing education data, teachers can now develop lesson plans and school programmes that can help them to better respond to the specific needs and challenges their students face.
Too often, implementers and, even, funders see robust data as a luxury or a distraction, diverting resources and effort from the “real work” of delivering good outcomes. This attitude is detrimental for two main reasons:
- It presumes that all outcomes are good or cannot improve.
- It deprives others of the opportunity to objectively and precisely perceive an intervention’s value and to be motivated and guided to participate in or borrow it.
Extrapolating from the experience of Speed School in Ethiopia, the sorts of discussions, decisions, and actions that evidence can inspire include:
- A validation of an intervention’s core model and strategic elements, signifying its relevance for broader adoption
- The identification of any shortcomings or challenges
- The articulation of solutions to these shortcomings and challenges
- The identification of other related interventions by which to strengthen and even expand the outcomes from the original Theory of Change, letting the successes of an intervention stimulate even greater ambition
Planning approaches to the collection, analysis, and use of evidence is built into the warp and woof of Geneva Global’s work across its clients. Whether helping to validate and communicate the attainment of goals of their initiatives, demonstrating the impact of initiatives implemented on their behalf, or making the case for investing in a new area or with a new organization, evidence represents a fundamental strategic element. Try it. You’ll like it.
