Digital technologies and innovation in international development cooperation
Challenges and solutions by monitoring and evaluating
International aid money should be spent wisely. The number of people that are displaced is higher than ever and the world is not on track to reach the sustainable development goals. At the same time, budgets for international development cooperation are limited and the (inter)national urge for accountability of these budgets is increasing. This stresses the importance of efficient and effective spending of development cooperation budgets in general, and of monitoring and evaluation (M&E) budgets in particular. We explore to what extent digital technologies, combined with a diligent application of innovation methodology, could prove to be of help to overcome these challenges.
Challenge 1: Mitigating the lack of resources - by experimenting with digital tools
M&E experts within NGOs in the Netherlands report a lack of resources to execute their work. Still, their efforts are key to ensure that resources for cooperation and aid are used in a way that maximizes their outcome and impact for beneficiaries. For the M&E budget, therefore, a focal question is: Can data collection and analysis be performed in a more efficient manner? It is often assumed that resource scarcity leaves little room for innovation in the way data is collected and analysed. The use of digital tools carries a perception of high costs and high uncertainty, which causes organisations to refrain from experimenting with them. However, experiments to find out which tools can positively impact daily monitoring and evaluation operations are key to reaching (cost) efficiency, effectiveness and reliability in the long run.
The untapped potential of digital tools for M&E
A survey we conducted among M&E experts presented them with digital tools for data collection and analysis. All surveyed experts admitted they don’t use half of them. That this is not always a well-considered choice can be derived from the fact that more than half of the respondents are not familiar with many of the tools mentioned, and thus have not explored whether they could be helpful. Exploring the potential of the tools that are not yet commonly used, could increase the efficiency of M&E activities by reducing manual and administrative tasks for M&E experts.
A way to overcome the high cost/high uncertainty paradigm: starting small
The apparent hesitation to leverage the potential of digital tools may well stem from concerns about costs and uncertainty. Testing new technologies without knowing their effect on beneficiaries, which are often vulnerable populations, is an unacceptable risk. Yet, if you start with small experiments to uncover the potential for your organization, uncertainty decreases. Small experiments—or as we like to call them: design sprints—are cheap, fast, and low-risk. When these short experiments prove either successful or not, swift action can be taken to adjust projects or redesign experiments, ultimately finding the right solution that fits your challenge.
A human-centred design approach to increase the effective use of resources
The likelihood of success increases when organizations adopt a human-centred design approach. A human-centred design approach requires putting beneficiaries, and a thorough understanding of their needs and wants, at the heart of the programme design process. The chance of meeting actual, rather than perceived needs increases radically.
M&E experts can play a crucial role in fostering a human-centred design approach by developing M&E frameworks that are not only based on impact creation (does the project deliver the benefit we expected?), but also include actual value creation (is this something beneficiaries want?).
Challenge 2: Overcoming the lack of usable data - by using new types of data
For-purpose active data
Looking at the data sources the survey-respondents use, we can conclude that they primarily use data actively provided by beneficiaries requested to do so specifically for the relevant project or programme.
Non-purpose active data
Non-purpose active data is data that is actively provided by beneficiaries, but not requested specifically for the purpose of M&E, such as social media information.
Passive data
Passive data is information not actively provided by beneficiaries. This data, such as geodata from satellites or drones, can also provide valuable insights for monitoring and evaluation. These types of passive data are used by only 14% of the respondents.
The use of non-purpose active data and passive data is an interesting addition to active data collected directly from the beneficiaries because it adds objective information, but also because it allows for much more frequently collected monitoring information. Including and combining these different types of data in your M&E research increases the richness and the reliability of the data and opens up previously impossible ways to triangulate M&E information. This greatly contributes to being a data-driven and flexible organisation.
Challenge 3: Accessing beneficiaries in remote locations - with remote sensing and geodata
Beneficiaries of aid are often located in remote areas. It can be a costly undertaking to send M&E specialists to these locations, especially if the organisation strives for a certain level of frequency or even continuous collecting of monitoring information.
Remote sensing
Remote sensing technologies enable M&E specialists to monitor ongoing changes from a distance. As counterintuitive as it might seem, you can even use Internet of Things (IoT) devices such as sensors off the grid, with no internet or mobile phone coverage.
Geodata
Geodata, captured e.g. by drones or satellites, combined in geographic information systems can rapidly increase the information available on hard-to-reach locations. Geodata allows the user to observe (changes in) certain geographic areas. The development of smaller, lightweight, intelligent, network-connected and sometimes autonomous geodata tools has increased the accessibility of this technology.
Challenge 4: Ensuring the uptake of M&E results - with incremental learning loops
One of the most reported issues that M&E experts encounter is insufficient uptake of M&E results. 88% of the respondents state they encounter this problem, of which nearly one third finds the uptake of M&E results a big challenge.
Feedback loops to ensure continuous learning
To ensure the process of learning as quickly as possible, organisations are increasingly using the Build-Measure-Learn loop. This is a feedback loop where you test hypotheses by first building an experiment with the minimal version of the change you want to test. You then measure, or monitor, the change, collecting the most relevant data to validate or reject your hypothesis. Based on the measurement you can learn, draw the right insights from the monitoring data, and make an informed decision about whether or not you are on a fruitful path and want to iterate further.
At PwC we have developed our own innovation process and an Experience Centre to help you with it.
Rapidly changing environment and intervention needs a new M&E framework and a different kind of indicators
While the overall goal of international development programmes is generally quite clear, the specific actual problem and solution are often less evident and subject to change.
The M&E practice should then, as quickly as possible, generate insights from monitoring that facilitate the need to learn what works and what does not. Systematically, the most important assumptions in the theory of change will be (in)validated.
As the world of international development cooperation is changing equally, or even more, rapidly than in other sectors, measuring whether everyone is sticking to the plan becomes less important than measuring if the plan is leading the implementation in the right direction. Therefore, M&E experts should add an indicator reflecting the speed of learning within a development programme.
The speed of learning can be measured by indicators on the costs per learning loop, the validation velocity (speed at which critical hypotheses are validated) and the knowledge-to-assumption ratio (amount of unknown assumptions the intervention has). Ideally, the speed at which the effects of changes in interventions are measured should be increased from sometimes years to weeks or even days.
Donors can catalyse innovation efforts
Donors can play a key role in encouraging innovation and experimentation within the development cooperation sector. This requires a change from the more traditional course in which success is defined as the successful completion of a project, including an implementation that is exactly as planned in the application phase. Donors need to step away from this and define success more in terms of a successful learning curve.
Suggested solutions require the right expertise
In our survey, 79% of the M&E experts report they lack knowledge to choose the most appropriate data collection tools, and 87% report they lack knowledge to choose the most appropriate data analysis tools. Social media analysis and drones stand out as tools that most respondents had never used but of which they did see the added value. When asked why the organisations had not adopted them yet, 70% of the respondents indicate they do not have enough expertise and/or knowledge on how to use these tools.
Diversify the pool of M&E experts
Iincreased complexity of data collection and data analysis means M&E staff needs to be well equipped to deal with these intricacies. Experts with STEM-profiles (Science, Technology, Engineering and Mathematics) and designers would certainly be an asset.
PwC has an Experience Center which consists not only of advisory experts but also of programmers, user experience designers, industrial designers, and other creative professionals that allows you to experience what a (organizational) change will mean for your beneficiaries. Prototyping, mock-ups, and simulations will help to rapidly gain insight into the next steps.
Do not generate digital data that you cannot protect
A lack of knowledge and expertise can even be dangerous. Insufficient understanding of data protection and privacy laws can result in ethical challenges or unforeseen negative consequences.
Do not use technology when collected data cannot be adequately protected and is so sensitive that it could put people at risk. It is critical to develop a thorough understanding of who has access to the created (meta-)data in the specific context, and to take necessary action for data security.
Do not reinvent the wheel
Issues that development cooperation organisations are trying to solve are often highly complex and inevitably connected to multiple stakeholders and institutions in their specific societies. This requires new relationships with public organisations, companies, research institutions and citizens to reach sustainable solutions.
Also more practically, there is a lot to learn from pilots and projects that have already succeeded (or failed!). It is helpful to reach out to other organisations that have succeeded in implementing new technologies or tackling complex issues with innovation methodologies.
Conclusion
M&E professionals report to encounter a number of challenges with data collection and analysis. Firstly, they face a lack of resources and usable data, especially for beneficiaries in hard to reach locations. Additionally, there is currently insufficient uptake of insights from M&E. Exploring new ways of data collection, using digital tools, can complement current data with new data types that enrich the insights. Collecting data digitally can also reduce costs and (therefore) increase the possible frequency of the data collection. Innovation methodologies ensure that experimenting with applying digital data collection tools does not equal high costs at high uncertainty, as it puts the focus on starting small and experimenting fast. This methodology can also be used to quickly learn what parts of an interventions are working. M&E then becomes a central component of a programme to enable quick measuring and learning.
Originally, innovation methodologies are developed for complex environments of extreme uncertainty in tech companies. Working in complex and extremely uncertain environments is of course equally, if not more, true for organisations active in international development aid. Although innovation has become a popular term in the development sector, and there are examples of organisations in the development sector that are leading by example, there seems to be a gap between the potential and actual (large scale) implementation. The development sector cannot be compared to tech companies and, arguably, innovation in the development sector is much harder than in the tech sector. The international development sector faces specific obstacles such as highly restricted budgets and working with vulnerable people for whom the notion of experimentation may seem irresponsible. We believe that if we apply innovation methodologies, adapted and applied to this specific sector, it can greatly benefit the sector to increase the value, reach and impact of its efforts. Just like private sector companies use new technologies and innovation methodologies to maximize their shareholder value, international development cooperation can use them to their advantage to maximize impact.
Related content
Follow us
