Data: Use It or Lose It!
Humanitarian action is tilting toward the big data revolution, but we need to abide by a few guiding principles to keep from falling over.
In this world of ubiquitous data, we can sometimes get caught up in hyperbole associated with what data will enable us to do. Don’t get me wrong. We need tried-and-tested tools for how we work. Yet, we actually have thousands of tools for every aspect of humanitarian action. It is like the biggest DIY-humanitarian action store on the planet.
What we need are tested models. The models we know work are based on solid evidence. Tools that we've used over and over again so that we know precisely when and how to use them in different contexts and for other needs. Otherwise, we are stuck using hammers and nails when we should be using lathes and injection moulding. To make this shift, we need data.
Big Data: Until Donors Are Willing to Put Up ‘Big’ Money to Go along with ‘Big’ Data, this Remains a Pipe Dream (And, honestly, why should they?)
Big data assesses and tracks broad variations, be they market preferences, social trends, financial fluctuations, commodity prices, or population movements, only the latter of which is related to humanitarian action. Combining social and news media with push-pull telephony can provide near “real-time” information on population movements and/or sentiments regarding a rapidly evolving crisis.
This type of big data analysis is super important, yet we should leave such things to the likes of Google, Facebook, Palantir, or Premise. We will never have the computer's number-crunching power and associated analytics to do the requisite analysis. Nor should we. These players have core competitive advantages and are happy to leverage them toward humanitarian action.
What we should be doing is forging partnerships with these big players. They are eager to learn about the complexities of our work and, if we stick with the Googles of the world, they won’t become competitors. Instead, we can find a win-win within their philanthropic ventures. We can better leverage their big data, substantial computational power, and deep pockets for macro-level analytics as crises unfold.
This is great, but most of our work focuses on the micro-level. We focus on individuals, their families, and their communities. We focus on their needs and how we can work with them to better withstand and recover from a crisis. This requires a range of tools. This is where we find ourselves back in that massive DIY store, searching the aisles for the right tools but with no real experience (data/evidence) to choose amongst all the shiny contraptions on the shelves.
Small Data is Our Friend Too!
We need valid, compelling data at the micro-level. To test existing tools and models, we need to start collecting statistically valid data at the single activity level. This will tell us which tools have worked where and under which conditions, providing insights into that particular context.
The trick is to ensure that any micro-level dataset uses common standards, is complete, and answers key questions about a specific intervention and this type of intervention more generally. These data sets can then become fundamental building blocks. If we use common standards (as below) and well-designed protocols, each data set can be combined to provide much broader trend-level analysis.
These micro-level datasets, once combined, will show which tools worked in which conditions and, when combined with cost analysis, the return on investment for different interventions. We will get much closer to knowing how many people we can serve with which interventions. We can understand which partners work best in which contexts. We can scale up or down different combinations, depending on the needs. We will actually have solid science behind our work, and rather than wandering through the maze of a DIY store, we will have a highly refined, tested toolkit that we can use over and over again, no matter the job.
Stop Messing with Standards
To start collecting data at the micro-level, we need standard questions/protocols that are consistently used across sectors/activities.
We have some standard tools, like SMART nutrition surveys, food consumption scores (FCS), the coping strategy index (CSI), household dietary diversity scores (HDDS), etc. These are great.
The issue is that we need to maintain their integrity! I have seen far too many organisations take one of these standard tools and modify the questions, dropping some and adding others, because they think this will make the tool better. Craziness! Would you expect a doctor to change a standard diagnostic tool because she thought she knew better than the medical community? Would you expect engineers to create their own load-bearing ratios and calculations? Would you want the police to decide what’s wrong and right? Of course not.
We can’t let our overly developed linguistic and analytical skills allow us to be dismissive of accepted models. Use the models. Collect the evidence. Then, it can be compared across activities/contexts.
In fact, we need to go further. We need standard survey protocols for every sector and activity. The Sphere Project has gone some way toward this. (www.sphereproject.org) We need to use these standards—not adapt or change them, but use them, over and over again. Don’t go about re-inventing the mousetrap.
Good God, Please! Move beyond outputs!
I am tired of working with organisations that spend loads of money sending monitors into some of the most dangerous operating environments to count buildings or infrastructure, or to review tally sheets. We shouldn’t need to point this out. We should not ask people only the most mundane output-level questions, e.g., "Did you go to the clinic? Did you receive your Plumpy'nut? How often do you use the latrines?"
I have seen the last one —ridiculous as it is —on countless surveys. Imagine the response: “Umm, uh, once a day???” The question basically asks how often they poo. Come on!
I know how vital latrines and WASH facilities are. I know how important it is to confirm that people get the right amount of food support or that they actually went to the clinic there to support them. We need to validate that organisations have done what they are supposed to, at the right time and for the right people. This is fundamental. However, the cost of data collection is in getting the people out to the intervention sites. If we just ask simple output questions, we are missing a significant opportunity.
We need to ask outcome-level questions. For instance, if the intervention is a clinic, we can ask about the quality of care and the general impression of this type of support within communities. For nearly all humanitarian actions, we can ask about how the support enables individuals, families, and communities to better withstand and recover from shocks. We can ask investigative questions about community dynamics (including host communities). We can ask about traditional coping mechanisms and their relation to direct aid. We can assess shifts in gender relations due to the crisis. We can ask how the intervention enables people to focus on other issues, such as improving food security and livelihood strategies.
Some of these are complicated, and actually collecting valid data can be challenging. Perception questions are influenced by current dynamics, e.g., the person's state at the time of the interview. This means these are, at best, a “snapshot.” If done well, however, even these snapshots can become baselines for other longitudinal surveys. Every piece of data becomes a brick toward a sturdier evidentiary structure. So long as we keep to our standards and use common sense, we can contribute to building something great.
Mobile Phones Are Everywhere. Use Them.
Yes, we should use smartphones and sexy, groovy online user interfaces to manage “live” surveys and perform fundamental analysis. The barriers to entry for these have become so low that no organisation can sensibly refuse to use this basic technology.
We should also recognise that most of the people we serve have mobile phones, and some may even have smartphones. SMS push-pull engagement strategies via telephony are an exciting and quickly developing arena. UNICEF’s RapidPro was an early mover in this arena and remains compelling as it expands to new areas.
Of course, technology also includes card-based cash disbursements, biometric registrations (or even more advanced systems like UNHCR’s IrisScan), and satellite imagery for infrastructure, agriculture, urban contractions/expansions, and waterway changes (droughts and floods). It includes a range of new packaging materials and technologies for everything from clean water to solar power. The issue with these is that they require a cadre of technical experts and only make financial sense when used at scale. These are big tools that need big budgets, similar to big data. Don't let the hoopla in the press melt your hard, rational humanitarian heart. Get back to what works for the people we are there to serve.
This brings us back to the humble survey. Surveys should be integral to every intervention. They should use standard protocols and move beyond simple output measures, as described above. We should conduct these on phones, tablets, or laptops, using geo-referencing and collecting video/audio information as applicable.
We use smartphones in the most sensitive areas in Somalia, so there are no real security issues. In fact, if anyone anywhere is still using paper and pen, they should be fired and/or have their funding halted. Pen-paper-Excel is prone to massive errors, and it simply isn’t necessary. The ubiquity of survey software and mobile interfaces makes using pen and paper seem like a relic of some distant past.
Make Your Graphs Pop!
Once you have the data, don’t waste it on monotone layouts or impenetrable graphs generated from the latest Excel package. The graph is a perfect distillation of data into a user-friendly visual that demonstrates trends and differences and that answers specific questions unequivocally. Don’t mess around with shite! Make your graphs pop!
In brief, they should be simple enough so that a lay person can understand them, use different but complementary colours for variables (although avoid using more than five colours), use complete words/phrases for legends and labels, and include expository text in the body of the report that explains the key issues.
In fact, we should move to interactive reports that use graphs/charts and maps to show progress over space or time. Hans Rosling’s Gapminder (www.gapminder.org ) is an excellent example of this.
Get the Data Out of the Hands of M&E People
A common complaint is that data remains within M&E teams rather than becoming an operational tool. I hate to say this, but this is typically due to two basic reasons. First, the M&E team is being overly sensitive and territorial about its data. This should be remedied by direct managerial action. Data is a tool to be used, not a tool for reports. Second, the data, its graphs, its tables, and any exposition are too poor for operational utility. We will write a specific post on this. Still, really, if operational people aren’t using your data, it is probably the data and analysis, not the operational people.
Share Data Far and Wide
Like an M&E team that hoards its data, any data hoarding should be discouraged. Data is meant to spark conversations, provide new insights, prompt project adaptations or changes, and deepen individual and organisational learning. It can only do so if it is shared.
Fine. Take the time to QA it, but don’t let it drag on so long that it becomes dated. Fine. Share it first with those immediately concerned, but then broaden it to other divisions/departments, organisations, and god forbid, even to donors.
Share it warts and all. It needs to be shared so we can get better. It needs to be shared so that all our actions improve. The people we are there to serve deserve this level of analysis, thought, and improvement.
