Measuring Resilience (No, really! Measuring Resilience!)

DFID’s Work in Somalia

DFID strives to deliver better aid more quickly in some of the world's most complex operating environments. Somalia is a case in point. We are conducting an impact evaluation of resilience-building activities as part of DFID’s multi-year humanitarian programme. It assesses longitudinal change and the impact of different combinations of interventions. We are looking to define what combination of interventions and service providers has the most demonstrable positive effects on other communities. We will then cost those combinations to determine models that could be scaled up across similar communities. In fact, we expect to find a few combinations, such as food distribution, WASH, and education, that have a greater positive impact across community types. 

Measuring Resilience

Resilience is dynamic, as a household’s resilience can crest and trough according to the nature and severity of climatic and conflict-related shocks, and at a much greater velocity than in development contexts.

Trajectories of change in development and humanitarian activities

Our approach considers when shocks occur and then gathers evidence about where households lie on the trajectory of a shock—are they preparing, withstanding, or recovering from a shock. Households may return to where they were, to a common state of equilibrium, but they should do so quickly and with fewer dire effects because of the support they receive from DFID-supported programming.

Shallower and shorter negative impact for intervention communities

Solid Technology and Innovative Statistical Analysis to Measure the Complexity

The evaluation includes a 2,600-household sample, combined with paired counterfactual villages identified by proximity to target villages, general demographic characteristics, and similar livelihood zones. It includes four Household Surveys conducted during the drier seasons in Somalia.

Target communities within a 2k spillover buffer (blue circles) and 10k radius buffer (green) that also keeps within prescribed livelihood zones

The diversity of interventions, combined with the inherent dynamism of resilience, implies that direct single-variable linear relationships, as compared to a control, will not suffice. We are conducting Multivariate Analysis (MVA), which allows analysis of standard outcomes across a range of interventions and counterfactual situations. This includes variance across different outputs (services/interventions) in various communities, as well as other static and dynamic variables. In brief, MVA is a practical way to measure complex systems.

Leveraging Technology to Ensure Data Efficacy

We are using a range of technology solutions to enable real-time quality assurance of enumerators in the field, integrated platforms for data analysis, including easy export to statistical packages, and a range of data visualizations that make the impact evaluation a live, actionable tool for DFID. 

Households near Luuq, Somalia, showing prevelance of gender-based violence.