I spent a decade in the DRC watching people in air-conditioned offices in Geneva debate the nuances of how impact is evaluated, using metrics that looked beautiful on a PowerPoint slide. They talked about “sustainable empowerment” and “systemic shifts,” while I was staring at a crumpled manifest for a shipment of oxytocin that had been sitting in a humid warehouse for three weeks because the bridge at Kasindi had washed out. To those analysts, the project was still “on track” according to their KPIs, but to me, the math didn’t matter if the medicine was rotting in the heat.
I’m not here to teach you how to build a more sophisticated spreadsheet or how to massage qualitative data to please a donor. I’m going to tell you what actually matters when you strip away the jargon and the sentiment. We are going to look at the hard logistics of accountability—the difference between a successful report and a successful delivery. If you want to know how to measure if you are actually changing anything on the ground, we need to stop looking at the theory and start looking at the cargo.
Table of Contents
The Fallacy of Measuring Social Program Outcomes
The problem with how we talk about measuring social program outcomes is that we’ve become obsessed with the math of “why” something happened, rather than the reality of “if” it happened. I’ve sat in briefing rooms where people debated the merits of randomized controlled trials in impact assessment as if they were discussing theology. They want to isolate a single variable, stripping away the chaos of a broken road or a seasonal flood, just to prove a point. But in the field, you don’t have the luxury of a sterile environment. You can’t run a perfect experiment when the fuel supply is uncertain and the local warehouse is understaffed.
The academic push for strict attribution vs contribution in evaluation often ignores the messy reality of the ground. We spend millions on complex statistical models to prove that our specific intervention caused a specific change, when in reality, progress is a blunt instrument. If a community’s health improves, was it our vaccine drive, or was it the new well built by a different agency three towns over? We get so caught up in the intellectual gymnastics of proving we were the sole cause that we lose sight of the fact that the service was delivered. If the metrics look perfect on a slide deck but the supply chain is failing, the math is meaningless.
Why the Spreadsheet Misses the Point of Arrival
I spent a decade watching academics try to squeeze the chaos of a supply chain into a neat little box. They love to talk about randomized controlled trials in impact assessment as if a controlled environment exists once you cross a border with a heavy load of vaccines. You can run all the math you want to isolate a single variable, but the math doesn’t account for a washed-out bridge in South Sudan or a corrupt checkpoint that decides your fuel budget just doubled.
The problem with relying solely on high-level modeling is that it confuses statistical significance with operational reality. You can apply a difference-in-differences methodology to a dataset and claim you’ve proven a program worked, but that doesn’t tell me if the cold chain held during the final fifty kilometers. A spreadsheet might show a positive trend in health outcomes, but it won’t tell you that the crates were sitting in the sun for six hours because the driver didn’t have the right paperwork. In my experience, if you can’t track the physical movement of the asset, the data is just a ghost.
The metric that matters
You can spend millions on longitudinal studies and qualitative surveys to prove a program’s success, but if the supply chain breaks at the last mile, the data is just a collection of expensive lies. Real impact isn’t a percentage increase in a report; it’s the physical presence of the goods at the point of need.
Yusuf Lindqvist
The Bottom Line
We spend far too much time debating the nuances of qualitative metrics and the elegance of our data modeling. But after years of watching supplies sit in customs or vanish in transit, I’ve learned that a perfect impact report is useless if the actual utility never reaches the person in need. You can measure sentiment, you can track long-term behavioral shifts, and you can build the most sophisticated dashboards in the world, but none of that matters if the foundation is broken. Evaluation shouldn’t be about proving we did something good; it should be about verifying that the chain held together from the warehouse to the final mile.
If you want to truly understand the success of a mission, stop looking at the glossy summary pages and start looking at the manifests. Real impact isn’t a theoretical victory captured in a donor’s quarterly review; it is the unbroken sequence of a shipment moving through chaos to reach its destination. We need to stop chasing the high of a successful fundraising campaign and start obsessing over the grit of the delivery. At the end of the day, the only metric that carries any weight is this: Did the goods arrive?
