The most effective way to do this, though one that is not always feasible (due to cost or lack of available data), is to get empirical evidence of your impact. With this, you can assess whether your organization’s efforts are delivering the change you want to see. Of course, you can rely on anecdotal evidence or your own judgment, but these are not substitutes for rigorous evaluation.

Efforts to eliminate malaria are instructive. Insecticide-treated bednets are known to be effective tools in preventing infection. But experts disagreed over whether nonprofits could better promote the use of bednets among vulnerable populations in Kenya by giving them away or selling them cheaply. Those advocating for selling the nets (for a nominal fee) believed this would screen out those who would not use the nets and increase the likelihood that those who bought them would use them. Yet a randomized control trial^* provided clear evidence that the better strategy was to give nets away.⁶ Now, based on this research, a number of highly cost-effective nonprofits like Against Malaria Foundation have built their missions around giving away free nets.

2. A Powerful Fundraising Tool

Clear evidence of impact provides validation to your existing donors, and allows you to make a strong case to potential donors. Donors—from members of the public to the largest foundations—want to know that their support will make a difference. In the absence of evidence, we tell them stories and share details of activities. These can be compelling in building engagement, but they are no substitute for evidence of genuine impact.

One of the most important recent trends in philanthropy demonstrates this. Donors have long been resistant to the idea of giving cash to poor individuals in developing countries, instead of, say, providing food or job training or shelter. They worried that support in the form of direct cash payments would cause local prices to rise and stoke resentment from community members who did not receive cash. Underlying these concerns were deeply held and often discriminatory beliefs that poor people don’t know how to spend money responsibly and shouldn’t be trusted to make their own financial decisions. Over time, evidence built up by early pioneers of cash payments have led to widespread donor support and significantly increased funding for cash transfer programs⁷ and the growth of effective organizations like GiveDirectly. Studies in rural Kenya have shown that basic income not only positively impacts individual households (which invest in things like livestock and better housing) but also benefits those in nearby villages and provides a stimulus for local economies. This evidence has disproven negative donor assumptions and prejudices and built support for not just GiveDirectly but the whole model of direct cash payments to reduce poverty.⁸

3. Mobilizing Others

In addition to helping you raise resources, robust evidence of impact allows you to amplify your influence, accelerating progress in achieving your mission. It helps you mobilize others in support of your cause. Evidence of impact can generate media interest. For example, stories about Give-Directly’s impact have appeared in the Economist and other major publications and generated additional support for its mission and work.

Evidence of impact carries more weight with policymakers because they are often inured to inflated and unverified claims of nonprofit impact. And it can aid the work of peer organizations—particularly those who cannot afford to invest in robust impact measurement—as they can utilize your findings in their own programs and communications. Perhaps most critically, evidence can unlock more sustainable sources of funding and uptake by governments. In the case of cash transfers, longterm investment in evidence eventually convinced governments to begin using cash transfers as a form of poverty alleviation, with governments like Pakistan⁹ and India¹⁰ now distributing hundreds of millions in cash transfers to their populations.¹¹

Another example comes from the organization More in Common, mentioned earlier. It works on a wide range of programs to reduce community polarization and the threat of “us-versus-them” divisions. For several years, they have advocated Canada’s model of community-based refugee sponsorship as a policy that both inspires greater public confidence and results in better outcomes than traditional top-down refugee resettlement. More in Common conducts extensive polling of public perceptions, and they have consistently found that significantly higher numbers of Americans and Europeans will support the intake of refugees when they are directly sponsored by local communities. This evidence has been used widely by networks of refugee advocate policymakers and elected officials and has been highly influential with the US and UK governments in particular.^*

4. Accountability to Those You Serve and Your Staff

Properly measuring and communicating impact also allows your organization to build accountability and trust with the individuals and communities your organization serves, and provides them with evidence that you’re upholding your commitments. It’s not just donors who care about how charitable funds are spent. Those you serve also care deeply. If you want community members to actively participate in your work and give feedback, you need to show them you’re holding up your end of the agreement. Providing concrete evidence of impact is also helpful if you work in an environment where, for whatever reason, people tend to be skeptical of civil society groups. The same goes for staff. Effective measurement of impact gives your staff a clear understanding of the change that their work is contributing to and can be a powerful motivating force.

THE CHALLENGES OF MEASURING IMPACT

When it comes to measuring impact, nonprofits commonly make four errors.

They measure overhead, not outcomes;

They measure what’s easy, not what counts;

They decide that it is “impossible” to measure impact, so they don’t try, instead of trying to find suitable proxies; and

They fail to distinguish between when they are having direct impact and when their impact is indirect (for example, by influencing systems).

We’ll explore each in turn.

1. Overhead Is Not Impact

Given the challenges of assessing nonprofit performance, those seeking to do so (most often, donors) often turn to one thing they can measure easily—overhead. But overhead is an input. It is an organization’s administrative and fundraising costs as a percentage of total expenditure. Some arbitrary overhead percentage (10 or 15 percent) is often set as a target. These costs are easy to measure, as they will be broken out in the organization’s budget, but they tell you nothing about the organization’s impact.

Overheads are a necessary cost of running a nonprofit. If you want to raise funds, you need to spend money on fundraising. If you want high-quality staff to carry out your ambitious mission, you need to invest in recruitment and training and benefits. If you want to run a tight financial ship, you need a strong finance team and good financial systems. All of these costs are overhead. Of course, it’s always appropriate to ask whether or not you are spending the right amount on administration and fundraising, but that is about good management, not impact.

The superficial attraction of using overhead as a proxy for impact is the assumption that an organization with lower overhead is more efficient than a peer organization with larger costs. But this is not necessarily the case, as overhead is only relevant to the extent it drives impact or diverts resources from it. Nonprofits that invest more in their staff and internal systems will have higher overhead, but that may translate into greater impact than a peer that spends less on staff and internal systems. It may also reduce the risks of something going wrong, such as financial malfeasance.^* Those who judge organizations primarily by their overhead lack a fundamental understanding of how nonprofits work, and they overlook many high-performing organizations in the process.^†

Business practice can provide a useful counterpoint. Because they have a clear metric to measure performance—namely, financial returns— no one seeks to compare businesses primarily on their overheads. Investors in Amazon or Tesla are not spending a lot of time comparing those organizations on the basis of what they spend on their sales, finance, or human resources departments. Rather, they’re focused on their financial returns (impact) and factors that most directly impact those returns over time. Nonprofits should be judged similarly.

2. Measure What Matters

Inputs and activities are much easier to quantify than the results of those activities. But as has been wisely observed elsewhere: “Not everything that counts can be counted, and not everything that can be counted counts.”¹²

Let’s focus on the second half of that statement. As we saw in the previous section, overhead is often used as a measure of impact, as it’s easy to measure, even though it doesn’t tell you much about impact. The same often goes for activities. In this section we’ll look at how the Freedom Fund moved from measuring what was easiest to count, to measuring what really mattered. And then we’ll look at the different situations that think tanks and advocacy organizations find themselves in as they try to measure their impact.

As I set out in the introduction to this chapter, at the Freedom Fund we started measuring direct impact in the form of “lives liberated” (as this was easier to count) and only later added other, more indirect, measures of impact, such as measurable reductions in slavery at the community level and at a broader systems level. This was our effort to measure what counts (i.e., overall reductions in the level of slavery).

The sustainability of your impact also counts. While the number of people in situations of trafficking (or poverty, hunger, etc.) in a community may decrease over the life span of a two-year program, will the rates go back up after the program is over, or after a few years? Are the interventions taking aim at alleviating the symptoms of a problem, or actually getting at root causes? If long-term sustainability is a part of your impact targets—that is, you aim to actually make a permanent or at least longterm impact on a person, community, or issue—then simply measuring impact at the end of a short-term program won’t give you a good sense of whether those changes will stick.

These examples show us that the most obvious, direct output figures often don’t tell the whole story, or even the right one. One way to improve impact measurement when it comes to modern slavery (or poverty, or hunger, or other societal change) is to focus on prevalence, i.e., the percentage of the population affected by the issue you are seeking to address. If we can start by establishing a baseline prevalence rate in a particular area before we start a program, then (ideally) we should be able to compare that to prevalence three, five, or ten years later to understand change over time. Using this approach, the Freedom Fund has been able to build a robust body of evidence, most of it generated through evaluations by leading research institutions, that shows that our approach has resulted in dramatic reductions in the prevalence of modern slavery in targeted communities over four-year periods.¹³

The second step to measuring prevalence reduction is assessing attribution. For example, modern slavery is tied to complex economic, social, and political systems and trends, so we can’t automatically assume that our programs have caused a documented fall in rates of modern slavery; there could be other causes. For instance, if at the same time we were working with a community, the police (independently of our collective efforts) launched a sustained crackdown on traffickers, that would likely cause slavery numbers to fall significantly, and the change couldn’t be attributed to our programs.

One way to assess attribution is to have a “control group,” which in the Freedom Fund’s case would be a group of vulnerable community members not being served by our programs, but similar in all other respects to the community being served. If the situation of the group being served improved more than the control group, that would be a strong indicator of the positive impact of the program.^*

Rigorous prevalence studies are expensive and time-consuming, so we also measure activities that are strongly tied to a reduction in vulnerability to modern slavery—e.g., the number of at-risk children helped to return to school, the number of people provided with access to social and legal services, the number of micro-enterprises started, and the number of legal cases supported. More broadly, we also look at overall lives impacted by Freedom Fund–supported programs to understand our reach in comparison to funds spent.

The Freedom Fund’s method of measuring impact is appropriate for our size, approach, and resources, but it won’t be a fit for every organization or issue area. We have found success in tracking both outputs and outcomes over time. We’ve found that different audiences tend to seize on specific indicators that they deem important, and it has taken some effort to educate donors about why effective prevention efforts should eventually result in lower numbers of people liberated from slavery (because there will be fewer people left to liberate). But with a range of metrics that indicate lower levels of vulnerability to exploitation, complemented by long-term prevalence data, we are able to paint a much richer and more complex picture of the Freedom Fund’s impact.

Think tanks face a very different challenge. Their difficulty in measuring impact often leads them to measure what they can count, not what matters. Think tanks will invariably count how many reports or opinion pieces they have published, or how many conferences they have convened. They may track how often their experts appear in the media, and how much traffic their website receives. These are all straightforward to count. And they are certainly a way to track productivity and profile. But they don’t really capture impact.

This is because think tanks rarely see their primary purpose as publishing reports. Rather, their purpose is usually defined as generating ideas and evidence and influencing policymakers. As Ken Roth, the former CEO of global human rights organization Human Rights Watch, put it, his priority was “[e]nsuring that our programmatic work is always impact-oriented—that publications are not ends in themselves but tools to change governmental behavior.”¹⁴

Counting reports or conferences won’t tell you that. It won’t tell you if those reports informed policy change or if they were even read. Measuring influence can be particularly challenging for think tanks and advocacy organizations because there are no direct measures. Policymakers rarely come out and say, “We took X action because we were persuaded by Y’s report.” However, that is not a reason for not trying to measure impact. In these cases, the best way to measure is via proxies for influence.

3. When You Can’t Measure Directly, Find Proxies

When it is not possible or feasible to measure the impact of your work directly, then you must find proxies that give some clarity, and measure those rigorously, tracking your trajectory.¹⁵ Perhaps the change you are looking to make is quite broad, and you need to find a specific metric that can be tracked relatively easily. For example, many international public health organizations looking to measure a community’s access to healthcare use the percentage of children who have received the third dose of the diphtheria, tetanus, and pertussis vaccine as a proxy. These organizations figured out that since the vaccine requires three doses and therefore three separate interactions with the healthcare system over time, the third dose serves as a fairly accurate indicator of how strong and accessible the system is.¹⁶ In some low-income countries, especially when economic data is sparse, nonprofits use the replacement of thatched roofs with more expensive corrugated metal ones as a proxy for poverty reduction.

CASE STUDY

How to Measure the Impact of a Think Tank

Faced with an economic recession, Arthur C. Brooks, the new president of the American Enterprise Institute (AEI), a free-markets think tank in Washington, DC, was under pressure to articulate the organization’s impact more convincingly. But since much of what think tanks do is indirectly influence policy, a clear link between AEI’s reports and its intended target, the thoughts and actions of American leaders, was hard to identify. Traditional evaluations like public opinion polls or legislative votes were not helpful in evaluating how demand for AEI’s ideas measured up with those of other organizations. So Brooks turned to a set of proxy metrics that, when combined, allowed AEI to assess how well their ideas were being taken up. Over time, they tracked metrics like how many op-eds they landed in the most competitive media outlets (such as the New York Times, the Wall Street Journal, and the Washington Post) and how often they were invited to participate in congressional testimonies as compared to other think tanks. Each of these proxies was of limited use on its own but, when combined, Brooks could argue that AEI was living up to its definition of success: the influence and uptake of the ideas it generated. These metrics weren’t only helpful for persuading donors; they also helped AEI leadership better assess how its programs were contributing to its ultimate goal.¹⁷

4. Indirect Impact and Systems Change

Organizations that aim to tackle thorny, deep-rooted social issues like poverty, gender inequality, and violence know that change doesn’t happen through one type of intervention; many contributory factors need to be addressed. You don’t sustainably reduce domestic violence just by providing shelter for victims or tackle poverty simply by providing cash handouts, even though these may be important interventions as part of a package of measures. Many factors are at play in these situations, and sustainable change requires addressing a number of them at the same time.

Complex systems—be they economic, political, social, or some combination of the three—call for complex solutions. As this understanding has taken hold in the nonprofit sector, the term “systems change” has emerged as a buzz-phrase. Though definitions can vary widely (Does it mean addressing root causes? Adapting to complexity? Bringing in “social entrepreneurs” who catalyze change across sectors?¹⁸), those of us focused on systems change can all agree that it is difficult to achieve and equally difficult to measure. If we are seeking to address these complex systems, we have to find ways to measure how our efforts and resources contribute to overall change, despite the challenges. This will often start with mapping out the various actors and issues that contribute to the problem, working out which of those actors and issues you can influence, and estimating how that might impact the system as a whole. As noted, none of this is easy, but if you don’t try then everyone invested in your work is being asked to take it on trust that you are making a difference.

CASE STUDY

Measuring Changes to Communities’ Vulnerability to Slavery

At the Freedom Fund, our work initially focused on direct support to those in slavery or at high risk of it. It was relatively straightforward to measure this impact in terms of individuals receiving services (such as victims liberated, individuals accessing social and legal services, at-risk children in school). But we soon became more ambitious and began to explore how our programs could address the systems that led to, and kept people in, slavery. The priority was not just getting individuals out of places where they were already being exploited but ensuring that those at risk didn’t end up in slavery in the first place. This meant making sure that laws were properly enforced and government officials did their jobs (instead of turning a blind eye to exploitation); it meant helping vulnerable individuals get access to government benefits and subsidies they were entitled to, so they didn’t get pushed into economic desperation; it meant changing norms, for example, by running information campaigns so people didn’t just assume this was always the way things were done, and that nothing could or should change.

Our partners were engaging with governments and businesses, helping strengthen local social protection systems and running social and behavioral change campaigns. They were having striking success with these joined-up efforts, and we recognized this would have long-term, sustained impact far beyond the life cycle of our programs. When a law is changed, a legal precedent is set, or a business changes its sourcing practices, the effects are felt by populations far larger than those with whom our partners directly interact. But our impact measurement framework was set up to collect direct impact figures (e.g., numbers of individuals reached by specific services, numbers of people sustainably liberated from modern slavery), and other key proxies (e.g., numbers of media stories, prosecutions, and children returned to school). We had no way of counting the impact on those we couldn’t see. We weren’t measuring our impact on the system.

So we turned to our research team and gave them a challenge: experiment with a framework for measuring “indirect impact.” (We define “indirect impact” as “beneficiaries who do not come into immediate contact with our grassroots partners and program activities, but nevertheless gain from the systems change that our program has contributed towards.”¹⁹) We also set a new organizational target of reaching ten million individuals by 2025 by focusing on and measuring our contribution to systems change. This target was driven partly by interest from our board, as well as an observation of the great potential for systems-level shifts in our program areas and our desire to capture it.

Our team began by analyzing various forms of systems change and coming up with a three-level system of categorization: protective environment, resilient communities, and empowered movement. Each of these categories describes a different level of impact, ranging from the broadest population affected by changes like national legislation and public attitudes (“protective environment”) down to the more micro shifts experienced by stronger effectiveness and collaboration between anti-slavery leaders and organizations (“empowered movement”). For each change that we have either observed or are seeking to achieve in our programs, we place it in one of these three categories and, using available data, estimate the size of the population that would benefit. By the end of 2022, we estimated we had indirectly impacted over seven million people.