Stop Guessing What the Most Important Problems Are

Instead, use the SNT Method.

I’ve always liked compassionate people – people who want to use their limited time on Earth to leave the world a better place than they found it.

Unfortunately, most are failing in one critical area: priority-setting. They don’t have a methodology that informs which problems they should be working on. And, as a result, the opportunity costs of their efforts are often sky-high without them knowing it.

But isn’t it impossible to know which problems are most important? And if it were indeed possible, wouldn’t it be obvious which problems are, in fact, the most important?

The answer to these questions is no. 

In this piece, I would like to encourage the use of a  basic framework – the Scale/Neglect/Tractability Framework – to help us figure out which problems are the most important.

I would like to note that this is merely my interpretation of the framework. If you find wisdom in this piece, give credit to Holden Karnofsky and other creators of this framework. If you find mistakes, the blame is on me.

To start us off, I want to explain why we cannot rely on our intuition, popular culture, or fuzzy heuristics to tell us which problems are the most pressing. Some problems are literally thousands of times more important than others, and it’s not always obvious when this is the case.

The Spread Premise

Imagine a world in which every dollar donated to the best anti-poverty nonprofit is worth 1000 dollars donated to the average anti-poverty nonprofit. Every phone call to a Congressman about an important policy is worth 1000 phone calls about policies of average-level importance. Every article written about an important problem is worth 1000 articles written about the average problem. 

Well, that’s the world we live in! But most people don’t realize it. They may believe that some policies are more important than others, but only modestly so. 

If it were the case that the most important problems were only 2x or 3x as pressing as the average problem, then it would be okay for us to make vague guesses about which problems we should prioritize. This is because the opportunity cost – the value of the best available alternative to what we’re currently doing – wouldn’t be that high.

Unfortunately, this is not the case. The differences between the most and least important problems are really, really huge – often counterintuitively so. That means we need to put serious effort into figuring out which problems matter the most.

Without a rigorous methodology to figure this out, we could be wasting money, political capital, and years of people’s time.

The idea that there are enormous differences between the most and least important problems has a name: The Spread Premise. It’s an idea developed by the effective altruist (EA) community – a group of people dedicated to finding out what the most important problems are and acting on that information.

To see the Spread Premise in action, let’s look at a couple of stark examples.

Consider different methods of improving educational attendance in the developing world.

Let’s say we have 4 options:

1. Give cash transfers to girls so their parents don’t have to rely on them for labor. 

2. Hand out de-worming tablets that improve health outcomes for students.

3. Provide merit-based scholarships. 

4. Donate clothing so schools have free uniforms.

So, which of these methods do you think would increase school attendance the most? 

To me, none of these methods jumps out as being the obvious best. Thankfully, global poverty researchers decided to conduct randomized control trials to try and figure out the most effective method of increasing school attendance. The answer? 2: Handing out deworming tablets. 

How much more effective is deworming vs other methods? Check out this chart from William McAskill’s book Doing Good Better, where I pulled this example (Figure 1):

Figure 1

It turns out that, per dollar, distributing deworming tablets increases schooling by roughly 700x more than giving cash transfers to girls! This is not something we could’ve figured out through intuition. The reason we have access to these figures is because people decided to rigorously examine the question.

For our next example, let’s say you want to fund the treatment and/or prevention of HIV/AIDS. We have 5 options:

1. Fund surgical treatment for Kaposi’s sarcoma (an AIDS defining illness).

2. Distribute antiretroviral therapy to fight the virus in infected people. 

3. Prevent the transmission of HIV during pregnancy. 

4. Distribute condoms to prevent sexual transmission. 

5. Educate high-risk groups like sex-workers on HIV.

Again, if I were to ask you to guess which of these solutions did the most good, the answer would not be obvious. But check out this chart from Toby Ord’s essay “The Moral Imperative Towards Cost-Effectiveness” (Figure 2)

Figure 2

As we can see, the most cost-effective option (in terms of disability-adjusted life years) is over 1000x more effective than the least cost-effective option! Again, we know this not via intuition, but rigorous research.

If we were to plot these kinds of distributions on a graph, they would look like Power Law Distributions, where a small percentage of interventions are responsible for an outsized impact. More on this in the following section. See Figure 3:

Figure 3

So far, we’ve established that when it comes to certain real-world charitable activities:

1. There can be enormous differences between the most impactful and least impactful actions.

2. The most impactful actions can often only be identified via rigorous examination. Intuition/simple reasoning is not good enough.

This matters - a lot. Scarcity is a fundamental reality we face when trying to solve any problem. We have limited time, money, and energy we can use to try to make the world a better place.

So if there are such astronomical differences between the most and least effective actions we can take and if it’s not obvious what those optimal actions are, we can’t just make guesses about how to prioritize our resources. We must do rigorous research.

But the cases we discussed have been limited to charitable actions that are relatively easy to measure. So should we expect the spread premise to universalize to other cases? Could it be that, while the most effective charitable actions follow this kind of distribution, political or entrepreneurial actions have a more balanced spread of impact? 

There are very strong reasons to believe that the Spread premise is all around us. For one thing, power law distributions are everywhere, and so we should expect prima facie to see them across a wide variety of domains.

Power Law Distributions are Everywhere

You’ve probably heard of the 80/20 rule. It’s the idea that 80% of outputs come from 20% of inputs. For example, in a business, 80% of productivity might be generated by 20% of workers. Or in finance, 80% of returns might be generated by the top 20% of companies. 

If this concept is familiar to you, you already have a sense for what power law distributions are. The implication for trying to have a positive impact is that the more extreme the distribution is, the more important it is to identify problems that have outsized impact.

Power law distributions can be observed across many different domains. In some cases, like the distribution of family wealth in the United States, one might suspect that the observed power law distribution is artificially created at the behest of the rich (see Figure 4).

Figure 4

Setting the merits of this suspicion aside, there are clearly examples of power law distributions that do occur organically. Take the case of Wikipedia, where the top 0.1% of pages constitute a whopping 25% of all page views (see Figure 5). Or the case of bird species, where across New York State it was observed that roughly 20% of the bird species constituted roughly 80% of the total birds observed.

Figure 5

What this tells us is that power law distributions are wide-spread enough that we shouldn’t be particularly surprised when we see them. There is no quality that exists in charities such that we would expect to see power law distributions in charities but not in other domains like politics,  business, finance, or technology.

With that said, let’s attempt to explain why, if we’re trying to make the world a better place, we might see power law distributions (and the spread premise) across different domains.

What Factors Contribute to Impact?

Our positive impact is the extent to which our actions contribute positively to the common good. But as we’ve seen, not all impacts are created equal. So what are the main factors that contribute to positive impact?

Luckily, there is an entire research project and social movement dedicated to studying this very question: the EA movement.

EA’s have come up with a pretty nifty methodology called the SNT Framework. Its purpose is to use the lessons of economics, rationality, and the scientific method to figure out how to benefit others as much as possible. The basic impact function looks like this:

Impact = Scale x Neglect x Tractability

Here’s the gist: Given the fact that our resources are scarce, we should seek to prioritize problems that (1) are greatest in scale, (2) don’t already receive a lot of resources, and (3) are highly solvable. (1), (2), and (3), refer, respectively, to scale, neglect, and tractability.

We established earlier in this piece that some charitable actions are over 1000x as impactful as other charitable actions. One reason why we should expect similarly huge variance (the spread premise) to hold in all areas of social impact is because there’s always the potential for huge differences in scale, neglectedness, and tractability of different problems.

Furthermore, these differences compound on one another. If problem A is 20x bigger in scale, 5x as neglected, and 10x more solvable than problem B, then we should expect there to be huge differences between the cost-effectiveness of working on problem A vs problem B.

To see why, let’s explore each of these factors, starting with scale.

Scale

Scale is a function of both how many people are affected by a particular issue and how badly they’re being affected (Figure 6).

Figure 6

I think scale is obvious to most people, so I won’t spend a ton of time on it. It’s clear that some problems affect more people and affect them more severely than other problems. 

For example, let’s compare influenza and COVID-19. COVID-19 infects more people yearly and harms them more severely than influenza does. 

Let’s say COVID infects 100x more people yearly than influenza and results in 10x more severe disease per-person. Assuming no new variants of either illness change this calculus, the scale of the COVID problem is clearly much bigger than the scale of the flu problem. The difference is not minor – it looks like a difference of orders of magnitude.

There’s no reason why we shouldn’t expect huge differences – sometimes orders of magnitude – between the scale of different problems. Whether we’re working in politics, charity, business, or other avenues of change, different problems we’re dealing with can have massively disparate effects. So, if we care about making the world a better place, we must take scale seriously.

Now, let’s look at what I think is the least appreciated and understood impact factor: neglectedness.

Neglect

All things being equal, working on an issue that’s neglected is much more effective than working on an issue that already receives a lot of attention. When I say “much more effective,” I mean potentially enormous differences. To see why, let’s familiarize ourselves with the Law of Diminishing Marginal Utility.

The Law of Diminishing Marginal Utility states that, as the amount of some good increases, the amount of additional benefit we receive from that good, per unit, decreases. If this sounds confusing, let’s illustrate with some graphs and examples.

Suppose you own zero cars, but you really need a car to get around and you’re a car enthusiast. Luckily, you somehow win a contest in which your favorite car company will give you one new car per year for the rest of your life. Congratulations! Now how would this affect your well-being?

Obviously, the difference between having zero cars and one car is a large difference in your well-being. The difference between having one car and two cars would probably lead to a much smaller difference in your well-being. But what about the difference between 11 cars and 12 cars? Or 25 cars and 26 cars?

As the number of cars you own increases, the marginal increase in your well-being actually decreases exponentially. At a certain point, it barely adds to your well-being at all to receive yet another vehicle.

This pattern of diminishing marginal utility looks like this graph below (Figure 7), and it appears everywhere in the social sciences. Considering which issue areas are neglected vs which issue areas already receive a lot of attention is crucial in determining what to work on. If it turns out that many of society’s resources are already dedicated towards solving a problem, it would be a huge waste of resources to try to solve that problem, instead of dedicating those resources to solving more neglected problems.

Figure 7

Let’s use a political example. Suppose you have $100,000 dollars to spend on a public advocacy campaign, and you have two choices for where to put the money: a ballot initiative to reform occupational licensing (OL) laws or a ballot initiative to decriminalize marijuana. 

Suppose further that the occupational licensing campaign currently has $200,000 in its war chest, but the marijuana decriminalization campaign has $40,000,000 in its war chest. Additionally, the marijuana decriminalization campaign already has overwhelming public support, but the OL reform campaign has very little public awareness.

In this scenario, all else equal, we should fund the OL reform ballot initiative. 

Why? Because OL reform is far more neglected. The additional $100,000 would increase funding for that campaign by 50%, but would only increase funding for the marijuana decriminalization campaign by 0.25%. This would make the funding increase 200 times more significant for the OL reform campaign than for the marijuana decriminalization campaign.

Furthermore, let’s consider public awareness. If support for marijuana decriminalization is already very high, then the chance of the initiative failing might be quite low. The additional funding, therefore, has little effect on the chances of success. On the other hand, if public awareness is very low for OL reform, and most people who are asked about it don’t have a strong position, then the injection of $100,000 would likely make a far greater difference in the likelihood of OL reform passing.

Of course, it’s possible that the problem of marijuana criminalization is bigger in scale than the problem of overly burdensome OL regulations. But is the problem 200 times bigger? How confident are we that marijuana decriminalization is that much more important? These are good questions, but they are rarely carefully and rigorously considered. This is why the SNT framework involves multiple factors: Scale, Neglect, and Tractability. These factors must always be weighed against each other. 

I do want to flag that it’s not always the case that the more neglected a particular problem, the higher-impact it is to work on it. Sometimes, especially when there are exceptionally few people working on a problem, too much neglectedness can mean there’s very little value in working on that problem. 

Imagine you’re a cancer researcher and you’ve been transported to the year 1200, where there are literally 0 people working on curing cancer (with science, at least). Where do you even begin? Research projects and social movements often benefit from economies of scale, meaning that a certain amount of initial infrastructure is necessary for it to make sense to work on a problem to begin with. 

That said, when it comes to problems that we typically encounter, I think it’s a pretty useful rule-of-thumb to say that the more neglected it is, the higher-impact it would be for you to work on it. There are sometimes differences in orders of magnitude between the neglectedness of different actions we can take. So if we care about making the world a better place, we must take neglect seriously.

Finally, let’s briefly touch on tractability.

Tractability

Tractability, or solvability, describes how easy it is for a particular problem to be solved. If we’re faced with a problem that is easy to solve, then it’s a tractable problem. If we’re faced with a problem that’s extremely difficult to solve, it’s intractable.

Another way of expressing this concept is by asking, if we doubled the amount of resources going towards solving problem X, what fraction of the problem would we expect to solve?

I think most people understand on an intuitive level that some problems are much, much harder to solve than others. For example, getting the state of California to legalize marijuana in 2016 was a relatively easy task. It took hard work, but political headwinds were already moving in that direction by the time voters passed Proposition 64. On the other hand, getting the state of Texas to abolish the death penalty would be a far more difficult task.

How much more difficult? It’s hard to say, but I don’t think it’s unreasonable to suggest that abolishing the death penalty in Texas would be 100x more difficult than legalizing marijuana was in California.

What’s the point of these comparisons? Am I saying that we should only work on problems that are highly tractable?

Absolutely not. But what I am saying is that all else (scale and neglect) being equal, we should put more resources into problems that are easier to solve than problems that are difficult to solve. To not consider tractability would be a serious mistake – a waste of resources.

Some problems – like curing cancer, predicting economic recessions, and inventing safe artificial general intelligence – are very difficult to solve. They require breakthroughs in our fundamental understanding of highly complex systems. We work on these problems anyway because the benefits (scale) would be *so enormous* if we were able to succeed.

But in general,  the harder it is to solve a problem, the more cautious we should be before putting more resources into solving it. If we care about making the world a better place, we must take tractability seriously.

Per this nice demonstration from 80,000 Hours , we can see what happens when we multiply Scale, Neglect, and Tractability together. It seems to simplify (at least in this basic model) to “Good done per unit of resources.” Cool, right?

Conclusion

In this piece, we’ve shown that 

1. There are sometimes vast differences between the most and least effective ways of doing good. I.e. the spread premise.

2. The most effective ways of doing good are often not obvious. We must use careful reasoning and evidence to identify them.

3. The reason for such vast differences is because of differences in the scale, neglectedness, and tractability of different problems and their respective solutions.

4. We should expect the spread premise to hold true regardless of which types of problems we’re interested in solving – whether technological, social, political, or economic.

5. We must use frameworks like the SNT Framework if we want to properly understand the best ways of doing good.

Trying to do good without having a methodology to prioritize important problems is like trying to dig a subway tunnel with a spoon. 

Sure, you can totally do that. But you could also do a lot better.

Figuring out what the most pressing problems are and their most effective solutions is not an easy task. It requires rigorous research. We have to answer tough questions. We have to make difficult tradeoffs. We have to conduct control trials, and when we are unable to, we have to make thoughtful guesses. 

This research project – also known as priorities research – is so essential. And yet, it’s missing from most major social movements. No matter your ideology, worldview, or other framework for understanding the world, if you want to do good without wasting your time, priorities research is essential.

So stop making guesses about what the most important problems are! Don’t assume that whatever everyone around thinks is most important is actually most important. We must rigorously think about these issues with the help of tools like the SNT Framework.

Because for all you know, there could be a problem out there that is 1000x more important than anything you’ve ever considered.