by Carl V Phillips
I taught a class today to a group of public health students, with the theme that policy recommendations made based on an empirical study of a risk factor (e.g., an epidemiology study about the health effects of a behavior, or a study of the chemicals found in e-cigarette vapor) are never justified. Or, in the terms of this blog, are lies (not in the sense that they do not reflect the authors recommendation, obviously, but the claim that the recommendation follows from the study is a lie). There are five distinct reasons why making such recommendations are inappropriate. That list is, I think, rather informative for disciplined thinking about promoting THR, so I thought I would share a summary of the basic points from the class with my rather larger audience here.
I started out by asking them if they had ever read a paper where the authors do a single study about a possible risk factor and then make broad policy pronouncements at the end. I interrupted before they answered to assure them that I was joking – they are in public health, so of course they have read papers like that.
As motivating examples for the discussion, I had them read the post from a few days ago, about proposals to either ban cigarettes or drastically reduce the nicotine content, and read enough to know about plans to develop nicotine “vaccines” that would prevent someone from experiencing the effects of nicotine. I also threw in Bloomberg’s soda ban (I love it when the lead headline in the New York Times is on-topic for the day’s class!).
The reasons why it is a lie to tack on policy recommendations to a risk-factor study:
0. The results of the study might not be right.
I did indeed start the counting at zero because this one is a bit different. It is not about the wisdom of the policy, but about the study result itself. A single study does not give us a definitive assessment that an exposure causes a particular outcome. If it is the only such study that exists (which is rare — happens only once per exposure+disease, obviously) there is still whatever other knowledge we might have. In theory a good paper could review the other evidence and draw conclusions about the totality of the evidence, but that is exceedingly rare (it usually requires a dedicated review paper to try to do that). Thus, the implication of the particular study in isolation cannot even tell us too much about the risk, let alone how to respond.
Note that this applies to studies that suggest there is no risk. Indeed, even more so. The same possible errors that might cause a single study to exaggerate a risk could also cause it to miss a risk that really exists. In addition, there are plenty of ways to do a study that will miss a phenomenon even if it exists. Thus, pointing to a single study and claiming it is evidence that we do not need to act is an even greater mistake. (Thus the reason that I and CASAA make it a point to avoid doing that.)
1. The proposed policy might not accomplish the goal.
It might be that an exposure is really causing a disease, but that a specific proposed intervention might not actually reduce the disease even though a naive knee-jerk impression say it might. It might even be that no conceivable intervention could accomplish the goal, so even a general “something should be done to…” recommendation cannot be justified.
For example, Bloomberg has been furiously attacking the overturning of his soda ban by repeating observations about obesity being a problem. But would banning 20 ounce Cokes do anything significant to reduce obesity? The best guess is “no”, but more important, there is no reason to believe the answer is “yes”. Governments like to engage in the “logic” of saying “there is a problem and something must be done; this is something; therefore this must be done.”
2. The intervention might create other harm (in the same realm where it is intended to do good — i.e., it might cause other health problems).
Bloomberg also moved to make food less flavorful (by reducing salt); this tends to make people want to eat more and thus become obese. The proposal to reduce the nicotine in cigarettes would make them less appealing, no doubt, but it would also cause many people to smoke more of them. The question of whether an intervention might cause other health problems is not answered by the study of a particular exposure+disease combination.
3. There will be costs to implement the policy; is it worth it?
The question of policy making becomes far more complicated still when we realize that most policy actions entail costs, often quite substantial. No risk-factor study could possibly address this. Assessing the costs and benefits of a policy generally requires more analysis than an entire risk factor study.
Why not just ban smoking? If it worked, it would eliminate the health costs. One reason is that the costs (causing people to lose the benefits of smoking and enforcement costs) would be enormous. On a less dramatic level, even if Bloomberg’s plan would reduce obesity some, would that be enough to justify the various rather high costs? It does not appear that anyone bothered to ask that question.
4. Is it ethical to do (even if it would work)?
This is, of course, the question that generates the most animated conversations. I will not rehash the basic libertarian arguments here. Nor will I attempt to delve into more subtle points.
But I will mention an observation I made to the students: Some portion of the population would probably support giving their kid a vaccination that would prevent the child and the adult he will someday be from experiencing any benefits from nicotine. Some portion of the population would argue that it should be mandatory (or close to), like the pertussis vaccine. But probably roughly the same portion of the population would favor a hypothetical vaccine that would ensure that the kid is not gay or a similar magic bullet that would prevent him from ever embracing the teachings of the Koran.
The implication of that is that “public health” — the activist movement, as opposed to actual public health — is a special interest group filled with people who do not seem to realize that the interventions it demands are widely considered just as deplorable as anti-gay or anti-Islam interventions. I took the opportunity to point out that any student who was planning to go into “public health” (as opposed to working in some more acceptable way to improve people’s health) should realize that they are on the wrong side of history. While policy advocates in that area were once, legitimately, considered heroes, the generally celebratory reaction to Bloomberg’s plan being struck down by the court should give them pause.
[There are some concrete implications of this list for THR advocacy. I will come back to that in a later post.]