by Carl V Phillips
There is a cliché about certain types of institutional crimes, that it is not the act itself but the cover-up that causes the real trouble. The erroneous headline claim in the recent Public Health England (PHE) paper, that e-cigarette use is 95% less harmful than smoking, would generate nothing more than a fleeting rebuttal and represent nothing more than footnote to the history were it not for the equivalent of that cover-up. But some of the more serious commentators in the e-cigarette advocacy world, along with countless others, have rushed to defend the error. And I have found this truly scary.
For context on the subject matter, see my previous post, which you can follow back for more detail. The short version is that the PHE authors repeated what is just a made-up number (and not even a made-up number that is (a) plausible or (b) even supposedly a measure of comparative risk) that appeared in a junk science paper by Nutt et al., presenting it as if it were scientifically valid; this claim appeared in the headline under which PHE published the paper. For context on what I am responding to below, you can look at my twitter feed, at the comments others directed toward me. (I trust it is obvious that while Twitter is a great medium for announcements, quips, and simple observations, it does not offer a sensible way to try to have a conversation. Thus I mostly did not try to respond there, but rather wrote this post, which has a comments section that allows for better discussion. See the previous post for a good example of such conversation. Also, you presumably realize that this extrapolates to me not intending to “discuss” this on twitter in response to the next round of quips there.)
Why is it scary? For one thing, it is difficult to not fear where we end up a year from now, moving down this path. Pro-ecig advocacy’s relationship with science went from being way ahead of the legitimate science c.2008, to mostly highly serious and science-based c.2010, to the movement becoming so large that the science was part of the conversation but was mostly treated as a weapon c.2013. But even as that last bit occurred — as is almost inevitable when a mass activist movement forms — the serious opinion leaders remained anchored in a commitment to good science. But the recent events show that there are cracks in that anchor, suggesting we may be approaching the point where many other good and legitimately science-based causes took a permanent turn for the worse, becoming both less effective and less beneficial.
In addition, down the path of even the serious opinion leaders treating science as a mere weapon for a political cause lies disdain for any good science that does not benefit the politics. Sometimes a “bad” result is correct and represents useful information for consumers. But if the responses to every concern about a possible harm from e-cigarettes degenerate into knee-jerk rebuttals — and we are getting rather close to that point — then potentially beneficial knowledge will be ignored. When scientific claims are just weapons, the ability to detect whether a concern is legitimate tends to be lost to those within the cause. Consumers need accurate health information, not claims that are most convenient for advocacy, so that they can make informed choices that are best for themselves.
Even more information is lost to those outside the cause. Consider the various “healthy eating” movements, many of which contain bits of good advice for consumers. But the useful information is embedded in a corpus of junk science that exists to defend the cause against all critique. As a result, people either subscribe to a particular food cult, buying into a lot of misinformation, or dismiss everything they say, losing some good information. It becomes something like cheering for one sports team over another, as is common for impersonal political issues (e.g., macroeconomic policy, climate change). Similarly, there are legitimate concerns about childhood vaccines, particularly for some identifiable individuals, but anti-vaxxers and their junk science have so poisoned the well that there is little useful discussion or understanding of this point.
The following are responses to some of the claims I have seen over the last few days that the 95% claim was just fine. I have ignored the ones that were less cogent and I have tried to distill them down to their best essence.
“But we need to make such claims because….”
This is nonsense at multiple levels.
First, even if this were a valid response, it would only be a response to a statement that included “…and therefore we should….” But it is being presented as a rebuttal of the mere observation that the claim is junk. If someone writes “I concede that this claim is junk science, but we should endorse it anyway because…”, then perhaps they are being serious. But instead the implicit claim is often “we need to make such claims, and therefore the scientific criticism of the claim is wrong.” Um, no.
Second, what possible good can come from making up a clearly wrong number and pretending it is scientific when the truth is sufficient to make the point? It makes no sense. Is the target audience more likely to be persuaded by junk science that is easily debunked rather than a true statement that sounds pretty much the same to the audience? It is difficult to imagine why. Meanwhile, when the specific claim is debunked by those hostile to the overall message — and it has been already — it makes it much easier for those who are inclined to dismiss the entire legitimate message to do so.
Just in case I have to repeat this five times (and I have a feeling I do), the scary problem is not the initial mistake, but the desperate response to defend it. It is my assessment, based on my knowledge of advocacy movements from the inside and via study, that the initial mistake is itself actively harmful. But even if you doubt that conclusion, it should be obvious it was not helpful compared to just substituting something accurate. So why man the barricades to defend the mistake? It reminds me a lot of the tobacco controllers’ code of omerta, to never criticize any anti-tobacco claim that anyone makes, no matter how wrong it is.
Third, if trafficking in junk science really were required to make our case, then we would face a serious ethical problem. This is not just because of the moral case against lying, though that is fairly compelling argument in itself. But if you need to lie to make your case, you should probably stop and ask yourself whether what you are trying to argue is actually right. As noted, this is moot in the present case because the truth can be used to make the same point even better. But the mere willingness to say that a junk science claim is needed suggests a rather scary degradation of ethics.
Fourth, the unstated premise is that trafficking in junk science would be good for the cause, so long as it is the right junk science. Oh, really? I cannot think of a single example of a cause like this (insurgent, narrowly focused, consumer oriented) where that was a good strategy. I suspect that most of those implicitly making this argument have little experience with such causes, either personally or by way of study. I am open to suggestions, but I cannot think of anything. I can, however, think of movements that foundered there, and reasons why we should expect exactly that to happen. Plenty of good and defensible causes have turned into fringe cults that no one outside the cult takes seriously because they pursued the junk science track.
Keep in mind that trafficking in junk science is an effective strategy for those in particular positions, but this does not generalize. It works great if you have a powerful echo chamber and are just using it to keep the useful idiots in line. The upsides (when they exist, unlike in this particular case) can outweigh the downsides when you have pet media, are a government, or otherwise have a very loud megaphone, and thus can just shout-down the debunking. Junk science is often very effective at attacking science-based insurgent movements. I trust it is obvious that none of these describe the situation for pro-ecig advocacy, though it happens that all of them describe the tobacco control industry’s situation. Wars are often asymmetric, and so adopting the enemy’s tactics can be a losing strategy. In this case, given the lack of a loud megaphone and only a tiny non-influential echo chamber which does not really need to be stoked, it is undoubtedly a losing strategy to mimic tobacco control’s dishonest tactics, even apart from the ethics of it.
It is worth noting that the anti-smoking movement started out science-based and honest. Almost all the reduction in smoking from its peak was caused by honest education. The morphing into the lie-based tobacco control industry came at the same time that they stopped succeeding at reducing smoking beyond the steady downward trend that was caused by the honest approach from past decades (not to say this is causal; it just points out that they clearly cannot be said to have become more successful when they started lying).
The modern success of tobacco control in keeping people smoking, by opposing THR, is attributable to them seizing power and money, not to some magical effect of their junk science. That is just fodder for their echo chamber. It is difficult to argue that the trafficking in junk science has itself made any difference in terms of policy: The American, Canadian, and Australian governments shovel out tons of anti-THR junk science to their people, while the Europeans and Asians basically just impose autocratic rules from on-high. Yet all of these places have e-cigarette bans or severe restrictions, either in place or in the pipeline. It is the power, not trafficking the junk science, that makes that happen.
“This is not using junk science to further a political aim”
This is really a subordinate point, but it bugs me enough that it gets its own entry. In response to arguments that it is simply never ok to employ junk science to further one’s political goals, there have been responses along the lines of “this is not political: this is an effort to correct widespread misperceptions.” Um, yeah.
If someone wants to argue that it is acceptable to use junk science to further a political aim, they can try to defend that. But claiming that this is not political is a truly awful step on the descent toward becoming a cult. If the goal is changing the world, then you are talking about a political act; that is what the word means. If someone genuinely thinks “this is not political because it is doing something good”, that is a serious problem. News flash: every political activist thinks they are doing good, including the ones who want to create the misperceptions you object to. As soon as your defense of what you are doing descends to the point of saying “whatever I do is fine because I am doing it for a good cause” — and “this is not political” claim is a version of that — there is a severe erosion of the barriers that normally provide a check against very unfortunate actions.
“The estimate is close enough to the real expert estimate”
This is actually false, and not in a harmless way. As I and others have noted, the affirmative claim that vaping causes 5% of the health risk that smoking causes is a claim that vaping is quite bad for you. Parsing that with the accepted propaganda about smoking, it means that about 1 in 40 vapers will be killed by vaping. That makes it worse than any other common behavior other than smoking — far worse than driving, worse than all but the most extreme overeating. Also keep in mind that estimates are properly interpreted as being point estimates with some uncertainty on both sides (in spite of the PHE authors reciting their claim as if it were exactly precisely right), which means this claim says that there is roughly a 50% chance the risk is worse than that. Defending that is not clever.
But if it were true, why source it to a made-up number — and one that does not even say what the PHE authors claims it did, even apart from it being just made up (see previous post) — and declare that to be a source of scientific information? Why not instead present this real expert estimate that is being referenced? (Note that I am not entirely sure what real expert estimate such claims refer to. The evidence-based expert estimate I am aware of is an analysis along the lines of: “The chemistry of vapor suggests that vaping is almost certainly only slightly worse for you than smokeless tobacco (which is our only measure of the effects of using nicotine without smoke). ST use appears to be about 99% less harmful than smoking, though it may be completely harmless or even beneficial. Therefore the risk from vaping is most likely in that same range.” Yes, it is possible that vaping is so much more harmful than ST use that it is 5% as harmful as smoking — as I noted in the previous post, some versions of it undoubtedly are — but this seems vanishingly unlikely for “normal” vaping. It is worth mentioning that the junk paper by Nutt et al. made up a number for the risk from ST use that are enormously higher than the number they made up for vaping. There is no conceivable scientific basis for making such a comparative claim, further emphasizing the worthlessness and of their made-up numbers.)
But most important, and here is the crux of all of this again, why would it ever be a good idea to man the barricades to defend the junk claim based on this? If a claim that is sourced to junk science is close enough to what the real science shows (which is not true in this case, but imagine some other case where it was true), then why defend the use of junk science, making the credibility of the whole message depend on defending the original junk source? Instead, push for the substantive claim to be based on real science. This might seem to entail a bit of short-run cost, due to not being able to uncritically embrace some particular publication, but it is far better in the long run. Moreover, it is not as if the particular publication will stand up to scrutiny, so the apparent short run cost is probably moot. It is better to get ahead of the inevitable criticism of the junk science that will come from those who wish to dismiss the entire substantive claim.
“Just an estimate”
Some commentators have sought to excuse the reporting of a clearly inaccurate made-up number under the claim “the best estimate” by suggesting “hey, they said it was just an estimate.” But the word “estimate” does not mean “we are reporting something that is wrong.” It is a scientific term that refers to the best available scientific measure of a quantity (with an implication that we think it is a pretty solid measure), as in “the estimated speed of light in a vacuum”, or if clearly specified as such it can mean the specific measure that comes from a particular measurement effort, as in “the estimate of the increase in heart attack risk from nicotine consumption from this data and model is…..” The number reported by the PHE authors was neither of those.
Perhaps that observation gave you a sense of “where have I heard something like this before?” It is remarkably similar to the anti-scientific games played by those who deny the history of the planet and life by saying “evolution is just a theory.” Presumably if you are savvy enough to be reading this blog, you know the response is that a scientific theory is the best evidence-based understanding of a phenomenon that we have available, and moreover one that is well enough established that we are pretty confident about the core claim. But the anti-scientists think (or, more realistically, pretend to think, and try to trick others into actually thinking) that “theory” refers to a mere musing, as it is sometimes used in common language. In a scientific context, authors cannot just make up a story and call it a theory or just make up a number and call it an estimate.
“What they really meant was….”
If they really meant something different, why did they not say something different? If these same authors were previously on record as saying “at least 95% less harmful” and that is what they meant to communicate in this report, they sure picked a funny way of trying to do it. (Besides, that claim is also wrong, as explained in the previous post.) If they really meant to claim that their estimate was based on something more than the one junk science paper, why did they just cite it to that paper (along with mentions of some commentary papers that included no attempt to construct a scientific estimate for this figure) and say nothing about any other basis for the number? In any case, saying they really meant something different is not a rebuttal of the observation that what they said was wrong, but an endorsement of it.
If they really meant something different, you would think the authors would be anxious to post a correction. Some commentators have suggested this can be found in a subsequent note by the PHE authors in which they seem to be trying to imply — without stating this as their purpose — that they were not basing their claim entirely on one junk paper. But if you look at their text, they say just the opposite. The have a couple of sentences about why we can be sure the risk from vaping is much lower than from smoking, but this is not in dispute and is obviously quite different from asserting a particular quantity. They then assert (without further methodological elaboration) that they are not aware of any new research, appearing after the number was concocted, that shows the Nutt et al. number to be wrong and state that it “remains valid as the current best estimate based on the peer-reviewed literature.” This is not a denial that they are just channeling the original made-up, clearly inaccurate, junk science claim, but an explicit confirmation of it. If they had any legitimate scientific basis for their number, they undoubtedly would have mentioned it in their note. Also — and I really hope this goes without saying — if you just make up a number and then observe it has not definitively been demonstrated to be false, this does not make it true.
“Subjective”
One of the authors of the original junk paper apparently tried to defend their “methodology” of just making up numbers on the basis of that all science is subjective, and some others have picked up on this. This is another word game. It is true that if you divide everything into objective and subjective, then all empirical work and scientific conclusion-drawing is subjective. It is done by people and includes countless human judgments. Indeed, when someone tries to claim that a scientific process is objective, or that some particular scientific process is suspect because it is subjective, they clearly do not know what they are talking about. However, the subjectivity of science is not license to just make stuff up.
As with “estimate” and “theory”, which imply a solid scientific basis for a claim, for a subjective process to be scientific some minimum standards are required. It is true that we cannot write down a set of clear rules and call it “the scientific method”; there is no such objective concept, which creates a lot of consternation for people who cannot handle complication. But that does not mean that any method of arriving at a number is as good as any other.
“But that report was great, and it is helpful, and the authors were so brave, and….”
What does any of that have to do with the issue at hand? Yes, the existence of the report is a good thing for the cause, and the vast majority of it is valid. Both of those would still be true had they not headlined it with a junk science claim. Both would still be true if others did not man the omerta barricades over that claim. Neither changes if someone agrees that the particular claim is not valid.
The fact that such non sequitur statements are made in response to criticism of a particular scientific claim is extremely disturbing. The apparent reason is the notion that we have to defend every single word written by “our team”, just like the ANTZ do. That is not a good place to be.
I assume that some of those making these “but…and…and…and…” claims are genuinely worried that this flaw is a threat to all the benefits that the existence of the PHE report offers. But it is not e-cigarette proponents challenging “our team’s” junk science that poses that threat. Opponents will run with it. They are running with it. This is the type of mistake that serious opinion leaders can easily get out in front of, conceding the flaw with a shrug and pushing the focus elsewhere. That can be done based on a moral belief that telling the truth matters or as purely a matter of political tactics. If it is the latter and you find a situation where defending some bit of junk science really seems to be tactically useful and can actually be pulled off… well, I am probably still going to argue the moral position, but you might have a case to make. But this is not that situation.
Instead, this is a situation that needlessly give opponents an opening to claim that the core scientific claims of the movement are junk science. That is not true, obviously. But since it also sends the signal that junk science will be tolerated and actively defended if presented in an important context, it also dramatically increases the chances there will be plenty more of it generated. This will make it even easier for opponents to claim that e-cigarette advocacy is not science-based. It will also dramatically increase the chances that consumers will not be able to trust their own advocates to provide accurate decision-relevant information.
A political movement that ultimately relies on scientific information cannot afford this. If sciencey talk is just used to rally the faction, then accuracy does not much matter. If the crux of the cause were, like many causes, about emotion or “morals” or zero-sum fights, then the sciencey stuff would just be window dressing and the details would not matter. Indeed, if this fight were grounded in freedom of informed choice, as I have suggested it ought to be, it might not matter so much (though I would still be deeply concerned about what this would do to the “informed” bit). But so much of the focus of this fight is selling scientific claims, so there is no room to allow the impression that the scientific basis of e-cigarette THR is fast and loose. Moreover, consumers need to be able to make informed decisions about their own lifestyles, but embracing junk science to try to manipulate the uninformed masses — even if it were an effective tactic — makes it likely that even relatively informed consumers will become immersed in bad information. The well-worn path of other insurgent consumer movements that turned to junk science suggests that this will result in most consumers making their decisions based on whichever cult guru they end up falling in with, rather than on real evidence.
Tobacco harm reduction, anti-THR lies, and related topics 