Detecting bullshit in claims about comparative risks across tobacco products

I interrupt  the posts about the Tobacco Control editorial (I expect two more) to respond to a query from a reporter (who, I should note, writes good stuff about THR), who I criticized on Twitter for repeating the nonsense claim that smokeless tobacco is 10% as harmful as smoking in a recent article. He, quite reasonably, asked if I could hook him up with better information. I said that is not so easy (not Twitter-level easy, for sure), and that I would instead write a post for him.

It turns out that the best easy information I can offer is summarized in this passage from my IEA White Paper, Understanding the basic economics of tobacco harm reduction:

The only low-risk [tobacco] product for which we have useful epidemiology, smokeless tobacco, causes only about 1/100th the disease risk from smoking, based on the only existing attempt to calculate an evidence-based estimate of overall comparative risks (Phillips et al. 2006), and calculations for specific diseases support that estimate (e.g., Lee and Hamling 2009). Estimates for other products must be based on what we know about smokeless tobacco, but since there are no reasons to believe they differ much, this is adequate. There is no affirmative evidence that the risks are different.(4) This estimate of harms ignores the apparent health benefits of nicotine consumption (e.g., protection against neurodegenerative diseases), so it is plausible that the net effects are actually positive. Thus it cannot be claimed with confidence that use of [low-risk tobacco] products is less healthy than abstinence.

Footnote 4 reads:

4. A popular claim at the time of this writing is that e-cigarettes are merely 95% less harmful as smoking – i.e., pose five times the gross disease risk of smokeless tobacco – but this is not supported by any evidence.

Later I tweeted this in response to my original comment about the article:

Hypothesis:

False claim: ecigs 5% as harmful as cigs
+
False claim: ST more harmful than ecigs
=
ST 10% as harmful

I really think that is the story. Pro-ecig tobacco controllers have made up the claim that e-cigarettes are 5% as harmful as smoking. I have called bullshit on this claim, as well as the various attempts to finesse it and defend it quite a few times on this page (example). Some e-cigarette advocates are convinced smokeless tobacco must be more harmful than e-cigarettes because… um? well… because those always-trustworthy tobacco controllers told them it was harmful. And they think this is plausible because… um? well… because it contains identifiable bits of the original plant rather than an extract, so it must be worse. Just like chewing coca leaf is worse than smoking crack and apple juice is more healthy than whole apples. (Note: just in case it is not common knowledge, neither of those is true.) As I have noted on this page, if forced to bet on which of the two fairly benign consumption choices is a little bit worse for you, the smart money is clearly on e-cigarettes being less healthy than smokeless tobacco, though we will probably never know with much confidence.

Oh, and the choice of 10% as the result of this “logic” of falsehoods is simply digit preference. No one would say 6%.

Actually there is a really good reason no one would say 6%: It does imply more precision than 10%, and it might cause the reader to start asking questions. Like, “how did you calculate that?” or “what diseases are caused that add up to 6%?” No use calling attention when the answer to those would be “I did not calculate anything” and “hell if I know — I just made that number up.”

I do not know who made up the number that was then repeated in the article (the reporter presumably did not pull it from thin air himself). Maybe it was stated or could be inferred between the lines in the article. Presumably there is some commentary in a journal out there that asserts it. But I am not going to check because I would just as soon not know. There is little to be gained by focusing my disgust on one particular author.

Oh, and if you did not hear the sneer when you read my use of the word “commentary” there, circle back and add it. Because in case it is not obvious, this is a scientific question that should be answered using scientific methods. Opinion pieces do not inform about scientific estimates. (In public health, they also seldom produce informative opinions, but that is another story.)

A claim about risk needs to be risk of something, some particular diseases. So what disease risks from smokeless tobacco add up to 10% of the risk from smoking? It is obvious the claim is bullshit if you understand the research on the possible health risks, and thus know that the worst-plausible-case scenarios, summed up across all diseases, could not add up to close to 10%. Not even 5%. But even if you do not know that, it should be clear that the risk must be the sum of risks for particular diseases. There is no such thing as “just risk”. So if whoever is making the claim does not even specify what diseases are being caused, it is obviously bullshit. Of course the next step must be to justify the quantitative claims — using epidemiology if at all possible.

Another scientific approach is to divide the exposure into understandable component exposures. This is the approach necessary for assessing e-cigarettes, since there is no useful epidemiology (and probably will never be for the current techs). So we have nicotine (close to harmless or maybe beneficial), the carrier chemicals (seemingly harmless at doses that have been studied, though vaping doses are in largely unstudied territory higher than that), and the various flavoring chemicals and pyrolysis products (in doses that are trivial compared to what is considered sufficient to create a health concern). This is the scientific approach taken by the serious effort to make the assessment by Burstyn. The claims of 5% are not serious or scientific.

Here is another clue about bullshit: The comparison to smoking can only be an additional step, after already figuring out the absolute risk. Anyone who has skipped the step of estimating the absolute risk, using some method other than just making up numbers, is trafficking in bullshit. Consider what it would take to figure out the risk from e-cigarettes as a percentage of the risk from mountain climbing. You can look up estimates of the latter’s absolute risk, but then you need to somehow estimate the former, and calculate a comparative statistic. There is no magical way to just figure out the percentage. It is no different if the comparator is smoking. It is obvious nonsense to say “hmm, e-cigarettes seem to me to be 5% like smoking and 95% unlike smoking, so they must be 5% as harmful.” It is no more valid than saying “hmm, mountain climbing is really dangerous and using smokeless tobacco seems a lot less so, so let’s just say vaping is 10% as harmful as mountain climbing”. I trust it is clear that this would be anti-scientific nonsense. It is no less nonsense to do that for smoking.

Vaping and smokeless use have a bit more in common with smoking than they do with mountain climbing, but not a whole lot more. It is not unreasonable to say that reducing your vaping or smoking by half probably reduces your risk (whatever that is) by about half. In that case, we are talking about two very similar exposures (using the same product) and a meaningful quantitative difference in exposure. That estimate actually turns out to be too optimistic for smoking and to be ambiguous (half of what?) for vaping, but it is still a reasonable rough estimate. But you cannot do the same across completely different exposures. I am not sure if I can communicate just how baffling it is, to anyone who actually understands epidemiology, that anyone would think otherwise.

To sum up, to estimate the comparative risk requires first estimating the absolute risk. That estimate requires estimating the risks for particular identified diseases. This is true whether the goal is to create a point estimate or bounds. That is, it is just as necessary to do this to justify a claim that “vaping is more than 95% less harmful than smoking” as it is to justify the claim “vaping is 95% less harmful than smoking”. You would deal differently with uncertainty when doing one rather than the other, but the necessary basic calculations are the same.

Readers of this blog are justifiably incensed with “public health” people just making up pseudo-scientific claims to further their anti-THR political agenda. But these comparative risk claims you see from self-styled THR advocates (who really do not support THR, but that is another story) are every bit as much pseudo-scientific bullshit. It is really hard for those of us who expect scientific claims to be based on scientific analysis to see much difference, in terms of their honesty and relationship with real science, between the two groups.