Detecting bullshit in claims about comparative risks across tobacco products

I interrupt  the posts about the Tobacco Control editorial (I expect two more) to respond to a query from a reporter (who, I should note, writes good stuff about THR), who I criticized on Twitter for repeating the nonsense claim that smokeless tobacco is 10% as harmful as smoking in a recent article. He, quite reasonably, asked if I could hook him up with better information. I said that is not so easy (not Twitter-level easy, for sure), and that I would instead write a post for him.

It turns out that the best easy information I can offer is summarized in this passage from my IEA White Paper, Understanding the basic economics of tobacco harm reduction:

The only low-risk [tobacco] product for which we have useful epidemiology, smokeless tobacco, causes only about 1/100th the disease risk from smoking, based on the only existing attempt to calculate an evidence-based estimate of overall comparative risks (Phillips et al. 2006), and calculations for specific diseases support that estimate (e.g., Lee and Hamling 2009). Estimates for other products must be based on what we know about smokeless tobacco, but since there are no reasons to believe they differ much, this is adequate. There is no affirmative evidence that the risks are different.(4) This estimate of harms ignores the apparent health benefits of nicotine consumption (e.g., protection against neurodegenerative diseases), so it is plausible that the net effects are actually positive. Thus it cannot be claimed with confidence that use of [low-risk tobacco] products is less healthy than abstinence.

Footnote 4 reads:

4. A popular claim at the time of this writing is that e-cigarettes are merely 95% less harmful as smoking – i.e., pose five times the gross disease risk of smokeless tobacco – but this is not supported by any evidence.

Later I tweeted this in response to my original comment about the article:


False claim: ecigs 5% as harmful as cigs
False claim: ST more harmful than ecigs
ST 10% as harmful

I really think that is the story. Pro-ecig tobacco controllers have made up the claim that e-cigarettes are 5% as harmful as smoking. I have called bullshit on this claim, as well as the various attempts to finesse it and defend it quite a few times on this page (example). Some e-cigarette advocates are convinced smokeless tobacco must be more harmful than e-cigarettes because… um? well… because those always-trustworthy tobacco controllers told them it was harmful. And they think this is plausible because… um? well… because it contains identifiable bits of the original plant rather than an extract, so it must be worse. Just like chewing coca leaf is worse than smoking crack and apple juice is more healthy than whole apples. (Note: just in case it is not common knowledge, neither of those is true.) As I have noted on this page, if forced to bet on which of the two fairly benign consumption choices is a little bit worse for you, the smart money is clearly on e-cigarettes being less healthy than smokeless tobacco, though we will probably never know with much confidence.

Oh, and the choice of 10% as the result of this “logic” of falsehoods is simply digit preference. No one would say 6%.

Actually there is a really good reason no one would say 6%: It does imply more precision than 10%, and it might cause the reader to start asking questions. Like, “how did you calculate that?” or “what diseases are caused that add up to 6%?” No use calling attention when the answer to those would be “I did not calculate anything” and “hell if I know — I just made that number up.”

I do not know who made up the number that was then repeated in the article (the reporter presumably did not pull it from thin air himself). Maybe it was stated or could be inferred between the lines in the article. Presumably there is some commentary in a journal out there that asserts it. But I am not going to check because I would just as soon not know. There is little to be gained by focusing my disgust on one particular author.

Oh, and if you did not hear the sneer when you read my use of the word “commentary” there, circle back and add it. Because in case it is not obvious, this is a scientific question that should be answered using scientific methods. Opinion pieces do not inform about scientific estimates. (In public health, they also seldom produce informative opinions, but that is another story.)

A claim about risk needs to be risk of something, some particular diseases. So what disease risks from smokeless tobacco add up to 10% of the risk from smoking? It is obvious the claim is bullshit if you understand the research on the possible health risks, and thus know that the worst-plausible-case scenarios, summed up across all diseases, could not add up to close to 10%. Not even 5%. But even if you do not know that, it should be clear that the risk must be the sum of risks for particular diseases. There is no such thing as “just risk”. So if whoever is making the claim does not even specify what diseases are being caused, it is obviously bullshit. Of course the next step must be to justify the quantitative claims — using epidemiology if at all possible.

Another scientific approach is to divide the exposure into understandable component exposures. This is the approach necessary for assessing e-cigarettes, since there is no useful epidemiology (and probably will never be for the current techs). So we have nicotine (close to harmless or maybe beneficial), the carrier chemicals (seemingly harmless at doses that have been studied, though vaping doses are in largely unstudied territory higher than that), and the various flavoring chemicals and pyrolysis products (in doses that are trivial compared to what is considered sufficient to create a health concern). This is the scientific approach taken by the serious effort to make the assessment by Burstyn. The claims of 5% are not serious or scientific.

Here is another clue about bullshit: The comparison to smoking can only be an additional step, after already figuring out the absolute risk. Anyone who has skipped the step of estimating the absolute risk, using some method other than just making up numbers, is trafficking in bullshit. Consider what it would take to figure out the risk from e-cigarettes as a percentage of the risk from mountain climbing. You can look up estimates of the latter’s absolute risk, but then you need to somehow estimate the former, and calculate a comparative statistic. There is no magical way to just figure out the percentage. It is no different if the comparator is smoking. It is obvious nonsense to say “hmm, e-cigarettes seem to me to be 5% like smoking and 95% unlike smoking, so they must be 5% as harmful.” It is no more valid than saying “hmm, mountain climbing is really dangerous and using smokeless tobacco seems a lot less so, so let’s just say vaping is 10% as harmful as mountain climbing”. I trust it is clear that this would be anti-scientific nonsense. It is no less nonsense to do that for smoking.

Vaping and smokeless use have a bit more in common with smoking than they do with mountain climbing, but not a whole lot more. It is not unreasonable to say that reducing your vaping or smoking by half probably reduces your risk (whatever that is) by about half. In that case, we are talking about two very similar exposures (using the same product) and a meaningful quantitative difference in exposure. That estimate actually turns out to be too optimistic for smoking and to be ambiguous (half of what?) for vaping, but it is still a reasonable rough estimate. But you cannot do the same across completely different exposures. I am not sure if I can communicate just how baffling it is, to anyone who actually understands epidemiology, that anyone would think otherwise.

To sum up, to estimate the comparative risk requires first estimating the absolute risk. That estimate requires estimating the risks for particular identified diseases. This is true whether the goal is to create a point estimate or bounds. That is, it is just as necessary to do this to justify a claim that “vaping is more than 95% less harmful than smoking” as it is to justify the claim “vaping is 95% less harmful than smoking”. You would deal differently with uncertainty when doing one rather than the other, but the necessary basic calculations are the same.

Readers of this blog are justifiably incensed with “public health” people just making up pseudo-scientific claims to further their anti-THR political agenda. But these comparative risk claims you see from self-styled THR advocates (who really do not support THR, but that is another story) are every bit as much pseudo-scientific bullshit. It is really hard for those of us who expect scientific claims to be based on scientific analysis to see much difference, in terms of their honesty and relationship with real science, between the two groups.


10 responses to “Detecting bullshit in claims about comparative risks across tobacco products

  1. I for one have never been happy in backing the 95% safer claim,as there doesn’t appear to be any basis in science!

    • Carl V Phillips

      That is good to hear.
      There is a decent scientific basis for a claim, combining the known trivial risk of nicotine with what we know about the other exposures. It puts the estimate way below 5% of the risk from smoking, as I have noted before.

    • Agree
      The important comparison is not between ‘tobacco products’, rather its between Smoke, and, Smokeless products.

      • Carl V Phillips

        Yes. And extending that, if there were a new fad of, say, huffing the smoke from burning grass, the best evidence for estimating the health effects would be the effect of smoking (adjusting for dosage). Burning plant matter is all about the same — there is approximately nothing unique about it being tobacco.

  2. OK, let’s say short-term healts effects form vaping are promising, but I will just say this right here:

    “For vaping and the lungs, the available evidence is all short-term, so we can only speculate about the long-term risks, and there could easily be something that looks harmless in short-term studies that turns out to be very significant over time.”

    This is copied from one of the vaping blogs, so even vapers aren’t sure if vaping is any safer than smoking… We do know long-term health effects of smoking but we don’t know long-term effects of vaping and they may be worse than long-term effects of smoking. Smoking is safer bet because we know what to expect in the future (after 30, 40, 50 years of smoking), from vaping we don’t know (what will happen after 20, 30, 40, 50 years of vaping?)… Case closed.

    • Carl V Phillips

      Not sure if this is a troll or not (the two last silly words make me think so), but I have a few free minutes, so I will do a quick logic lesson for anyone reading. Nothing in the real world is “known”, particularly in the future — everything exists as a probability distribution. That is not just philosophy when it comes to health effects: The effect of exposures changes over time and space (e.g., with technology). The effect of smoking that we “know” will be different in the 21st century than it was in the 20th.

      If a magnitude (in this case, health effects) is more uncertain, that obviously does not mean it is larger than every more certain quantity. I trust that does not actually need to be explained to non-trolls.

      Less obviously, the uncertainty about the health effect of ecigs is, in absolute terms, actually less than that for cigarettes. If we set the best point estimate of the health effect of smoking to 100, then it is very possible it is 95 or 105, and quite possible it is 90 or 110, or even 80 or 125. (That is ignoring the TC rhetoric which claims it is at 150 even though 100 is the defensible estimate.) Meanwhile the risk from ecigs has a best point estimate of 1, and might be 0 or -1 or 3. But it is difficult to believe it is 5 and nearly impossible to believe it is 10.

      • No, a post is not a troll. We really don’t know what a long-term health effects of vaping are. Isn’t that true? We don’t even know what inhalation of VG/PG will cause over the long period of time. Let alone flavours. Lung cancer, COPD and other untreatable illnesses and diseases are not excluded. How can somebody say that vaping is close to harmless? You are sucking hot vapour from at least two chemicals which are heated with metal coils and wicked through cotton into your lungs for God sake. Lungs can’t handle this because are not made for this. Isn’t that true?

        • Carl V Phillips

          No, the reality is that we know quite a lot. We know that the carrier chemicals are harmless (which is to say, there is no disease causation at a detectable level) at doses not too much below that from vaping. We know that the quantities of other chemicals are down in a range that appears harmless (in the same sense). There is nothing fundamentally novel about this exposure — that is the whole point of the Burstyn analysis. It is unfathomable that there could be some lurking, previously undetected harm that is of such huge magnitude that the harms would be a substantial fraction of that from smoking.

          That said, your point is valid that lung involvement and novel exposures that have not been fully studied create unknowns. Very few unknowns of this type are resolved as showing benefits, of course. That is why I always point out that it is a safe bet that e-cigarettes are more harmful than smokeless tobacco, where we have evidence that there are no detectable health effects other than those of nicotine (which might be harmless or even beneficial).

  3. Carl, Sceptic87 is an anonymous antismoking sock-puppet who created his/her/its identity here and on Disqus just a two or three days ago and ran around to high profile blogs (Puddlecote, Siegel, yours, I think Frank’s? Maybe one or two others I’ve seen?) with basically a cut and paste posting.

    Never forget: The Antivapers are simply Antismokers dressed up in new outfits. They’re often quite well paid by organizations that never have to expose their records to any sort of real scrutiny, and they use the same tricks and techniques of building fear and doubt and regulations and rules, step-by-step in the same way the Antismokers have been doing it for the last 40 years with their secondhand smoke circus. As soon as they can get you or others to agree that there’s any reasonable basis for significant concern about the long-term effects of vaping then the doors will be flung wide open for vaping bans based on concern about “secondhand vaping.”

    Sceptic and its ilk-mates want to take a microscopic possibility and blow it up into something that appears to be worth serious concern and consideration to such an extent that it should influence legal and social policy.

    MAYBE if “Sceptic” was willing to reveal who they are and what their background in this area is, then they MIGHT be worth listening to and engaging with.

    Otherwise he/she/it is just a troll. Check out Sceptic’s DISQUS profile and you’ll see what I mean (sorry… no links here?? I can email it to you…)

    – MJM, Anonymous-troll-chaser

    • Carl V Phillips

      Well that is pretty pathetic. All it takes to get an honest assessment from those of us who do real science is to read what we write. I guess that is too much work for tobacco controllers; it is pretty clear they don’t read. Or maybe they just don’t realize that most people tell the truth because they are so used to doing otherwise.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s