by Carl V Phillips
Fairly often (e.g., in the previous post) I make reference to the concept that extraordinary claims require extraordinary evidence. That is, if something seems extremely unlikely based on a great deal of accumulated knowledge or an understanding about how the world works, and you wish to claim it is true, you really need to have done some tight work. It is a good principle in science. Research does not produce scientific knowledge without adherence to principles like this (note that there are no “rules” in science, so we have to make do with evolved principles).
Today I am thinking of that in terms of a new study that was reported in this BBC story, “E-cigarettes ‘help more smokers quit'” (quotes from there).
No I am not talking about this:
Public Health England says e-cigarettes are 95% less harmful to your health than normal cigarettes, and when supported by a smoking cessation service, help most smokers to quit tobacco altogether.
That was a non-extraordinary claim that was apparently based on nothing. I only quote this as part of my campaign to prove to Clive Bates et al. that when fake numbers like this (which in this case grossly overestimates the reasonable assessment of the risk from vaping) are out there, they get reported as if they were valid point estimates.
Nor was I talking about this:
But health professionals say the most effective way to quit smoking remains through prescription medication and professional support from free local NHS stop-smoking services.
The actual claim that is being made — that at least two people who might be called health professionals say this — is obviously true. The fact that the BBC wrote a patently false statement in a way that was designed to suggest it was accepted fact is also not all that interesting or surprising.
No, I was talking about the crux claim:
The authors [Robert West and his peeps] say vaping may have helped about 18,000 extra people in England successfully give up smoking in 2015.
The team, from University College London and Cancer Research UK, say theirs is an observational study, and therefore cannot prove direct cause and effect.
I include the second part of that as pure rhetoric, to point out that these authors apparently think that there is some type of real-world research that can prove cause and effect. That is not an extraordinary claim; it is simply wrong. Same with the suggestion that “direct” has any meaning in this context, though that word may have been the reporter’s fault.
The extraordinary claim is that the vast majority of smokers who switched to vaping would have quit anyway in 2015 if e-cigarettes had not been an option. No, they do not directly say that, but it is right there: Several multiples of 18,000 smokers switched to vaping that year or used e-cigarettes as a transition to abstinence. If vaping only “helped” 18,000 of them, then the authors are saying that most who switched to vaping as an ongoing practice (let alone tried it as a transition measure) were not caused to quit by vaping. Put another way, most switchers would have quit smoking in 2015 anyway and so their use of e-cigarettes was just a waste of their time and money.
Given the portion of switchers who report that switching was the only thing that ever worked for them, and given the many previous occasions on which the average smoker considers that he tried to quit but failed, this is quite extraordinary. This does not mean the claim flatly contradicts what we know (unlike a couple of the above points); it is possible. But it seems exceedingly unlikely.
What is the evidence? Well I have not fully gotten my head around it yet (but have been tasked with doing so, so expect me to circle back on this), but basically it is a complicated and assumption-laden model. Such models do not tend to create extraordinary evidence, especially when they are presented as just-so stories rather than being thoroughly tested for weakness with every doubt the researchers could throw at them.
Tobacco harm reduction, anti-THR lies, and related topics 
Glad you are looking into this Carl. I have always been sceptical about this research, or more to the point, what relevance does the author see in it?
I don’t pretend to understand what is going on, but on the surface it seems as pointless as saying that 1000 people gave up using NRT but only 100 of them wouldn’t given up without it.
What is the point of knowing that? What is useful knowing that? I suppose there is an answer, but I am scratching my head as to what relevance it has.
That is a very telling question. Of course the answer could always be “we are just curious”, and often is for most science. In this case, though, it seems to be rather less pretty. At best this seems like an exercise in doing dubious calculations just because you can, just a game. Worst, though, the subtext is that the grandees are suggesting the plebes should be granted _permission_ to try ecigs because they “work” for some of them. It should not be ignored that this is coming from people with a active political preference — for promoting ecigs to quit smoking but still vilifying all tobacco product use so that ecigs are merely the lesser of evils — so one has to be suspicious of the use of a model (which I am guessing could produce most any result you wanted without looking any less defensible) to produce a number that is just barely positive supporting their first goal, but so low that it is hard to get very impressed by it, supporting their second.
I must admit that when I saw that word ‘extra’ in the phrase “…..helped about 18,000 EXTRA people in England successfully give up smoking in 2015.” I was rather surprised. I immediately thought that there must have been rather a small number of ‘quitters’ in 2015. It has been reported elsewhere that ecigs and patches etc are about equally divided as reasons given for stopping smoking. If there were more than about 36,000 quitters, then the rest must have used will-power alone.
As Paul said, what is the point being made?
That “extra” construction is a sloppy way of saying “ecigs *caused* X people to quit smoking.” It is phrased that way either because whoever wrote the sentence does not understand what cause means (reasonably likely) or realized that most readers do not (undoubtedly true). I trust my readers understand the concept: Those are the people who would still be smoking if ecigs did not exist.
It is not really surprising that there is interest in knowing that number, for the reasons I noted. My point was that it is difficult to believe it is such a small fraction of those who used ecigs to quit smoking.
“Public Health England says e-cigarettes are 95% less harmful to your health than normal cigarettes,”
I continue to have a hard time believing that Clive Bates didn’t understand this comically tortured bastardization of the actual claim was going to be the primary real-world consequence of the 95% thing, and that this benefits no one but tobacco control lunatics.
“and when supported by a smoking cessation service, help most smokers to quit tobacco altogether.”
Setting aside the lazy and/or ignorant conflation of “cigarettes” with “tobacco,” it is charming how public health grandees just can’t bring themselves to countenance smoking cessation without state intervention.
Totally agree with the last bit (unsurprisingly).
Clive and I have several points of disagreement about the use of that number. I don’t have much more to say about most of them. But this was about one particular empirical claim that is easily resolved. My contention, based on what pretty much always happens in the world, is that if there is a scalar out there — regardless of how fuzzy or even clearly out-and-out wrong it is, regardless of whether it is based on nothing or close to it, and especially regardless of whether someone once wrote a nuanced paragraph around it — it will be repeated as if it were a valid point estimate. In this case, my observation is that this is exactly what is happening, as in the present example. Clive consistently responds to my observation that this will happen, and even that it is happening, with a paragraph from the RCP report that adds a bunch of nuances to the scalar. My point is that the existence of that paragraph does not mean that most observers are not simply repeating 95% as a point estimate (or even precise fact).
PHE stated that number without nuance, clearly declaring it a point estimate (based on nothing, but that is a different part of the story), and headlined it. At that point, a lot of the damage was already done, though it could have been substantially reduced had vaping advocates pushed back against it instead of embracing it. The RCP could have substantially undone the damage by giving a better scalar, like 99%, but instead they chose to cozy up to their fellow grandees (no surprise) and repeat their number. It matters almost not at all that they added words; they repeated the number. If they had said 99%, the press et al would probably be saying “95-99% less” which is an error of a different sort, but at least it would not be declaring the high risk to be the fact.
It is worth noting that the numbers that Glantz et co are trying to claim are only 4-6 times higher than what PHE claimed. That is a big difference, of course, but not an absurdly big difference. Errors of that magnitude happen. Thus the 95% number lends plausibility to those higher numbers.
@ Carl V Phillips. Recall a study prior to the RCP report that also stated 95% safer. Back story was that they actually wanted to say 99% safer but there was a 3% margin of error. So rather than a 96 – 102% range that would look silly and cause controversy they rounded it down to 95%.
There was never *any* study that supported the 95% number — and certainly there was not one that would be derived in a way that would produce a confidence interval.
Anyway, someone who knew what they were doing would know how to calculate the proper confidence interval around a point estimate that was close to the extreme. It is simple statistics. Not simple enough for most people doing public health research, who don’t know how to do it, but still simple.
Pingback: Godshall Chronicles 09/23/16 » eJuiceMonkeys News and Information
Pingback: Sunday Science Lesson: Debunking the claim that only 16,000 smokers switched to vaping (England, 2014) | Anti-THR Lies and related topics