by Carl V Phillips
Tomorrow a new paper about the supposed gateway effect from e-cigarettes will come out of “embargo”. Over the last few days, Clive Bates and Michael Siegel have published pre-rebuttals of it (Clive basically declared as much on Twitter. Mike did not, but the timing seems like more than coincidence.) Sometime I will analyze the paper based on the framework I developed for assessing whether evidence actually supports a gateway claim (which the authors of the paper ignored). For now it is interesting to go meta.
[Note: Please don’t mistake my delay as indicating any respect for the press release and embargo game, which is antithetical to proper science. That game, in turn, is a mere sequela of much more fundamental anti-science problems in health research. In particular that is the notion that it is ever proper science for journals to publish papers that have not been reviewed by the scientific community, and have been read by less than ten people ever. Anyone who complains about the press release and embargo game while accepting the latter just doesn’t understand what the real problems are. Another reason for the delay is that as a student of scientific and quasi-scientific debates themselves, I am curious to see what others are saying so I can analyze that also. That is what I am doing here.]
Clive’s post is entitled “How not to be duped by gateway effect claims”. I find that interesting because I am increasingly inclined to wonder if e-cigarette advocates have been very effectively duped into taking a tact that concedes the most important points, and sets up inevitable success for those attacking e-cigarettes with gateway claims.
For those who may not know, the claim out there is that some empirical evidence suggests that e-cigarette use is causing some people (the focus is on kids) who would not have otherwise smoked to become smokers. My methodology paper explains how to analyze whether that is really the case. All the supposed empirical support for the claim, it turns out, fails to meet some very basic minimal conditions for supporting the claim. Statements that there is evidence of a gateway effect are inevitably followed by pronouncements that, therefore, severe restrictions on e-cigarettes should be enacted. The missing steps in between the empirical claim and the policy demand are what e-cigarette advocates may be conceding when they respond to the empirical claims with some shadow of the analysis of methods that I presented in my paper. (Note that my paper did not address the missing steps either; it was explicitly about empirical methodology.)
Consider the first of the nine points that Clive argues in his post: “Is it clear what is meant by a gateway effect?” This appears to derive from my work, since I have been the one pushing that point and Clive cites me. He picks up on one of the three key points I make about meaning, that the jargon “gateway” clearly refers to the causation I described it in the previous paragraph. Some authors play a game of observing mere association of the behaviors, perhaps with the “right” order, with no analysis to try to sort out whether that is really causal, but then suggest the implications are those of having supported a causal claim. Clive offers a brief summary of the association-vs-causation point in his sixth point and the ordinality point in his third point. You can find greater depth on these points in my paper, but his summaries are just fine.
But one of the points missing from this version of “what is meant”, something I discuss in my paper, is quantification. Those making the gateway claim never (as far as I recall seeing) explain what magnitude of effect they are claiming. This is probably as much a result of the faulty thinking that permeates public health (using methods that are all about measuring quantities, but reporting results as if the dichotomy effect-vs-not is what matters) as it is an intentional tactic in this case. But it is a very effective way to trick readers into thinking that any evidence of any effect is tantamount to evidence of a large effect. Is there any gateway effect? Of course there is. Even though the gateway claim is scientifically extraordinary claim (as I discuss in my paper, and below), millions of nonsmokers have tried e-cigarettes and tens of millions more will. The chance that not even one, among those who would not have otherwise become a smoker, becomes a smoker as a result of e-cigarettes is basically nil.
Those arguing against gateway claims have inadvertently drawn their line of defense at the “this is not happening at all” point. This sets them up for defeat when any evidence is widely interpreted as suggesting there is any gateway effect. This might include those statistical claims that are critically analyzed on their merits as science (as they certainly should be — as I said, that is the point of my paper), but it might also be some ginned-up artificial clinical study or a testimonial from someone who claims the gateway happened to him (as I noted in my paper, there is no such testimonial out there, not even one, but there will be eventually). After all, there are going to be some gateway cases.
That brings up the second and even more important layer of the tactical error, ignoring the missing steps in between “there is a gateway effect” and “therefore we should do X.” Tobacco control has achieved enormous success through the tactic of getting others to buy into an explicit claim of “if any bad outcome happens at all, then any policy to stop it is justified.” See, e.g., smoking place bans. I do not have to tell readers of this blog that there are numerous arguments that can be made in that gap. But if they are not made as a matter of habit whenever that gap in instrumental in a discussion, the leap over the gap will win the day by default.
There is one particularly easy gap-challenging argument in this case, which avoids the need to appeal to pesky issues like ethics and policy analysis: If the number of gateway cases is less than the number of people in the same population who quit or avoid smoking as a result of e-cigarettes, then the net population effect is less smoking. This can be argued at the level of a particular subpopulation, such new cohorts that are coming of age, even apart from the cessation effects among established smokers. This challenges the tobacco control’s gap-based claim on its own terms, which measure outcomes merely in terms of caused change in smoking prevalence (though it is still probably a tactical error to concede that those terms are legitimate). This is probably why even the pro-ecig tobacco controllers sort of try to endorse this argument. However, they do a mindbogglingly bad job of it. They walk right up to it and then fail to make the point, claiming something that is patently wrong instead.
For example, Siegel’s thesis is that because smoking is declining in teenage population as e-cigarette use is increasing, there is no gateway effect. Nononono. This junk claim, which he and others have repeatedly made before, is simply false. There are several possible scenarios that are compatible with a gateway effect and the observed pattern. The most obvious is the aforementioned point that e-cigarettes are causing a net decrease in smoking even if they are causing some smoking. This is not, however, what is actually claimed. Rather, the common claim is that the net trend means there is no causation in the other direction, which is obviously false. Even Clive comes very close to saying this in his second point.
Of course, it is not the case that e-cigarettes are causing all of the decline in smoking among teenagers, or even all the dip below the trend, and there is not exactly overwhelming evidence they are causing any of it. The aggressive assertion that this association is causation, by those who have no support for it other than liking the conclusion, is a page straight out of the tobacco control playbook. It is interesting to consider one of the other stories for why there might be even a big gateway effect in spite of these statistics: The gateway claim could be that over the course of five years, a lot of people who became vapers at age 17 become smokers. (This is another element that is missing from the simplest “what is meant by gateway” claims: timing.) Indeed, it is not difficult to imagine the tobacco controllers arranging for the an upcoming official claim about a new increase in teen smoking rates (even just a return to trend from the extremely low — and thus probably downward biased — recent claims). Then they all cry, “see there is the gateway effect starting to appear.” They would not be obviously wrong. Live by unsupported claims of causation based on very crude population statistics, die by….
One additional neglected point within that conceded gap is the question of the evidence that a particular policy would actually have the intended result, and its other effects are ethically acceptable. Again, the problem is that “this phenomenon does not exist” arguments that are presented without the accompanying “even if [or though] there are gateway cases, that does not mean…” are a massive concession. So long as pro-ecig tobacco controllers dominate the responses, it seems safe to assume this will continue to happen. In most of what they do, they depend on no one noticing the missing arguments in the gap, and are not about to risk weakening their primary weapons for this side-battle.
A few other technical points are interesting. In his fifth point, Clive implies that if we do not know how many of the teenage vapers in the study used nicotine — many do not — then that is a problem. I cannot imagine why. If the empirical results actually did support the claim that vaping causes smoking, why would it matter? There are other mechanisms than acquiring a taste for nicotine that could be causing it. After all, isn’t the claim that e-cigarettes are so good a substitute for smoking because they mimic the feel and action of smoking? So why would we assume that if there were a gateway effect, it was not caused by those.
Moreover, consider what would happen if there really is a gateway effect, and it only occurs among those vaping nicotine. Imagine some hypothetical empirical result that genuinely supported the gateway claim, but it did not differentiate between vapers who used nicotine and not. Then the true gateway effect would be greater the estimated magnitude. The result would be attenuated by counting among in the exposed population some people who did not actually have the relevant exposure. Similarly in his fourth point, Clive suggests that poor measures of vaping and smoking status should cause us to doubt the claim. But if the measure of vaping was “ever tried one puff” and the study showed a gateway effect (again: hypothetically, genuinely), then the result would probably be attenuated for actual vapers (assuming that the effect only manifested after some quantity of vaping rather than one puff). On the other hand, if the measure of taking up smoking were similarly overly expansive, then the quantitative result would be biased upward: more people would be identified as gateway cases of actual smoking (rather than just trying a puff of a cigarette) than actually were. On the other hand, if a study showed an effect based on that expansive definition, they it would almost certainly mean there was some real gateway effect; it is difficult to imagine something that increased the trying of cigarettes that did not increase the probability of some of those trialers becoming smokers.
The point is here is that if you are going to try to analyze what the statistics actually mean, it is more complicated than just saying “they are not the best possible measures and therefore the result is not right.”
A final observation in this disjoint collection: In my analyses, I have emphasized the extraordinary nature of the gateway claim. I explain it more in my paper, but basically the issue is why would someone using a low-risk product, someone who would not have otherwise used the high-risk product, ever want to switch to the high-risk product? It is such an odd and unnatural behavior pattern that it would take extremely compelling evidence to legitimately overcome the doubt we should have about it occurring. (Which is to say, occurring much; as noted, given millions of chances, many very odd things will happen at least once.) However, I may have overstated this logic by not making clear it applies to people who have happily settled in to using the low-risk product. It does not necessarily describe someone who just tries the low-risk tobacco product and is enamoured with it but finds it unsatisfying and so shops around other options.
This scenario is interesting because it could mean that there is substantial gateway from the state “trying e-cigarettes and not adopting them” even when there is no measurable gateway from actually being a vaper. It would also mean that the gateway was caused by the fact that in the absence of e-cigarettes this individual would have remained ignorant of the fact that he really liked smoking — as opposed to being someone who would genuinely not want to smoke, and was made to want to smoke by using another product.
It is an interesting scientific question. Anyone who bristles at the presentation of the interesting question should probably just stay away from analyzing the science, since his interest is issue activism and not science. That is a valid role to adopt, of course, but if that is the chosen role, it might be wiser to focus on core arguments that live in that gap rather than fiddly scientific points. One more of those core arguments follows immediately from this interesting scientific question, by the way: In that story, stumbling upon the gateway and discovering the joy of smoking made this individual better off. Indeed, it is difficult to imagine any gateway case who is not made better off, since they still have the option of sticking with the low-risk product unless they like smoking a lot better. So a question to put in the unexamined missing steps is, “what business does anyone else have declaring that this individual should be denied the chance to make that choice?”