by Carl V Phillips
I have repeatedly tried to explain what gateway claims mean — that engaging in one behavior causes another behavior (in the present context, that vaping causes smoking), what they do not mean, and what constitutes valid evidence for or against their existence. Why bother? As I noted in a recent piece on the topic, for The Daily Vaper (which links back to some of my more in-depth pieces), “most of the claims by vaping proponents that there is no gateway effect are also nonsense.” Indeed, they create such a target-rich environment for criticism that even Simon Chapman can find the flaws.
He (or someone) recently posted an analysis he and David Bareham wrote about the recent J-F Etter paper in Addiction. On twitter, @jkelovuori (Jukka Kelovuori) called my attention to it and asked for my thoughts. I pointed out that it was generally valid. If a second-year student turned this in for an assignment, it would be B- material — even B if they took out the science fiction bits (see below). This made it a somewhat refreshing read compared to the average tobacco controller paper, which is a full-on F, or for that matter, compared to a lot of pro-vaping writing.
Chapman and Bareham made three main points. Two of them call out genuinely erroneous claims in the original paper. The third is a valid retort to some sloppy claims, but not to the real science that Etter presented.
To appreciate their critique, it is necessary to read it as science fiction. Proper science fiction makes up a few alternate facts about the world and then — in contrast with mere fantasy — stays realistic within those bounds. A story that posits faster-than-light travel can still offer insights about the human condition or the uncertainties of meeting aliens. It can even offer insights about the implications of FTL travel. It cannot, of course, demonstrate that FTL travel is possible — that is merely a premise.
In the present case, the authors spill most of their ink banging on about tobacco control’s usual fictitious premises: various anti-smoking measures are hugely effective, and also nicotine is magically captivating. (A good sci-fi author does not usually choose fictitious rules that are quite so close to being mutually-contradictory. I did not say these guys were writing good sci-fi.) If you read the critique and just gloss over these bits, it is not a terrible analysis.
(Of course, since I have written complete and thorough analyses of all these points, it is not entirely clear why anyone should bother with something that is merely not terrible. But let’s go ahead and break it down.)
Starting with the third of their points:
In criticising studies which do not differentiate adolescent occasional, experimental vaping from more regular vaping, Etter argues that it is “hardly plausible that a simple puff or a few puffs on an e-cigarette can cause subsequent regular smoking.” But of course every regular smoker started with a “simple puff”, mostly in adolescence. …. Just as no young smoker commences their smoking career by smoking a pack on their first day, few if any adolescent vapers commence vaping with heavy, daily use.
It is indeed a common trope to criticize a gateway study for looking at e-cigarette trialers or ignoring intensity of vaping. This is nonsense. Chapman and Bareham do not quite get the critique right or finish it, but their intuition is right. (Do not bother to read beyond that quote, expecting them to complete the thought. They immediately spiral off into a full page of mysticism about nicotine. Good sci-fi writers do not engage in asides about their conceits; they feed the necessary information to the reader organically.)
Not restricting the exposure measure to established vaping is fine, for a couple of reasons: (a) Someone might be claiming that any vaping, or that trialing in particular, causes smoking, and thus it is better to not restrict the exposure definition to dedicated vapers. (b) If the hypothesis is that dedicated vaping causes smoking, but not mere trialing, it can still be informative to look at trialers or a mix of usage intensity, if that is the data you have. Trialing vaping would cause smoking less often than dedicated vaping (which is the real hypothesized cause), because it is a weak proxy for dedicated vaping, but it would still cause smoking. Indeed any measured effect would underestimate the effect of dedicated vaping (this is called attenuation bias).
Of course this assumes that a study does not have other fatal flaws that make its results meaningless; every study to date has such flaws.
Chapman and Bareham, as nonscientists and non-readers (I have presented that analysis about attenuation from proxy measures of vaping exposure in greater detail), do not seem to fully understand this. But they still realized that the trope is wrong and have some intuition about why, so I will give them substantial partial credit.
In their first point, Chapman and Bareham observe:
Etter repeats a frequently made argument that the gateway hypothesis is incompatible with evidence from the USA and UK of declining adolescent smoking. The argument here runs that vaping has been rising while smoking continues to fall, so vaping cannot be causing smoking to any significant degree at the adolescent population level.
This argument relies on an assumption that the population-wide net impact of any putative gateway effect of e-cigarette use would be larger than the combined net impact of all other policies, programs and factors which are responsible for reducing adolescent smoking prevalence…. This is a ridiculously high bar that gateway critics demand that anyone suggesting gateway effects must jump over.
A spot-on takedown of what is probably the most common — and most obviously wrong — “there is no gateway” trope. We can just focus on “factors” and ignore the sci-fi claims about policies and programs. Anyone who recognizes that vaping replaces some smoking, and that we are still experiencing the echoes of the huge decline in smoking that resulted from universal knowledge of the harms, should see that this is true.
Of course, Chapman and Bareham do not mention those most obvious and defensible explanations for the net decline. Instead they go on to cite the junk science about implausibly large effects from tobacco control policies and programs. Thus their core argument is solid, but attempt to further support it fails.
This is where the sci-fi metaphor gets interesting. If someone does accept their conceit — that tobacco control policies and programs are having a big effect — they must concede that “but prevalence is declining so…” is nonsense. Yet some of the loudest proponents of that nonsense are pro-vaping tobacco controllers who claim to believe tobacco control policies are effective. This makes their error even more embarrassing.
Chapman and Bareham’s second point is a bit more complicated to analyze correctly. Under the heading, “Double standards on gateway and ‘reverse’ gateway effects?”, they argue:
E-cigarette enthusiasts often argue that vaping is demonstrably a reverse gateway out of smoking for those who quit, while being scathing about suggestions that it could ever be a gateway into smoking.
Soundbites like “kids who will try stuff, will try stuff” and “kids who will smoke, will smoke” have been repeatedly held aloft like an omnipotent crucifix before a gateway vampire. These responses are voiced as self-evident truisms, with their circularity being seductive at first blush. However, any cessation researcher offering the equally trite “smokers who will quit, will quit” as a serious contribution to understanding the complexity of transitioning out of smoking, would be rightly pilloried….
Their heading is silly, even if you set aside the bad phrasing about “reverse gateway”. (Another minor criticism of some anti-gateway claims: The jargon “gateway” is a lousy metaphor, but it has an established meaning. Playing games with terminology — saying it is a “gateway out” and whatnot — has no place in scientific debate.) Calling it a “double standard” that people believe some assertions and doubt others is some epic weak thinking.
Also, apparently Chapman cannot resist going a few pages without engaging in his preferred style of debate, which is the semi-adult version of saying “nyah nyah, you said….” In this case, he intentionally misconstrues a statement that obviously really means “kids who will try one thing are likely to try another” as being circular because of how it is often phrased.
But there is something that approaches validity at the core here. We can salvage their mess by rewriting the claim using some actual science: “often, gateway opponents claim ‘kids who try some stuff will inevitably try other stuff’ but they just assert this is self-evidently true, and imply this assertion alone sufficient to deny there is a gateway effect.”
This challenge to the incompleteness of the argument has nothing at all to do with whether or not someone dismisses the “all vapers would have quit anyway” claim (though any sensible person does dismiss that, obviously). That is pure red herring.
This turns out to only be a criticism of the vague hand-waving way in which the claim is sometimes made. It is not a criticism of the underlying claim when it is properly supported. Consider the point rewritten in scientific terms: “There is good reason to believe that strong confounding will create an association between vaping and smoking, so a lot of work has to be done before claiming an association is causal.” That is the main theme of my major paper on this topic (see first link).
This is not presented as a “self-evident truism”, like when people invoke meaningless phrases like “the common liability model”. We know — from evidence — that kids who try one drug, delivery system, or other forbidden/risky behavior are more likely than average to try another. This is pretty much true for whichever two of these you pick, even those that seem unlikely to be causally related (e.g., motorcycling and smoking). This is evidence of confounding.
For vaping and smoking, not only are the two behaviors both forbidden sensation-seeking, but they are use of the same drug. Obviously there will be even more confounding there. A third to half of the population dislikes consuming nicotine. They will avoid trying more than a puff or two of either product. This alone creates confounding — anyone who vapes beyond a puff or two is identified as (not caused to be!) someone who would not hate smoking.
For those who understand how epidemiology statistics work, there is further evidence still. The studies of these associations always include “control” covariates in the analysis. This means that even the researchers who are trying to show a gateway effect are admitting that there is confounding. They then — in an error that is common to most public health research — implicitly claim that they statistically eliminated its effects (“controlled for it”). But they did not.
Consider the confounding that is created by some people not liking nicotine. A perfect dataset would have a measure for that, in which case it could be a control variable (or, better still, a stratification variable). Instead, they probably just controlled for age, sex, SES, and such, which offer basically no measure of this. The best-case scenario is that they have a measure of parental smoking, which seems to be a predictor of taste for nicotine. But obviously this is a very noisy proxy for “taste for nicotine” so controlling for it might control for some of that particular source of confounding, but some will remain (“residual confounding”). Similar observations apply to attempting to control for confounding caused by willingness to break the rules, being more socially active, and so on.
The bottom line is that this confounding need not be self-evident because it is overwhelmingly evident in the data and specific existing knowledge. Thus Chapman and Bareham are ultimately presenting a strawman argument (another Chapman favorite). Indeed, Etter is not among those who suggest it is self-evident. He identifies this as a confounding problem and explains why it is a problem — not perfectly, but he basically touches on most of the key points.
This strawman keeps Chapman and Bareham from being a candidate for an B+ grade. The confounding problem, alone, is why existing research provides no evidence of a gateway effect. Etter’s paper has flaws — the two that Chapman and Bareham identified and others — but it also contains the key valid argument. If Chapman and Bareham were writing a generic criticism of anti-gateway broadsides, they would be right in noting that the confounding argument is usually unconvincing as presented. But it is not a fair criticism of Etter’s paper.
But overall, they got most of the criticism almost right, which is not too bad for amateurs.
Tobacco harm reduction, anti-THR lies, and related topics 
Carl something that seems to never get a mention in this context is what could be called the ‘Coming of age in Samoa ‘ factor.
Adolescents love taking the micky just for the sake of it etc – they often do not give ‘strait answers’ to questions, coming from adults .
Can you control for survey responses that are intended to ‘confuse’ etc?
Yes, that is a valid concern. And you are right, measurement error (the broader heading this falls into) does not get talked about nearly enough. It is a big problem for a number of reasons. In surveys, failure to understand the question or interpreting it in an arguably legitimate fashion, but one that leads to a different meaning for the answer than is interpreted, is a huge problem. The giant government surveys are better than average for this, but still not what you would call good. Almost every time I see the instrument for a one-off ad hoc survey, I find myself thinking “you must be kidding! did you not pilot this even a little bit?” Some of the questions are unfathomable or have obvious multiple interpretations. And then there is the simply matter of respondents being bored of the whole thing (either the whole time or after they have already spent too much time on it), and answering while only half reading or even randomly.
And none of that yet gets to your point about respondents intentionally lying, out of embarrassment or self-deluded bravado, or just for the lulz.
There are ways to try to deal with some of that, but they are almost never used in this stuff. You can ask what is effectively (or even exactly) the same question multiple times and check for inconsistency. You can have nonsense answers or put nonsense items in table-style series questions (so long as there is a “don’t know” type option). If you do commercial surveys, you have probably noticed this included. But they pretty much leave only the option of throwing out someone’s data. It obviously does not offer a way to fix bad responses. This is fine for a survey where tens of thousands of people are taking it to get a few extra frequent flyer points or whatever, and where you don’t really care about representativeness. But it is not an appealing option when you can barely afford a big enough sample already and part of your claim about the value of the survey is that it is representative of a population.
So people doing and interpreting these surveys are stuck pretending that all the answers are right. They could do a better job of eliminating the confusing or ambiguous questions, but there is no way to prevent randomness or intentional misrepresentations. There are ways to measure how robust the conclusions are against some posited level of that in the data but (surprise!) no one every does that.
So, in conclusion, I will try to remember to say something more about this in future analyses.
Thanks Carl.
If I may ask a related question or two:
are these surveys ,mostly, done via phone calls to fixed landline numbers? Or are they done by random door knocking , or are they done by advertising for participants ?
And : if they are done by advertising for participants, do they offer some kind of reward-inducement to encourage people to attend?
I have not made a count of it, but I am pretty sure phone surveys are most common for adults. Some are trying to improve coverage by including mobile phones. (But who answers their phone from an unknown number rather than letting it go to VM?) Door-to-door is a lot better, but obviously more expensive. Surveys of kids are often done in schools (which obviously misses a certain segment).
Any survey that employs advertising is going to be a convenience sample, not representative. Few if any of the surveys relevant to this analysis will be that — it would really be difficult to make sense of convenience sample results. Most advertising recruitment surveys are commercial, which often offer a reward (no idea what % of the time or anything). Sensible surveys all offer some token reward — not enough to really make it worth people’s time (so they are still being altruistic to do it), but enough that they feel good about paying attention.
“This is where the sci-fi metaphor gets interesting. If someone does accept their conceit — that tobacco control policies and programs are having a big effect — they must concede that “but prevalence is declining so…” is nonsense.”
But what if one doesn’t accept that? I believe in the UK the decline in smoking more or less stagnated from 2007 when they implemented a total ban indoors to around 2011 when vaping started to become mainstream. What explains the decline since then? I can’t see that Tobacco Control policies suddenly started to have an effect after 2011 that were more or less exactly the same as ones implemented in 2007?
Also what happened to the natural decline between 2007 to 2011? It would seem to me that a reasonable case could be made that TC policies had no effect and hence are not relevant to the decline?
If one does not accept that, then my previous point applies. The point you quote was presented as an “this is even more clearly true if…” point.
Carl
In the context of drug use etc, is there much evidence of gateway effects – I mean causal effects- anyway?
I remember all that BS about dope leading to the hard stuff etc.
I suspect that all the studies of street drugs suffer from the same problems that the tobacco studies do, though I have not made a careful study of them. So there is inevitably some causation, but the research is not able to detect it.
However, it is fairly clear that there is a major gateway from medical opioids to street drugs. Quite a few people who would never have been interested in buying heroin or fentanyl on the street end up doing so after taking medical opioids (and having them cut off — clearly part of the main causal pathway).
The science fiction analogy is excellent. There are different types of science fiction though. Asimov and even Star Trek are very different from Star Wars (more a “science fairy tale” than science fiction). Some science fiction movies are full of junk science, for example the film “2012” (besides crappy acting). Not hard to figure out that most hardline tobacco controllers should be much closer to Star Wars or 2012 than to Asimov.
From my perspective the “gateway” studies are hopelessly deficient from birth. They try to find a very special causality (vaping -> smoking) between two ill defined blurred fluid variables (the “who is a smoker” and “who is a vaper”) within a very complex phenomenon that involves many interrelated variables and causalities. In fact, perhaps statistics is not even the right tool to look at this, but games theory or neural networks.
Most studies I have seen seem to be attempts of a sort of sociological analysis (they use methods of social sciences). I try to find analogies, like suppose there are two competing political parties: extreme right X and extreme left Y, with X waging a crusade against Y, believing it to be absolutely evil (this is easily reversible). Then, suppose that a new party Z appears that is moderate left, it accepts a lot of premises of X but disowns Y and aims at substituting Y. Some leaders of X welcome the new party and even join ranks, but the orthodoxy of X remains obstinate in proving that Z is as evil as Y. They commission social scientists to prove that Z is acting like a “gateway” from X towards Y, that is, “good patriotic citizens” that have always voted for X that would have NEVER voted Y will end up voting for Y because of the existence of Z. Proving such gateway is really daunting and depends on a million factors besides the parties’ ideologies. Notice how polls tend to get it wrong. Obviously, some determined sociologists (aware of what their paymasters in X want to hear), using some sort of undergraduate statistics, will find the much desired gateway effect, but the point is (as you say) what is the significance and connection to a hypercomplex reality of what they found? Likely, nil. Yet they get paid and promoted (grants, reputation, etc).
Now, looking at the politics behind all this, since tobacco control orthodoxy has problems to deal with issues on vaping that involve health sciences: the question of “is vaping safe or toxic”?, the only front they have left is sociological and involves testing behaviours: does “vaping turns kids into smokers”? or “is it efficient for quitting efforts”?. These issue have nothing to do with the safety of vaping. Vaping could be 100% safe and still be inefficient for quitting or could attract some “kids” to smoking. This does not justify that vaping should be massively rubbished with junk science. Those concerned with these behavioral issues should deal with them and leave vaping and vapers alone. Perhaps tobacco control orthodoxy persecutes vaping because of their frustration for not being able to solve these issues.
Ok, first and most important, Star Wars is not sci-fi. It arguable is not even fantasy set in a sci-fi setting, but rather is basic formulaic medieval fantasy with with a space travel skin overlay. That is not just because most of the central characters are wizards, but because of other unexplained (and seemingly very difficult to justify in sci-fi terms) variations in physics. Apparently it is not possible to make weapons that are based on kinetic energy (does that mean you can’t even punch someone?) nor to create a guidance system (not even a WWI-level bombsight, and no guided missiles).
Anyway…
I was really limiting my sci-fi conceit to the four corners of the letter. But if you want to extend it, you could say that psychology research is sci-fi while public health research is fantasy. Psych research is terrible, but that is because (most of) it follows a set of rules that is wrong. Public health research is worse because for each bit of it, the rules are just made up ad hoc to advance the plot.
I have pointed out how difficult it would be to genuinely measure the gateway effect. It is much harder than, say, figuring out if a particular food is healthful. Considering public health researchers cannot even do a decent job of the latter, it is pretty hard to imagine them doing the former. It would take the brains and discipline of a good physicist or other serious scientist who is trying to answer very tough questions, and there are only a handful of thinkers of that quality in public health research.
The work does not even use the methods of social sciences. Economic science and quantitative polisci are sloppy compared to a lot of physical science for various reasons, but they do include methods designed to deal with some of these problems. Public health research does not. What you describe is really more news punditry, not social science. A social scientist might look at that and identify some way to measure, or at least some testable implications of the hypothesis, or might just say there is no apparent way to scientifically analyze this particular one-off event. But the social science questions about vaping are not actually like that. They are obviously potential targets of scientific analysis, just nothing as simplistic as has been done (by all sides, I should add — the retorts to the extreme tobacco control claims are barely a better excuse for science than the originals, if that).
You could tie the whole thing together and say that “research” is to real social science as Star Wars is to science fiction.
Hi Roberto
I like your analogy to political parties .
In ‘marketing’ terms TC starts from a ‘axiom’ : Vaping -and other smokeless products- as a product, are not a substitute-replacement for smoking.
Obviously IF you accept that Vaping is overwhelmingly a substitute-replacement for smoking , then the proposition that increasing demand-use of the substitute-replacement product will lead to increased demand-use of the product that it is a substitute-replacement for, is a rather weird proposition- Akin to saying increasing sales of mobile phones will ’cause’ more people to install landlines.
“… that ‘research’ is to real social science as Star Wars is to science fiction” Agree, Star Wars is good and evil interacting through wizardry and magics embedded in a “sciency” outer space scenario. Arguing that real gateway effects, with unequivocal one directional causality, can be accurately measured with relatively simple methods is also magics embedded in a “sciency” environment. I was not dismissive of real social sciences, notice in my political analogy that I highlighted the fact that only purposely commissioned sociologists would be able to show a simple “gateway” causality in highly complex systems such a voting patterns. Sociologists or economists not bound by this type of special interest would (as you rightly say) not oversimplify this reality and find something measurable, or simply give up and admit that measurements or predictions are not reliable.