For an explanation of what this post is, please see this brief footnote post.
The paper reviewed here is available, open access, here.
It is vaguely embarrassing to write a review of this piece, which is effectively a bad local newspaper story, dressed up as if it were scientific research (though not nearly as embarrassing as being the the journal that published it). However, it is worth a few minutes because it will inevitably be used in the absurd-but-persistent propaganda efforts to claim that bans — in particular, flavor bans — do not create alternative supply chains. The open access status of the article, unlike most articles at the journal, is a bit of a giveaway about its purpose.
On the upside, it is a good quick teaching example about the standard public health research practice of ignoring competing hypotheses and explanations. There are three layers of that in this case.
The author observed that after the 2015 ban of menthol cigarettes in Nova Scotia, government data did not show a substantial increase in seizures of illicit trade cigarettes. From this he concludes there was no “surge” in illicit sales. That is literally the entire analysis. It is basically a one-day’s-work feature story from a local news reporter (who, unlike someone writing an ostensibly scientific paper, could be forgiven for using a vague term like “surge” as a unit of measure).
The reported government data shows constant levels of seizures, modulo noise, from 2010 to 2018. The author chose to start reporting the time series in 2008, presumably because that year had three times as many seizures as subsequent years, thus creating a graph that fools the reader into seeing a downward trend where there is really no trend over the relevant period. There is actually an uptick at the time of the ban, though nothing should be made of this. Indeed, nothing should be made of any of this.
As is typical in “public health” papers, the observation “this data is consistent with the hypothesis” is erroneously interpreted as “this data supports the hypothesis.” There are multiple competing hypotheses for lack of signal that exist alongside the “no increase in illicit sales” hypothesis.
The most obvious — which is clearly true to at least some extent, and thus ignoring it is inexcusable — is that established illicit suppliers have gotten better over time at not getting caught. Amateur upstarts get caught, guaranteeing some photo-op seizures, but organized crime continues to exist because it is successful. (Perhaps tobacco controllers do not believe in evolution by natural selection.) Under this hypothesis, the established suppliers could have upped their game to meet new demand without having any effect on seizure data. The author did not even acknowledge this obvious competing explanation for the observation, let alone provide any analysis that suggests it does not invalidate the paper’s conclusions.
Another hypothesis involves the easy small-scale mentholation of non-menthol cigarettes. This can be for personal use, but it is also possible there was a “surge” in illicit sales as these adulterated products, sold in small quantities and would never end up in the seizure data. This does not seem terrible likely because there probably was not really much interest (see below), but the point it is possible, and the author (again, writing an ostensible scientific paper, not a quick newspaper story) failed to address it. It is worth mentioning because this DIY scenario is much more likely to happen in denser population where menthols are more popular, and for e-cigarettes in the event of flavor bans.
The author seems to be vaguely aware there is such a thing as a competing hypothesis. He asserts, “According to the local authorities…the enforcement efforts in Nova Scotia have not declined in the recent years”. That is literally the entirety of that analysis — no explanation for what it even means, let alone an assessment of whether that assertion is accurate — but it still belies a recognition that there are alternative explanations for the data. However, the only such explanation that is implicitly acknowledged is this one that the author implies he can dismiss. However, since does so only with hand-waving, a reduction in enforcement efforts is really still on the table as another competing hypothesis.
A second layer of failure to examine competing hypotheses comes at the level of extrapolation. Nova Scotia is geographically isolated, a relatively small and non-dense market, and is populated by eastern Canadians. It is not a worst-case-scenario for illicit traders, but it is certainly in the neighborhood. Thus, while it is possible that attempted noncompliance with this particular ban really was minimal and ineffective, that tells us approximately nothing about anywhere else. Yet the crypto-theme of the entire paper is about extrapolation to other jurisdictions (“implementation of new tobacco control measures almost never leads to substantial increases in illicit cigarette trade”; “in the USA, the industry claimed….”; “tobacco industry predictions that the new tobacco control measures will cause a dramatic increase in cigarette smuggling do not seem to ever materialise”). The implicit hypothesis is that if it worked there, it will work everywhere. This is unsupported and obviously dubious.
The third and most interesting layer of failure to address competing hypotheses can be found in the underlying assumption behind the ban, which is endorsed by the author: menthol cigarettes are substantially more appealing (at least to some consumers) than non-menthol cigarettes. The common implicit hypothesis in a lot of tobacco control propaganda is that if, given a choice between products X and Y, someone would choose X, then if X were banned she would choose abstinence over Y. The story in this paper is one example of that. In reality, a compelling alternative hypothesis is that few consumers care enough about menthol in their cigarettes, even though they prefer it, to seek out alternative supplies rather than just settling for non-menthols.
If it were (genuinely) observed that there was no increase in illicit trade in response to a menthol ban, and especially if no small-scale mentholation operations popped up, that would support the alternative hypothesis. That is, if there really is no illicit trade increase in response to an anti-consumer tobacco control measure like this, then it actually did not serve its purpose of hurting consumers enough that they cared. The author ignores this, the most interesting implication of his ostensible findings.
The author actually reported specific data that is relevant to this point, but is either blinded by the propaganda or was trying (badly) to hide its implications. The post-ban seizures did not show an increase in prevalence of menthol variety cigarettes among what was seized (indeed, apparently there was a decrease, though again this is all just noise). There was no apparent change in the mix of seized product, which consists mainly of tax-evading product from medium-scale manufacturers (unstated: primarily Native American) in the Great Lakes region. If many consumers really cared much about menthol, there would be a change in this mix, with more menthol product, including a supply of diverted menthol brands. If the reported information is correct, however, many consumers care about price (taxes) enough to become black market customers, but few if any care enough about menthol to do so. Thus anyone who claims to believe the conclusions of the “study” should be reaching the conclusion that menthol bans will not have much effect on consumption.
The introduction of the paper is a joke. Even though the section is entitled “Background”, it offers no background about the topic of the research (demand for menthol, Nova Scotia, the local market, the ban), previous related research, or the data and methods used. Continuing on the latter point, the Methods are even more of a joke, with the methodology and content of the data described only with “data…were obtained from the Audit and Enforcement unit of the Provincial Tax Commission, Service of Nova Scotia.” That is what passes for a reporting of the methods in this paper. The Discussion is a bit more useful, since that is where the author inadvertently reveals that he has useful data he avoided analyzing.
The Background and Discussion are mostly ham-handed rants about the evils of “the tobacco industry” as well as personal political statements in favor of bans. This makes the lie particularly apparent when the author claims he has no competing interests. The paper basically states that he is presenting this information for the purposes of advancing his political agenda. Moreover, the author has overwhelming financial competing interests that he also lies about when denying COI: He is employed by a political activist organization that supports bans like this, and so it is effectively his job to produce these particular conclusions. The only possible justification for claiming no competing interests is if he takes the (seemingly accurate) position that because he had no interest in producing accurate scientific information in the first place, there was really nothing for his political and financial interests to compete with.
Tobacco harm reduction, anti-THR lies, and related topics 