by Carl V Phillips
The latest kerfuffle around e-cigarette junk science comes from this toxicology study or, more precisely, this press release that is vaguely related to the study. Basically, a San Diego toxicology research group bathed cells in a very high dose of the liquids that come out of an e-cigarette, and eventually there were detectable changes in the cells. That is really all you need to know about the study’s actual results. (If you want more background, see Clive Bates’s post.) Contrived experiments like this provide nothing more than a bit of vague information that might someday lead to insight about the real world, though probably will not, and so might be worth exploring more using less ham-handed methods. That is all the information this type of research ever provides. No worldly conclusions are possible. It is vague basic science research that even at its best merely points the way for further research.
There seem to be two reasons this study got kerfufflized more than the usual stream of similar — i.e., *yawn* — research from all the researchers jumping on the anti-ecig gravy train: the press release is more lie-filled than average, and the behavior of one journalist. The main claim in the press release is:
“Our study strongly suggests that electronic cigarettes are not as safe as their marketing makes them appear to the public,” wrote the researchers, who published their findings in the journal Oral Oncology.
The little problem here is that this is utterly wrong, and at multiple levels. First, their study says nothing about the safety of e-cigarettes whatsoever. The fact that massive dose of some chemicals, applied to a very rough surrogate for the human body, causes something to happen tells us nothing about health. Anyone with a minimal understanding of these research methods knows that. Either the researchers who wrote that know they are lying about the implications of their research or they are lying about understanding their field of research.
But the statement actually is worse than that. How safe does “marketing” make e-cigarettes “appear to the public”? No real expert would even consider trying to answer that. We could provide some assessment about overall public perceptions of risk from e-cigarettes — which show that on average people grossly overestimate the risk — though even this is pretty unclear. However the claim is actually about the effect on this perception caused by marketing specifically, and there is no apparent way to assess that. Of course, these particular researchers presumably understand none of these points, because they are a bunch of lab chemists with no knowledge about public perceptions, marketing, or human health, nor any apparent knowledge about e-cigarettes beyond the uninterpretable results from their glorified high-school science experiment. What’s more, this is a quantitative claim that lacks any basis for quantification, a key failing I will circle back to.
Those authors’ apparent casual willingness to issue a mind-bogglingly anti-scientific statement leads us to the general observation described in the title of this essay. Public health, including the research, politics, regulation, and journalism surrounding it, is an inherently dishonest discipline. By that I obviously do not mean that every single person involved is a liar. I obviously do not mean that everything done or written in the area is wrong. What I mean is that the field has no checks on dishonesty. It does not discourage it, and it certainly does not punish it. In most cases, it actively rewards it. There is no culture of duty or honor in the field, and thus no notion of what it means to behave like a real honest scientist. As a result, nonsense abounds. Those who actively take advantage of the opportunity to be dishonest flourish and rise in stature. The resulting core rot permeates the entire space.
The above statement out of San Diego (ironically named after someone whose main claim to fame was that his body supposedly did not rot) is an utter travesty. It is both clearly wrong and unscientific. And yet there is no chance that those responsible for it will suffer any harm from having issued it. They obviously did not have any honor or duty qualms about touting it. Within their public health bubble, they will probably never even experience anyone asking them to defend this claim, let alone questioning their competence or integrity. When I was a professor of public health, I was appalled at the attitude in seminars and similar venues, that it was impolite to scientifically challenge someone’s supposedly scientific claims. People acted just scandalized — cue melodramatic swoon — by questions that would have been considered mild in a seminar in economics, political science, philosophy, natural science, and various other fields I have worked in and around.
Even worse than the thesis claim quoted above, the money-shot from the press release was placed prominently at the end of the text:
“Based on the evidence to date,” she [“Dr. Jessica Wang-Rodriquez, one of the lead researchers on the new study”] says, “I believe they [e-cigarettes] are no better than smoking regular cigarettes.”
So here is some chemistry lab hack, who obviously has no idea what she is talking about, making pronouncements about real public health issues that will inevitably do serious damage to both scientific understanding and actual people’s actual health. A failure of this magnitude in doing one’s job properly, or even just at the level of the first quote, would likely get someone fired from a real job. Even within most fields of academia it would earn the lasting scorn of serious members of the profession. It might still advance someone’s career for the same reasons it is wholly rewarded in public health — because it plays to the preferred politics of those in power and is great click-bait — but there would be scorn. (See, e.g., the effects of his series of gaffes on the professional assessments of one of my old economics professors, Alberto Alesena.) In public health, there is no such scorn because there is no notion of duty. There is no scorn for even blatant lying.
Similarly, there is no penalty for just making up concrete sciencey-sounding claims out of thin air. The quantitative comparisons quoted here were undoubtedly made without the benefit of any quantitative thinking. How safe does marketing make e-cigarettes appear? I doubt the authors gave that any thought whatsoever. How harmful does the chemistry lab result suggest that e-cigarettes are? Presumably they gave that a few minutes’ thought, in which case they must have reached the obvious conclusion, “I have no idea.” And yet they made the comparison.
Once again, it is not as if public health is the only news-generating field of inquiry where this happens. But what is unusual about public health is that no one seems to even notice such absurdity is occurring. It is such a dishonest field of inquiry that it does not even occur to anyone to ask, “what models of marketing and beliefs, or even of how to map in vitro toxicology results into real health effect estimates, are you invoking and populating when you make this quantitative claim.” By contrast, for example, here is a complaint, from this week, about the failure of some partisans in supposedly-scientific economics debates to produce even prima facie models in support of their claims. In both cases the model-free claims were made, but in the economics case the experts questioning the claims are discussing this obvious failure. I have read quite a few critiques of the San Diego claims, and not one of them brought this up, even the ones that pointed out that toxicology results do not necessarily imply health effects. The field creates such an aura of nonsense that even its critics overlook the fundamental problems.
Of course, I realize that the critiques were mostly focused on damage control, explaining that Wang-Rodriquez quote was a lie and trying to save some future victims of the desk-murder she committed. But… It’s a trap! To paraphrase Twain: Never argue about a detail with someone whose whole approach is dishonest; onlookers will tend to interpret it as you endorsing the rest of their approach. Worse, once someone gets in the habit of doing that, they subconsciously do end up endorsing the core of the dishonest approach.
The final Wang-Rodriquez claim was also a model-free quantitative claim. The underlying model needed to make such a claim is extremely simple; it require no great numeracy or expertise, as is the case for more sophisticated sciences. The obvious response: “What diseases are you claiming vaping is causing that could come anywhere close to the effects of smoking (after all, something cannot just be generically “harmful” — there has to be some particular harm it is causing)? Are you saying that each disease caused by smoking is caused by vaping, and to a similar degree? If not, you must be claiming that some diseases are caused by vaping to a greater extent then they are caused by smoking, so which ones?” But most of the responses ended up implicitly endorsing the core dishonest approach of making quantitative claims without any basis for quantification; they just quibbled about what particular unmodeled quantitative claim should be made.
In what is to this day still the only serious attempt to estimate comparative risk among categories of tobacco products, I and my colleagues endeavored to assess the overall risk of smokeless tobacco use compared to smoking by adding up risks for actual specific diseases. I cannot think of any other way this could possibly be done. Of course, we currently know a lot less about e-cigarettes than we knew then about smokeless tobacco. But you can still use the same model and offer best guesses about what form the harm takes. Wang-Rodriguez would have to assert a lot of absurd numbers into the model to get 100% of the risk from smoking. Indeed, you need to posit some utterly speculative big numbers to get 5% of the risk from smoking. If someone has a model in mind that is not based on adding up the risks for individual diseases (or disease categories), they need to explain what it is. I cannot think of any other option. But critics of the “just as bad as smoking” claim, having been dragged down to the level of public health discourse (to borrow from the Carlin variation of the Twain quote), respond to the anti-scientific nonsense on its own terms rather than noting the core failure.
Questioning the analytic model would be the first thought of most serious observers in other fields of inquiry when seeing a claim like Wang-Rodriquez’s. Yet none of the responses and critiques in the kerfuffle that I have seen addressed this core point (Bates seems to starts to move in that direction, but stops short). Why is it overlooked? Because the critics also are products of public health, either by being fully part of it or by virtue of having learned their approach to science by watching the public health people. It is a world that is all about seat-of-the-pants superficial claims rather than drilling down to what fundamental scientific claims are really being made. Yes, dear reader, do not get too comfortable with the thought that, “everyone who disagrees with my personal preferences is part of a dishonest discipline.” The whole space is rotten to the core, and that crosses party lines. It is a very rare individual who can live immersed in a discipline with such fundamental problems and not suffer from them personally.
But before circling back to that, consider briefly the other reason that this particular matter achieved such kerfuffledom, the press reporting. Before I woke up the morning this story broke, there was already a Twitter battle targeting Sarah Knapton (@sarahknapton), Science Editor at the Telegraph, who published a naive and harmful transcription of the lies from the press release. Several other UK papers did too, it should be noted (not so much in the USA, it turns out), but Knapton foolishly tried to defend this malfeasance and just kept digging herself deeper. Bates assessed this at length in his post a week ago, but it has actually kept going since then. I suggested she stop digging just two days ago:
Ideally, Knapton would have responded to the criticism by trying to assess whether she had indeed screwed up. That would have been the dutiful and honorable thing to do. At the very least, she should have tried to learn something before saying more (which perhaps would have meant she just stayed silent, a lazy but practical option). That would be the minimal caution exhibited by someone who faces penalties for colossal failure. But because public health journalism (as distinct from real science journalism) is part of the public health penumbra of dishonesty, I suspect it simply did not even occur to her that continuing to argue from a position of ignorance was a bad idea. Indeed, when you work in a field where there is no penalty for being blatantly incorrect, you tend to forget that there even is a distinction between correct and incorrect.
Again, just because public health journalism is part of a dishonest field does not mean every science journalist covering the matter is dishonest. Consider this nice assessment of the kerfuffle by Bradley J. Fikes at the San Diego Union-Tribune, about what went wrong with the reporting. Fikes summed up the situation with:
Everyone makes errors — journalists and scientists alike. But failure to correct errors once pointed out, as happened here, is just wrong. And that so many reporters got it so wrong reflects a systemic problem. Copy and pasting press releases is more popular than actual thinking about the underlying claim.
He goes on to further condemn the transcriptionist approach to reporting. Of course, some reporters in other areas are plenty guilty of that. “All governments lie” is a truism that motivates responsible public policy journalism, and yet most corporate media reporters just transcribe government claims uncritically. But again there are differences: All but the most unskilled or pandering reporters tend to have some understanding that the claims are politically motivated and include alternative views. When they fail completely, there is a professional community in and around journalism who criticize them for it — not just for getting a fact wrong due to the transcription, but for the fundamental dereliction of duty of transcribing government lies as truth.
By contrast, in public health — where political activist researchers (and “researchers”) lie as freely as any government — transcription is the accepted norm, usually without even the little bit of critical assessment found in public policy reporting. Meanwhile most criticisms from in and around the field focus on specific errors without recognizing the dereliction of duty itself. Fikes offers a rare exception, drilling down to the problem. Bates’s examination delves further into the practice. But these are the exceptions. The series of tweets from Knapton make clear that there is no feeling of shame for failing to uphold any journalistic standards when reporting on public health claims. Duty? What’s that?
Circling back, I noted the critiques of the San Diego lies never got to the core problem, the casual willingness to make and repeat quantitative claims without any hint of a quantitative model. So what did they criticize? Many of them focused on the fact that the researchers also immersed some cells in a cigarette smoke solution and those cells died before the researchers could even look for the minor effects they reported about the e-cigarette samples. The authors tried to hide this fact in their paper and, of course, did not offer a hint of it in their press release. You could use this observation to argue, somewhat legitimately, the following: “If you people (falsely) believe the effects of the e-cigarette treatment in your artificial experiment can be used to draw conclusions about actual health effects, then how can you simultaneously believe that the more dire effect of the smoke treatment does not imply it is worse?”
Now even that is not quite legitimate, but the claim that was actually made by critics — repeatedly, by lay readers and researchers in the space alike — was clearly wrong. They made the affirmative claim that because the cells from the smoke sample fared worse, the experiment showed that smoking was worse for you. Um, no. If you put the cells in a solution of ethanol, or sodium chloride, or an antibiotic drug, you could make them all die also, but those are all good for you to consume (in the right dosage). Of course we know that smoking is worse for you, but the point is that this experiment is not a basis for claiming that.
You cannot have it both ways — you cannot (rightly) deny the health conclusions that are (inappropriately) stated by the original authors, and then try to use another (equally uninformative) result of the same study to make an equivalent claim. But, again, we are dealing with the penumbra of public health’s core dishonesty, and it leaks across party lines. Those who take the public health approach to supporting e-cigarettes tend to exhibit the same casual disregard of proper science of those who take the public health approach to attacking e-cigarettes. There are no professional penalties for making scientifically incorrect claims that are pro-ecig either, after all. There is no sense of duty to be accurate and restrict your definitive declarations to matters you really understand, simply because that is the right thing to do.
Be careful who you choose for enemies, and all that.
Of course, some of the responses to anti-THR lies, at least about e-cigarettes, come from card-carrying public health people. In those cases, it is not even an issue of penumbra. The most prominent debunking of the San Diego claims was Linda Bauld in the Guardian blog. Bauld is smart and prominent, not an unknown lab rat like Wang-Rodriguez, but she is still thoroughly part of the public health and tobacco control world. Most of what she wrote in that post was spot-on. She properly emphasized that cell studies cannot show real health effects. But she then goes on to imply that if the researchers had been able to compare the smoke-exposed samples, the comparison would have been meaningful, and she too suggests that the death of the smoke-exposed cells offers such a comparison.
In addition, she states:
Since then, many new studies have been published, including a Cochrane review showing their promise for aiding smoking cessation, and a comprehensive review for Public Health England that concluded, as previous reports have done, that e-cigarettes were significantly safer to users than continuing to smoke.
While this is really a tangent from her main message about the misleading health claims and the harm they cause, it was the second sentence of her piece so is fair game to scrutinize. Cochrane reviews use a methodology that is basically junk science when applied outside the narrow context of homogeneous medical treatments for which it was designed (the method implicitly assumes that whatever different studies happen to have been done were really part of one big flawless study — never mind that they measured very different exposures, in different populations, different outcomes, etc. — and just combines the results as if that creates new valid information). Worse, in this particular case, the “review” just looked at a couple of studies, clinical interventions that tell us nothing useful about how e-cigarettes actually work for smoking cessation. It is as much a stretch to say that Cochrane paper shows e-cigarettes are useful for smoking cessation as it is for the San Diego researchers to say their results show e-cigarettes are harmful. Sadly, this is not unusual: If you strip out all the throw-away conclusion statements, it is not entirely clear that most pro-ecig public health research is any higher quality than the anti-ecig public health research.
So why does someone who has the brains and courage to challenge (a selected few of the) anti-THR lies from within her field still endorse some junk science and fail to drill down to the core problem. Why does she suggest that a variation on the study might have been informative? The answer, I think, is the same as why that Public Health England report she endorses headlined a quantitative claim that was just a made-up number. It is simply impossible to exist in a world as dishonest as public health and fully rise above it.
When it is the norm in your world to embrace whatever conclusion someone has written down that happens to support your preferred position, ignoring its scientific legitimacy, it is hard to avoid it even when the truth is on your side. If making up quantitative claims without suggesting, let alone populating, an underlying model is the norm in your world, it is easy to just do it yourself. There is no penalty for it. You get rewarded for playing the game. Moreover, even as you challenge some of the details, you will often not even recognize the more fundamental problems, like a fish not recognizing water. Recall that the attacks on that quantitative claim in the Public Health England report were entirely focused on the fact those who made up that number included a few people who had received research grants from industry. The attackers did not seem to even realize that a group of people getting together to just make up supposedly scientific quantitative claims was a more fundamental issue.
(I should note that though I highlight Bates and Bauld by name when identifying scientific gaffes in the present kerfuffle, their average scientific accuracy is much greater than pundits in this space who claim scientific expertise. The examples thus illustrate how even those who most know what they are talking about and are trying to be accurate cannot fully escape the downward pull of public health dishonesty. I will also note it is inevitable I will be attacked for criticizing “our team”. That kind of proves part of my point — it is so impolite to challenge bad science in the public health space. The notion that criticism helps you get closer to the truth, and the truth is the goal, is just foreign in this space.)
In his post, Bates speculates about the press report’s junk claims, suggesting there is a time trend that needs to be explained. He emphasizes structure of modern journalism. Another piece that responded to the San Diego nonsense tried to make it all about the influence of anti-ecig funding sources. While this was clearly the pet facile claim of the author, several prominent e-cigarette proponents volunteered statements in support of her thesis. But there is no need to seek a specific explanation for anti-ecig junk science and the uncritical reporting of it. Everything that e-cigarette supporters decry about the behavior of public health, including those who publicize the claims, is a mere symptom of a core problem.
Indeed, any topic-specific explanation is almost certainly misleading. Uninformed casual observers often try to kludge together causal explanations, trying to assemble the small subset of the facts they are aware of into a causal story — it is basically the same process that results in the formation and persistence of superstitions. But when more knowledgable people do it, I think something deeper is at work. It is an attempt to portray anti-ecig activism as some sort of anomaly that is unrepresentative of public health more generally. Even setting aside the fact that public health has been anti-THR since THR was first seriously advocated 20 years ago, attempts at rational explanation are inherently misguided.
When there is no moral rudder and no rational following of science — when there is only doctrine — then everything is just a matter of a random walk and path dependence. Attempts to construct rational explanations are all superstition, just-so stories designed to back-fill an explanation for happenstance. Public health is anti-ecig because public health is anti-ecig. Yes, you can look back at the path to figure out why it was taken, but that misses the real point. The scripture was penned and here they are. And here they will remain because there are no mechanisms in their system that change doctrine on the basis of evidence. It is easy to imagine recent history taking us down a different random path, where the intensity of anti-smoking doctrine won out over public health’s more general “your body belongs to us, not you” tendencies. Indeed, that fork in the road is why there are a few pro-ecig public health heretics.
But is there actually any room to be a genuine heretic within public health? A generation or two ago, it was possible to be a member of a major party in the U.S. Congress and vote one’s conscience. Today, the only thing that matters for determining any major issue is the R or D after their name. Members are allowed to vote their conscience only for trivial issues or when their vote will not affect the outcome. Party discipline has evolved similarly for special interest groups like public health.
Simon Clarke reminds us that just a few years ago, Bauld authored a UK government report that was demolished for being standard tobacco control junk science. E-cigarette fans were shocked (shocked!) a few months ago when she came out in favor of severe advertising restrictions. It will be interesting to see how things play out with the UK’s cadre of tobacco controllers who support e-cigarettes (there are more there than here in the USA — weaker party discipline, apparently). In a few years, freedom to choose will become the crux fight, rather than freedom to quit smoking. But that digresses a bit, because the point here is whether any of them could openly challenge any of the dishonest core of public health, and whether they would even think to do so, rather than what they do around the edges.
Clarke also suggests that perhaps hounding reporters on Twitter is not a good strategy (and remember, he has more experience at that game than probably any e-cigarette advocate; maybe Clive comes close). Indeed, I would broaden his observation and suggest that I am not sure anyone came out of this looking good. Even those who were on the side of truth got dragged down to public health’s level. There, as Carlin notes, they can usually beat you due to their greater experience, and I will add also because they get to make up the rules.
I guess I should try to summarize my core point here, which is something along the lines of: Fighting around the periphery, playing according to public health’s rules — i.e., casual disregard for core scientific thinking — is a losing game. In part that is because of that last bit of the previous paragraph, but more important, it leaves on the table the opportunity to cut to the core of their nonsense. Trying to pry loose fellow travelers, including honest reporters who are really victims, seems rather more promising. That too is probably better accomplished by cutting to the core. But trying to win over public health by playing their game is fantasy; the party discipline is far too strong. Marginalizing them, as is appropriate given the marginal extremists that they are, is really the only hope. Early in my career there was a small civil war within academic public health, fought by the scientific wing of the epidemiology profession, but the good guys never really had a chance and we are not likely to see another attempt.
(And on that note, I will conclude with an apology to those who go this far: Hemingway was able to turn the pathos of a hopeless civil war experience into For Whom The Bell Tolls, while all I can offer is this blog.)