Letter re fatal flaws in Schober et al. paper on environmental vapor

by Carl V Phillips

Below is a letter to the editor and Igor Burstyn and I will be submitting to the journal that published this article — “Use of electronic cigarettes (e-cigarettes) impairs indoor air quality and increases FeNO levels of e-cigarette consumers” by Schober W, Szendrei K, Matzen W, Osiander-Fuchs H, Heitmann D, Schettgen T, Jörres RA, Fromme H — which made alarmist claims about the content of exhaled e-cigarette vapor and the biological effect on users.  It is wrong on almost every count, with the conclusions not supported by the data and various other problems.

This might come as a shock to all of you who believe in the perfection of the academic journal system, but journals are often not so good about publishing letters that point out that articles they published are fatally flawed (in spite of that wonderful institution of peer review!).  So this might be the only place it is published.  Please point to it any time someone inaccurately suggests the claims made by those authors actually reflect their results.

Note in particular the analysis of “particulate matter”, which is a good general response to any of the anti-THR liars who traffic in alarmist rhetoric about particles, either out of scientific illiteracy or actual malice.  [Update: Thanks to a useful observation by McFadden in the Comments, we have updated this discussion.  The new bit is underlined (which is obviously just to illustrate the update here and will not be included in the submitted version).]


The recently published article by Schober et al. [1] about environmental air pollution from e-cigarette use suggests lab error, misunderstanding of the measured results, and unacknowledged political bias on the part of the authors.  The authors should be commended for analyzing the e-cigarette liquid as part of this study of the aerosol, which has been noted as a serious omission in past research [2].  The outcome measurements, however, are suspect, and the conclusions even more so.

The methodology is poorly described:  The authors seem unaware of the fact that vaping behavior strongly influences the environmental release of the aerosol from e-cigarettes, and failed to either control or report relevant details about subjects’ behavior.  Experienced users can maximize efficiency and almost eliminate environmental release, whereas new users (or experienced users who are intentionally producing visible clouds at the cost of inefficient consumption) can release a large portion of the aerosol into the environment.  The authors fail to report the efficiency of the behavior, though we know they recruited inexperienced users (smokers who were only briefly instructed on how to use e-cigarettes).  The environmental measures were higher when the users were using nicotine-free e-cigarettes (Table 2), which demonstrates the behavioral effects: without nicotine, the users have no reason to puff in a way that efficiently absorbs the aerosol.

The authors emphasize the increase in PAHs in their test chamber, but correct use of e-cigarettes does not produce PAHs.  This suggests either the inexperienced users were overheating their devices, the devices were faulty, the PAHs were outgassing from the users (who were smokers), or lab error.  In past studies, high PAH values have been attributed to lab error, probably cross-contamination with cigarette smoke, which was seen as calling all the study results into question [2,3].  But even if the results were not lab error, they do not appear problematic:  The authors show no increase in the most carcinogenic PAH tested (benzo[a]pyrene) and all other reported “increases” were within experimental error reported by the authors.

Results about metals that are presented without attempting to determine the parent molecule are misleading, evoking thoughts of dangerous molecules, but probably actually representing common harmless ones [2].  The presences of sodium almost certainly represents trivial quantities of NaCl.  The authors emphasize aluminum, but that seems also likely to be in the form of biologically unimportant salts.

Results about FeNO exhalation (Figure 1) are intriguing but simply confirm that there is no evidence that there is a biological effect: In half of the subjects there is a weak effect (those with high baseline of FeNO) and there is no effect in subjects with low baseline FeNO.  It is likely that if the data were represented as percent change of baseline, there would be no systematic increase. This suggests there is nothing more than experimental noise that the authors interpreted as meaningful.  Further investigations of biomarkers in exhaled air, if done on a large scale and under representative vaping practices, could prove informative about early effects of vaping on health.  The present results can only be viewed as reassuring, in contrast with how they were spun.

The authors found (Figure 2) that subjects’ urinary nicotine concentration increased after using nicotine-free e-cigarettes, which they attribute to the subjects smoking, contrary to the stated methodology.  Whatever the explanation, it indicates a serious error in the methodology which was inadequately investigated.

The biggest problem, suggesting both a lack of understanding and possible political bias, is repeatedly referring to the aerosol droplets as “particulates” and suggesting they behave similarly to solid particles.  While droplets are particulates in the broadest sense of the term, in the context of environmental pollution that term generally refers to fine solid particles that can lodge in or be absorbed through the lungs intact.  A liquid, of course, just dilutes into the bloodstream or other bodily liquids, regardless of particle size and deposition location.  Thus, the extensive discussion of particulate size, let alone the explicit claims about health implications, is highly misleading.  Indeed, the results they found are not all that different from the “particulate” exposure when someone takes a cold shower in terms of both “particle” size and concentrations [4], which illustrates the need to characterize the tiny bits of matter that disperse light, not merely determine that they exist.  The device the authors used to detect “particles” does not distinguish between droplets and solid particles; to assess any health-relevant particles the authors should have used gravimetric techniques that determine the mass of solid particles emitted into the air.  As such, the authors’ work suffers from inadequate testing of their major conclusion and confirmation bias: they assumed health-relevant particles would be present in the aerosol, performed a test that was incapable of ruling that out, and then interpreted their results as confirmation.

The authors demonstrate a further political bias – not disclosed in their conflict of interest statement – by (1) cherry-picking the existing biomarker literature, (2) making unsubstantiated claims about motives for using e-cigarettes, (3) including an irrelevant (and inaccurate) discourse about the effects of nicotine, (4) suggesting that the policy-relevant comparison is between a straw man, “emissions free,” and any emissions whatsoever, and (5) recommending restrictions on e-cigarettes.  They fail to assess the implications of their results for bystanders, the crux of their political claims, which would show how trivial the impact would be even assuming their results were valid.  It is, of course, impossible to make a legitimate policy recommendation without analyzing all the impacts of the proposed policy; context-free results from a single study are clearly not sufficient.  Various regulations may or may not be warranted, but that cannot be determined based on the mere observation that e-cigarettes are not “emissions free.”  The inclusion of such recommendations in a research report suggests that the authors were just doing the science as an excuse for expressing their political beliefs.

Carl V Phillips, PhD, Scientific Director, Consumer Advocates for Smoke-free Alternatives Association (CASAA), USA

Igor Burstyn, PhD, Associate Professor, School of Public Health, Drexel University, USA

1. Schober W, Szendrei K, Matzen W, Osiander-Fuchs H, Heitmann D, Schettgen t, Jörres RA, Fromme H. Use of electronic cigarettes (e-cigarettes) impairs indoor air quality and increases FeNO levels of e-cigarette consumers. International Journal of Hygiene and Environmental Health, 2013 (online publication ahead of print).

2. Burstyn I. Peering through the mist: systematic review of what the chemistry of contaminants in electronic cigarettes tells us about health risks. BMC Public Health 2014, 14:18.

3. McAuley TR, Hopke PK, Zhao J, Babaian S: Comparison of the effects of e-cigarette vapor and cigarette smoke on indoor air quality. Inhalation Toxicology, 2012, 24:850–857.

4. Zhou Y, Benson JM, Irvin C, Irshad H, Chen YS. Particle size distribution and inhalation dose of shower water under selected operating conditions. Inhalation Toxicology 2007, 19(4):333-342.

16 responses to “Letter re fatal flaws in Schober et al. paper on environmental vapor

  1. Carl and Igor, excellent analysis!

    I have a suggestion that might greatly strengthen your point about droplets (Your explanation btw was very informative: I’ve wondered about that in the past!)

    Here’s my suggestion: Find a grubby hippie (I’m willing to be the guinea pig.) and have him take a nice hot shower while you stand outside the shower curtain and measure the “particulates” produced by the steam. Heck, spring for a bottle of that nice formaldehyde-rich baby shampoo and watch the e-cigs’ readings get blasted into nothingness by comparison! Throw in some deodorant (It’s against my religion, but for the sake of science I can make the sacrifice.) and some aftershave fumes, and that bathroom atmosphere will look as deadly as the surface of Venus when compared to an e-cig party!

    – MJM

    • Carl V Phillips

      That is a great suggestion. It is definitely a good way to shoot down the claim. Well not the DIY part — fortunately once you get away from tobacco, you discover that scientists have actually done the useful obvious studies, such as what is the droplet size distribution in a shower: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2874882/ It looks like the droplet size distribution for the e-cigarettes, predictably, skews smaller than shower fog, but not so much that they are fundamentally different. There are plenty of the small “particles” that the authors claim to worry about. Quantities from showering are much greater than four of Schober’s measurements, lower than their huge outlier highest, about the same as their second highest. (I wonder if Glantz avoids showering because he is afraid of the “particles” — wouldn’t surprise me.)

      Might be worth adding before we send in the letter (couldn’t do it yet because it turns out the journal does not even tell you how to send in letters, so we need to investigate further or find another journal).

  2. Thank you! It’s a shame the peer reviewers didn’t pick up these points

  3. Nicely found Carl! :) Now, two points not mentioned in that piece of research that might be of some interest (and which I tried following up unsuccessfully back in 1999 when my “working title” for “Brains” was “Deadly Doorknobs, Sinister Showers, and Secondhand Smoke” (seriously!) I was intending to examine the “dangers” of asbestos fibers in that water being released into the air along with chlorine gas. I *did* eventually get an opinion from someone at the EPA or Water Dept or someplace that asbestos in the water system “wasn’t carcinogenic” but they seemed to be basing that on digestive tract cancers and seemed to have a mental stone wall against relating the idea to inhalation. Over the years since then I have explored (several times) the concept of the danger of living with clean-living yuppies as opposed to grungy hippies due to “secondhand showering” but the concept seems to be a bit beyond the scientific horizon. However, if we take the idea that nonsmokers flock to the showers after being exposed to secondary smoke (to wash “the stink” off themselves) then it could certainly explain any tendency of studies finding increased lung cancers in smoke-exposed nonsmokers!

    – MJM

  4. Jim Rothenberger

    Michael, you’re the best. You get a smile out of me every time I read your stuff.

  5. The article reminded me of the excellent “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity” by Alan Sokal, (1996). http://www.physics.nyu.edu/sokal/transgress_v2/transgress_v2.html

    It looked very clever, but was entirely made up…

    The difference is that Sokal did it to expose the poor rigour and miserable peer review standards of the cultural theory literature. In this case, that is an unintended outcome.

    • Carl V Phillips

      LOL. The think about the Sokal paper is that at the time it made a splash but now the reaction of anyone who is thoughtful and aware is more like, “that seems like an awful lot of effort to try to make such an obvious point.”

      Sadly, I would have to say that the Schober is better than the average paper in a public health or medical journal, so much so that it did not even strike me as unusual that it was fundamentally misleading. I do have to say — based on rather less observation and expertise — that it appears the journals that publish about air chemistry seem to be about as willing to let through junk as the epidemiology-based journals.

      Other than the one critical variable they omitted, it seems Schober et al. basically described what their methods were, and so someone could actually replicate their work. This is almost never the case for epidemiology papers (with the exception of a few highly-controlled protocols, like drug trials), where the methods are so opaque that your really do not even know what they did. The typical observational epidemiology paper includes unreported details about how they analyzed the data, such that you have no idea if they were cooking their results. At least in Schober, the results were probably what they reported unless they were out-and-out fabricating data.

      Their worst errors (or, perhaps, intentional lies) were misinterpreting their own results so badly, and claiming that they could draw policy conclusions from one lab study. This is fairly typical of epidemiology too. Hmm, no, that is too charitable. This is nearly ubiquitous in epidemiology (again with the exception of the highly-controlled protocol side of it).

      Bottom line, it is fun to joke about how absurd journal articles in PoMo critical theory are, but what passes for science in public health is really no better. Unfortunately, it is more widely taken at face value by naive readers.

  6. Pingback: A paragraph-by-paragraph dissection of an unusually good article about e-cigarettes (part 3 of 3)) | Anti-THR Lies and related topics

  7. Pingback: Glantz takes an vacuous swing at Bates; Pruen eviscerates Glantz | Anti-THR Lies and related topics

  8. Pingback: Scientific sleight of hand: constructing concern about ‘particulates’ from e-cigarettes « The counterfactual

  9. Pingback: What is wrong with ecig particulate claims – the simple version | Anti-THR Lies and related topics

  10. Pingback: Déjà-vu Ökotest - Zwischen und hinter den Zeilen

  11. Pingback: Serious ethical concerns about public health research conduct; the case of vape convention air quality measurement | Anti-THR Lies and related topics

  12. Pingback: What is peer review really? (part 1) | Anti-THR Lies and related topics

  13. Pingback: Is misinformation giving you the vapors about vaping? – Lake of vape

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s