Serious ethical concerns about public health research conduct; the case of vape convention air quality measurement

by Carl V Phillips

A recent paper in Tobacco Control (official version; unpaywalled version), “Electronic cigarette use and indoor air quality in a natural setting”, in which Eric K Soule, Sarah F Maloney, Tory R Spindle, Alyssa K Rudy, Marzena M Hiler, and Caroline O Cobb, of Thomas Eissenberg’s FDA-funded shop at Virginia Commonwealth University, reports on the researchers’ surreptitious observations at a vape convention. The research methods employed are extremely troubling to me and many others.

Before getting to that, I need to point out what was wrong with the content of the paper itself. (You can read more about several of these points from Clive Bates.) There are also some technical concerns about some of the details which I might circle back to, though they matter little.

This research was about indoor air quality. But the researchers measured nothing other than the quantity of small airborne droplets, with no assessment of the droplet chemistry (which would determine if they posed a chemical toxicant hazard). I cannot figure out anything that could have been learned from this exercise. All they could report was that there were a lot more droplets in the air during a vape convention than are normally in the air; since we know nothing about the implications of particular quantities of these, the quantification of that contained no information. But we could already safely conclude that there would be more droplets by knowing what an e-cigarette does and what happens at vape conventions. If not, it would be sufficient just ask anyone who has ever attended, or just look at photos where you can see the fog. There is no need to attend a vape event, much less surreptitiously attend an event, without disclosing that you were studying the people there, sneaking in a hidden monitoring instrument, which is what they did.

The researchers misleadingly characterized the droplets they observed as PM2.5 — which refers to particles (i.e., tiny solid bits) of size less than 2.5 microns, which are believed to cause health risk independent of their chemistry due to their size — and likened them to smoke (which is the dominant source of real PM2.5). As has been noted numerous times by quite a few commentators since Igor Burstyn and I pointed it out more than two years ago, e-cigarettes do not produce particles, they produce droplets (i.e., tiny liquid bits), which have entirely different health implications. There is no reason to believe that droplet size matters. Some instruments cannot tell the difference between particles and droplets, but this does not make them the same. I have explained this in easily-understood terms several times; I recommend this one, and see also here. (Also note that the health claims about actual PM2.5 are rather more tenuous than is usually portrayed in public health discussions, as I pointed out in the first of those three links, but that does not matter since this is not about actual PM2.5.)

The authors then go further and call for policy actions which could not possibly have been supported by any result this research could have produced. In particular, in spite of having generated no information about health risks, and in spite of having studied only an extreme environment which tells us nothing about normal settings, their conclusion statement demanded, “establishing policies that prohibit ECIG [sic] use anywhere combustible cigarette use is prohibited.”

Frankly, it is almost uninteresting to note that a public health article contains policy demands that in no way follow from the study; that is so close to universal that you might think it was mandatory. It is only slightly more interesting to observe that an article in Tobacco Control makes substantive claims that are not supported by the data. But this particular project starkly illustrated a whole different level of troubling behavior found in public health research, which is particularly common in tobacco control research: a disdain for the ethics of research conduct itself.

I was curious about the reaction to this research from their IRB (the human subjects ethics committee that all federally funded and university researchers, among others, must seek approval from when doing research on people). Ethically justifying any research that studies people surreptitiously, in a way that is likely to distress them when they find out it happened, is difficult. This is even more true when the potential for the study results to teach us anything is so low. A colleague of mine with extensive experience on IRBs noted that he would have wanted to see a lot of justification for why they were acting in secret and how they would handle particular contingencies associated with that secrecy, even if the study were valuable and he favored ultimately allowing it to go forward. Another colleague with experience in workplace air quality monitoring (the closest common analogy to this project) noted that it is considered unethical to do such research without the consent of both the owners of the premises and the workers on site for numerous reasons, some of which do not apply to the vape event, but some of which do.

Since there was no statement of IRB approval in the paper, I wrote to the researchers and asked if they had gotten such approval. I received a reply from someone associated with the research that no such approval had been sought, based on an appeal to this this flowchart from HHS, the parent department of the agencies that funded the research, about when such approval is required for research they fund. (I step through it below, so you don’t need to study the flowchart.)

I should pause here and note that, despite my serious concerns about the research conduct and the content of the paper, the fact that I got this response and that the correspondence continued speaks well of this research team. We would never expect that level of professionalism from most people in tobacco control. I realize that “they were willing to answer a question and further discuss the matter” is kind of faint praise in the face of the rest of what appears here, but it should not be overlooked. It suggests genuine concern for engagement and a confidence in the validity of what they did that you do not find among the majority of tobacco control “researchers” who hide in their echo chambers, knowing that their writings and actions cannot stand up to scrutiny. Needless to say, however, this does not change my assessment of what was done.

Resuming the analysis, I agree that they are probably right in their interpretation of this flowchart. But that is hardly sufficient to discharge all consideration of ethics. I responded to my correspondent, CCed to several other members of the research team who were included in the reply to me, with the following, which lays out the very serious concerns about the ethics of their conduct:


Dear xxxxx,

Thank you for the reply.

I trust that you know me well enough to have already guessed that my statement about having great interest in the paper is accurate, but does not imply any positive feelings about it. Great interest in this case means deep concern. Please allow me to recount my concerns to see if you are able to say anything that reduces them.

I should disclose that I will probably go public with my thoughts about this, such as those recounted here. I will not identify you or any other correspondent without permission, nor publish any of what you write (other than generically noting that I brought this up with correspondents associated with the research, and paraphrasing the response about the flowchart). However, I will try to give due consideration anything you offer in response to this. And I would be delighted if you were interested in taking anything you might want to offer public also.

From your response, I surmise that you followed that flowchart to “[Does the research involve intervention or interaction with the individuals?]” (no), and then to “[Is the information individually identifiable (i.e., the identity of the subject is or may readily be obtained by the investigator or associated with the information)?]” (no). This does indeed seem to be a reasonable interpretation of this flowchart, which then gets you to the point that IRB approval is not required by HHS rules. (I will suggest that this is possibly a borderline case: There is enough information in the research report that it is possible to identify the event with a reasonable degree of confidence, and there is sufficient public information about those events that it is possible to identify many individuals. But I also realize that such identification poses no threat to the subjects; the ethical concerns lie elsewhere.)

There is a “BUT” at that point in the flowchart. I realize that technically this branches to a box that is just about other laws [“Other Federal, State, or local laws and/or regulations may apply to the activity.”], but sometimes ethical behavior means doing better than what is merely required by law. Also, I will note that the box reached after the “no” answers says this is not research involving human subjects, but this is rather Orwellian. It is fine for them to say “this is not research that our rules say requires IRB approval”, but it is absurd for them to say that it is not research on human subjects, which was already stipulated at the second “yes” answer down the path [which reads: “Does the research involve obtaining information about living individuals?”].

Some of the study’s observations (counting the number of people vaping) were indisputably studies of human subjects, however trivial. But even the air quality measurement has a very “people-ish” feel to it, since it is studying an intimate artifact of the individuals who were also being studied.

A lot of practical ethics consists of actors (assuming they are not sociopaths) checking their gut and asking “does anything about this feel a little bit iffy?” If the answer is yes, regardless of the law (and I am not trying to argue that any action broke any law), it is generally time to seek some consultation with the stakeholders, or at the very least with serious neutral observers who are concerned about ethics and have no stake in the project itself. I find it difficult to imagine that no one had such a gut qualm, given that this research consisted basically of entering a community’s own gathering under false pretenses and treating the community and its individual members as if they were animals, studying them and their artifacts using concealed methods. I am sure the vape event would be considered a public venue in the eyes of the law, but for many of those present it was considered a private gathering of comrades, what might be called a “safe space” in current academia parlance. Consider, by analogy, a religious service. Again, obeying the law is not sufficient for behaving ethically.

Another possibility is that no one involved had any gut qualms. Rather they all just thought “hah! we sure figured out a clever way to study these people without their knowledge or consent.” If that is the case, then there seems to be an even bigger ethical problem, an active contempt for the subjects of the whole research agenda that arguably renders the whole research program unethical.

I trust you are aware of the uproar about this research, and about several other surreptitious studies of vapers, within the vaping community that is being studied. It is certainly easily predicted. This should give pause to you, your colleagues, the government, and everyone else involved in such projects. This alone suggests there is some serious ethical concern, whatever a flowchart might say. Imagine a similar situation with, say, research on the LGBT community or American muslims, which provoked an outcry from opinion leaders and the grassroots in that community about the methods employed. You can be sure that this concern would be taken seriously. But the tobacco control research community seems utterly unconcerned about this recurring problem. Now you might respond that LGBT people and American muslims are minorities that face threats, discrimination, and similar concerns due to intimate personal choices/characteristics. But the thing is, it is hard to make a case that this is any less true for vapers. They are under siege — both in terms of sociopolitical attacks and threats of real material losses — as a result of a their cherished intimate choice. Nonconsensual studying of a threatened minority group, as if they were mere animals, without so much as a consultation with stakeholder representatives (which apparently did not occur), is an ethically fraught action, whatever a flowchart might say.

When such studies of vulnerable minorities are done in order to try to improve the lot of that minority, they are often appreciated even though they are still on shaky ethical ground. But no such claim can be made here. Rather, it is quite clear that the goal of this research was to harm the interests of the individuals and minority group being studied.

The scientific results of the study were perfectly predictable; it would have been shocking if any other results were obtained. They were also utterly uninformative about health risks because nothing useful was measured(*). Moreover, it makes no sense to extrapolate this extreme circumstance, where everyone knows exactly what they are exposing themselves to, to other environments. Any one of these three points is sufficient to show that the research provided no support for the conclusions about imposing severe restrictions on vapers’ freedoms (e.g., banning vaping in all the private and public spaces where smoking is banned). It thus seems safe to assume those conclusions were written ahead of time, at least mentally. Put another way, the study of these people and their artifacts appears to have been conducted for the purpose of rationalizing a recommendation that harmed the research subjects and their community, since it served no other apparent purpose. Regardless of any flowchart, this introduces very serious ethical concerns. It is difficult to imagine there would not be an outcry and ethics investigation if a research team that was clearly intent on recommending restrictions on LGBT or muslim freedoms entered one those communities’ safest spaces and surreptitiously studied them.

(*As anyone following this topic as no excuse for not knowing, there is no reason to believe that the presence of droplets of whatever size pose a hazard absent any evidence of important levels chemical toxicants. Suspended droplets are not soot, they are fog, which is only a health hazard if you are trying to drive in it. As an aside, this brings up an entirely different ethical problem of making misleading health claims and drawing conclusions that do not follow from the scientific evidence, an even more so making policy recommendations that could not follow from any observations in the absence of a policy and political philosophy analysis. This is an almost ubiquitous problem in tobacco control research and throughout public health research. I will stay focused on the methods and related behavior of the researchers here, but it is worth noting that there is a bright-line ethical failure at different level also.)

There is a general feeling in the vaping community — presumably including those who were present at the studied event to the extent they are aware — that this research methodology was quite invasive. It produces a “how dare they?!” feeling. Of course a lot of actions that provoke “how dare they?!” feelings are perfectly legal. But that does not make them ethical, particularly for social scientists. (And make no mistake, any researcher pursuing a public health agenda is doing social science, even when they are mostly counting droplets. If the project is ultimately motivated by social engineering goals or behavioral recommendations, it is transparently disingenuous to claim “hey, I was just following the natural science results.” Natural science results only bear on policies or behaviors via a social science analysis, even if that analysis is never presented.)

When study subjects, upon learning about a study, strenuously object to having been studied, it is an ethical concern. This is, by itself, a good indication that the research project should have been subject to serious ethical review — or, better still, explicit consultation with the stakeholders. No flowchart can change that. The flowchart discharges legal mandates, but not ethical considerations.

I am in no way suggesting that an IRB filing would be necessary or sufficient for dealing with this. Indeed, in my experience, the medical IRBs that public health people usually use do more harm than good when ruling on non-medical research (e.g., objecting to utterly harmless voluntary surveys while allowing the provision of harmful disinformation and abusively deceptive actions to no apparent end). The role of an IRB is supposed to be to supplement the ethical judgment of researchers, not substitute for it. When researchers are seeking to harm those they are studying, and are enraging those they are studying, they ought to be able to figure out for themselves that there is an ethical problem. If they feel the need to go to lengths to deceive those they are studying, to conduct their research in secret and without seeking the courtesy of permission, because they anticipate they will get pushback if they are honest, they should be able to figure out there is an ethical problem. If they cannot, then there are much bigger problems afoot.

I welcome your response to any or all of this. Thank you for your attention.


I received a somewhat productive reply to this letter, though it did nothing to change my mind about any of my concerns. I will not disclose any details of that, per the promise in my letter. I told my correspondents that I would be publishing this, and suggested ways this could become a dialogue rather than a monologue.

(to be continued) [Update: Or not. If it is, it will not be by me, due to this.]


17 responses to “Serious ethical concerns about public health research conduct; the case of vape convention air quality measurement

  1. “Consider, by analogy, a religious service.”

    That got me thinking about the wide-spread use of incense in religious services. Imagine if non-religious researchers attended a Buddhist/Christian/Hindu service and secretly measured “air quality”. I may propose that project to my own IRB and see what sort of reaction I get.

    • Carl V Phillips

      Hmm. I wonder if studying IRBs is considered ethical? It would be quite interesting to propose the research that was actually done — or what you suggested as a close-enough substitute — just to see what they would do. Let me know if you seriously consider it.

      • We could call it “quality assurance” research… I’m the Vice Chair of our IRB and may propose this as a topic for discussion at the next IRB meeting. Such meetings are meant to have an ongoing educational component.

        • Carl V Phillips

          It is a bit tricky to ask an IRB to sign off on research that would consist of sending them research proposals that you are probably not actually doing to do to see how they react. It would have to be a multi-center study. I’m in.

  2. Had the researchers involved the subjects (members of the vaping community) in their project, they could have avoided making the mistake of assuming that hundreds of people vaping simultaneously in an indoor space produce the same volume of fog as a few people vaping at a table in a restaurant or by someone taking a puff of vapor occasionally as they worked at their desk. They might also have learned that the ratio of propylene glycol (PG) to Vegetable Glycerin (VG) in the liquid strongly affects the volume of visible fog produced. Furthermore, they might have learned a bit of sociology regarding this group: There are some former smokers who take joy in producing huge clouds of vapor. Many vape meets even conduct competitions for their benefit.

    On the other end of the spectrum, there are vapers who wish to produce as little fog as possible. In fact, they produce so little visible fog that they vape freely in public–at work, in meetings, in restaurants, and in planes, trains, and buses–and nobody even notices. But sadly, the researchers kept their plans and their project so secret that they failed to learn any of this. Thus, they proceeded to draw conclusions, and suggest policies based on those conclusions, that are unwarranted. Unfortunately, following their unwarranted policy recommendations could have far-reaching implications that work to keep people smoking.

  3. Roberto Sussman

    Ethical concerns in this case override the issue of the abuse of science. This is a sever breach of ethics, and would remain so even if the study had followed top notch methodology and the results were solid. This is clearly an issue of violation of Human Rights of the vapers who were secretly surveyed. Is there a legal instance in the USA (besides IRB) where this incident can be reported as a violation of Human Rights?

    • Carl V Phillips

      We do not have a court of human rights or any such in this country. Civil lawsuits come into play for violations when damages can be claimed, and there is criminal fraud and such. But for something like this, the only options really are to complain to the funding agencies (never likely to accomplish anything; that system is all crony corruption), complain to the university (which sometimes gets traction, though it is long odds in this case), or the court of public opinion.

      As for which ethical abuses are worse, I would say it is rather difficult to compare. That is why I separated them out and did not try to compare them; the are arguable incommensurate. On one hand, we tend to categorize them differently, sort of like speech acts (the paper and its claims) versus physical assault (the research conduct). It seems like a good metaphor: violations of the latter are fairly bright-line and clearly punishable, while violations of the former are usually protected from any punishment other than within its own space (counter-speech; retractions), and it is a wide grey area for what is unacceptable. This way of thinking conforms to your position.

      On the other hand, the magnitude of the damage done can cut strongly the other way. Someone can shake off the harm from a minor assault, even when it is clearly a criminal act, whereas announcing he is a child molester, even if it is an honest mistake, can do far more harm to him. So in this case the injury resulting to the participants from the research conduct is limited to feeling violated; I am most certainly not saying that is nothing (I am the one who brought this up after all) but we are not talking the Guatemala syphilis study. [Aside: Everyone has heard of Tuskegee. Not so well known was this similar earlier experiment, also conducted by U.S. government public health people.] But if the policy demand that does not follow from anything in the study is accepted based on the misleading reporting of the results, then vape conventions will be banned, and vaping in one’s favorite bar, which is far more material harm, and it stems from sowing disinformation. Of course does one more bit of disinformation in favor of bans really matter? So we could say that Eissenberg’s original epic fail (which he has never admitted was an error), widely touting that e-cigarettes do not deliver any nicotine, had far bigger negative consequences even though it was presumably based on a series of honest errors. (This contrasts with the PM2.5 claims, where the authors have no excuse for not knowing they are misleading people about the truth.)

      Then there are questions of motive: Is it worse to show a lack of respect for people’s dignity who are involuntary victims or worse to mislead people who are already useful idiots about the science to engineer policy actions? Again, we are looking at fairly incommensurate considerations. And there are questions of what is right versus what is normal: If almost all public health papers contain unsupportable and/or misleading claims, is someone even violating the rules when the do that?

      Anyway, tough to rank them. Much easier to identify each separately.

  4. Setting aside the ethical issues for now, if we look at the data collected, we can see an interesting result. Hotel Restaurant 1 had lower PM2.5 readings than Hotel Restaurant 2, despite it being smaller, located in the same building as the convention and having more people actively vaping in it (2 vs 0). If the researches really did care about assessing “indoor air quality in a natural setting”, these results would demonstrate that e-cigarettes provided a net increase in air quality.

    • Carl V Phillips

      Yeah, well, I think that is swamped by the two orders of magnitude increase in the measured droplets during the convention. But it does mean that the effect of a few people vaping is unimportant compared to the air pollution generated by restaurants themselves, which vary by quantities far more than would be produced by a few vapers. Funny how those calling for bans do not try to ban cooking etc. practices that create this bigger difference. Indeed, some of what is in the air of restaurants is actually PM2.5 and some of the droplets are actually hazardous to breathe.

  5. Exactly. But their specific choice to use the wording “in a natural setting” is confusing (well not really, I assume it is a deliberate attempt to mislead). I would consider a restaurant a “natural setting”, not a vape convention. If I were to set out to study the amount of radiation from a nuclear reactor in a natural setting, I wouldn’t focus on radiation readings inside the Fukushima or Chernobyl exclusion zones.

    • Carl V Phillips

      Oh, I totally agree. The wording was designed to mislead, along with trying to trick people into believing it is particulates (smoke). It is about as unnatural a setting as you can find.

      (Oh, and the radiation inside the Fukushima exclusion zone is not that great. I have seen estimates that far more harm was done by the evacuation than would have been done by just allowing the radiation exposure. Just because public health people are wrong about tobacco products does not mean they are not also wrong about this. :-)

  6. Pingback: Research Methods Questioned > Dustyhawk :: Broken Mirror

  7. Here’s my take on the PM 2.5 game in TobakkoNacht:

    “This is where these researchers and activists pull the EPA into regions where angels would fear to tread. They take their one measurement of PM 2.5 and simply call it “air pollution” – as though it were actually the same chemical mix of automotive and industrial nastiness that chokes the air of our cities for days on end. Of course the PM 2.5 in outdoor air, mainly produced by automotive exhaust and general smog, has a very different chemical makeup than the PM 2.5 produced by the quiet burning of a few leaves wrapped in a thin sheath of paper. Equating the two is almost like saying that a teaspoon of sugar crystals is as deadly as a teaspoon of arsenic crystals because the crystals are the same size. But such a fine distinction can be overlooked if one simply wants to make a comparison designed to frighten non-scientists.”

    It was true for secondary smoke, and it’s clearly even MORE true for vaping vapor! It’s a con game, pure ‘n simple, and they get away with it simply because they have the money to control the microphone.


    • Carl V Phillips

      It is actually worse than that. It is possible to consider the whole of the epidemiology about PM2.5 and conclude that maybe there is no effect from it at all (not that there is no effect from any kind of air pollution, of course, but that PM2.5 itself does not matter). It comes down to trying to tease out any effect on health outcomes from episodes of high particulate air pollution, where the confounding from weather and chemistry is orders of magnitude bigger than the signal of interest, and then also parsing that data to decide that PM2.5 quantity is the best predictor of this maybe-extant elevated risk from particulate matter. There is no replication or triangulation of this result — it all depends on the statistical manipulation of one kind of data. It is only then that we get to your point about extrapolating from this tenuous claim about outdoor industrial air pollution to other types of PM2.5 or even DM2.5.

      Frightening non-scientists. Con game. Control of the microphone. <- Spot on.

  8. Well done Carl from a non-scientist, the world needs more people like you :-)

    • “Well done Carl from a non-scientist, the world needs more people like you :-)”

      Martin, I FULLY agree… although your note prompted an instantaneous scary image in my brain of walking around in a world of 7 billion people who all looked and sounded exactly like Carl….



  9. Pingback: Real implications of the RSPH “sting” of ecig vendors | Anti-THR Lies and related topics

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s