Serious ethical concerns about public health research conduct; the case of vape convention air quality measurement

by Carl V Phillips

A recent paper in Tobacco Control (official version; unpaywalled version), “Electronic cigarette use and indoor air quality in a natural setting”, in which Eric K Soule, Sarah F Maloney, Tory R Spindle, Alyssa K Rudy, Marzena M Hiler, and Caroline O Cobb, of Thomas Eissenberg’s FDA-funded shop at Virginia Commonwealth University, reports on the researchers’ surreptitious observations at a vape convention. The research methods employed are extremely troubling to me and many others.

Before getting to that, I need to point out what was wrong with the content of the paper itself. (You can read more about several of these points from Clive Bates.) There are also some technical concerns about some of the details which I might circle back to, though they matter little.

This research was about indoor air quality. But the researchers measured nothing other than the quantity of small airborne droplets, with no assessment of the droplet chemistry (which would determine if they posed a chemical toxicant hazard). I cannot figure out anything that could have been learned from this exercise. All they could report was that there were a lot more droplets in the air during a vape convention than are normally in the air; since we know nothing about the implications of particular quantities of these, the quantification of that contained no information. But we could already safely conclude that there would be more droplets by knowing what an e-cigarette does and what happens at vape conventions. If not, it would be sufficient just ask anyone who has ever attended, or just look at photos where you can see the fog. There is no need to attend a vape event, much less surreptitiously attend an event, without disclosing that you were studying the people there, sneaking in a hidden monitoring instrument, which is what they did.

The researchers misleadingly characterized the droplets they observed as PM2.5 — which refers to particles (i.e., tiny solid bits) of size less than 2.5 microns, which are believed to cause health risk independent of their chemistry due to their size — and likened them to smoke (which is the dominant source of real PM2.5). As has been noted numerous times by quite a few commentators since Igor Burstyn and I pointed it out more than two years ago, e-cigarettes do not produce particles, they produce droplets (i.e., tiny liquid bits), which have entirely different health implications. There is no reason to believe that droplet size matters. Some instruments cannot tell the difference between particles and droplets, but this does not make them the same. I have explained this in easily-understood terms several times; I recommend this one, and see also here. (Also note that the health claims about actual PM2.5 are rather more tenuous than is usually portrayed in public health discussions, as I pointed out in the first of those three links, but that does not matter since this is not about actual PM2.5.)

The authors then go further and call for policy actions which could not possibly have been supported by any result this research could have produced. In particular, in spite of having generated no information about health risks, and in spite of having studied only an extreme environment which tells us nothing about normal settings, their conclusion statement demanded, “establishing policies that prohibit ECIG [sic] use anywhere combustible cigarette use is prohibited.”

Frankly, it is almost uninteresting to note that a public health article contains policy demands that in no way follow from the study; that is so close to universal that you might think it was mandatory. It is only slightly more interesting to observe that an article in Tobacco Control makes substantive claims that are not supported by the data. But this particular project starkly illustrated a whole different level of troubling behavior found in public health research, which is particularly common in tobacco control research: a disdain for the ethics of research conduct itself.

I was curious about the reaction to this research from their IRB (the human subjects ethics committee that all federally funded and university researchers, among others, must seek approval from when doing research on people). Ethically justifying any research that studies people surreptitiously, in a way that is likely to distress them when they find out it happened, is difficult. This is even more true when the potential for the study results to teach us anything is so low. A colleague of mine with extensive experience on IRBs noted that he would have wanted to see a lot of justification for why they were acting in secret and how they would handle particular contingencies associated with that secrecy, even if the study were valuable and he favored ultimately allowing it to go forward. Another colleague with experience in workplace air quality monitoring (the closest common analogy to this project) noted that it is considered unethical to do such research without the consent of both the owners of the premises and the workers on site for numerous reasons, some of which do not apply to the vape event, but some of which do.

Since there was no statement of IRB approval in the paper, I wrote to the researchers and asked if they had gotten such approval. I received a reply from someone associated with the research that no such approval had been sought, based on an appeal to this this flowchart from HHS, the parent department of the agencies that funded the research, about when such approval is required for research they fund. (I step through it below, so you don’t need to study the flowchart.)

I should pause here and note that, despite my serious concerns about the research conduct and the content of the paper, the fact that I got this response and that the correspondence continued speaks well of this research team. We would never expect that level of professionalism from most people in tobacco control. I realize that “they were willing to answer a question and further discuss the matter” is kind of faint praise in the face of the rest of what appears here, but it should not be overlooked. It suggests genuine concern for engagement and a confidence in the validity of what they did that you do not find among the majority of tobacco control “researchers” who hide in their echo chambers, knowing that their writings and actions cannot stand up to scrutiny. Needless to say, however, this does not change my assessment of what was done.

Resuming the analysis, I agree that they are probably right in their interpretation of this flowchart. But that is hardly sufficient to discharge all consideration of ethics. I responded to my correspondent, CCed to several other members of the research team who were included in the reply to me, with the following, which lays out the very serious concerns about the ethics of their conduct:

———-

Dear xxxxx,

Thank you for the reply.

I trust that you know me well enough to have already guessed that my statement about having great interest in the paper is accurate, but does not imply any positive feelings about it. Great interest in this case means deep concern. Please allow me to recount my concerns to see if you are able to say anything that reduces them.

I should disclose that I will probably go public with my thoughts about this, such as those recounted here. I will not identify you or any other correspondent without permission, nor publish any of what you write (other than generically noting that I brought this up with correspondents associated with the research, and paraphrasing the response about the flowchart). However, I will try to give due consideration anything you offer in response to this. And I would be delighted if you were interested in taking anything you might want to offer public also.

From your response, I surmise that you followed that flowchart to “[Does the research involve intervention or interaction with the individuals?]” (no), and then to “[Is the information individually identifiable (i.e., the identity of the subject is or may readily be obtained by the investigator or associated with the information)?]” (no). This does indeed seem to be a reasonable interpretation of this flowchart, which then gets you to the point that IRB approval is not required by HHS rules. (I will suggest that this is possibly a borderline case: There is enough information in the research report that it is possible to identify the event with a reasonable degree of confidence, and there is sufficient public information about those events that it is possible to identify many individuals. But I also realize that such identification poses no threat to the subjects; the ethical concerns lie elsewhere.)

There is a “BUT” at that point in the flowchart. I realize that technically this branches to a box that is just about other laws [“Other Federal, State, or local laws and/or regulations may apply to the activity.”], but sometimes ethical behavior means doing better than what is merely required by law. Also, I will note that the box reached after the “no” answers says this is not research involving human subjects, but this is rather Orwellian. It is fine for them to say “this is not research that our rules say requires IRB approval”, but it is absurd for them to say that it is not research on human subjects, which was already stipulated at the second “yes” answer down the path [which reads: “Does the research involve obtaining information about living individuals?”].

Some of the study’s observations (counting the number of people vaping) were indisputably studies of human subjects, however trivial. But even the air quality measurement has a very “people-ish” feel to it, since it is studying an intimate artifact of the individuals who were also being studied.

A lot of practical ethics consists of actors (assuming they are not sociopaths) checking their gut and asking “does anything about this feel a little bit iffy?” If the answer is yes, regardless of the law (and I am not trying to argue that any action broke any law), it is generally time to seek some consultation with the stakeholders, or at the very least with serious neutral observers who are concerned about ethics and have no stake in the project itself. I find it difficult to imagine that no one had such a gut qualm, given that this research consisted basically of entering a community’s own gathering under false pretenses and treating the community and its individual members as if they were animals, studying them and their artifacts using concealed methods. I am sure the vape event would be considered a public venue in the eyes of the law, but for many of those present it was considered a private gathering of comrades, what might be called a “safe space” in current academia parlance. Consider, by analogy, a religious service. Again, obeying the law is not sufficient for behaving ethically.

Another possibility is that no one involved had any gut qualms. Rather they all just thought “hah! we sure figured out a clever way to study these people without their knowledge or consent.” If that is the case, then there seems to be an even bigger ethical problem, an active contempt for the subjects of the whole research agenda that arguably renders the whole research program unethical.

I trust you are aware of the uproar about this research, and about several other surreptitious studies of vapers, within the vaping community that is being studied. It is certainly easily predicted. This should give pause to you, your colleagues, the government, and everyone else involved in such projects. This alone suggests there is some serious ethical concern, whatever a flowchart might say. Imagine a similar situation with, say, research on the LGBT community or American muslims, which provoked an outcry from opinion leaders and the grassroots in that community about the methods employed. You can be sure that this concern would be taken seriously. But the tobacco control research community seems utterly unconcerned about this recurring problem. Now you might respond that LGBT people and American muslims are minorities that face threats, discrimination, and similar concerns due to intimate personal choices/characteristics. But the thing is, it is hard to make a case that this is any less true for vapers. They are under siege — both in terms of sociopolitical attacks and threats of real material losses — as a result of a their cherished intimate choice. Nonconsensual studying of a threatened minority group, as if they were mere animals, without so much as a consultation with stakeholder representatives (which apparently did not occur), is an ethically fraught action, whatever a flowchart might say.

When such studies of vulnerable minorities are done in order to try to improve the lot of that minority, they are often appreciated even though they are still on shaky ethical ground. But no such claim can be made here. Rather, it is quite clear that the goal of this research was to harm the interests of the individuals and minority group being studied.

The scientific results of the study were perfectly predictable; it would have been shocking if any other results were obtained. They were also utterly uninformative about health risks because nothing useful was measured(*). Moreover, it makes no sense to extrapolate this extreme circumstance, where everyone knows exactly what they are exposing themselves to, to other environments. Any one of these three points is sufficient to show that the research provided no support for the conclusions about imposing severe restrictions on vapers’ freedoms (e.g., banning vaping in all the private and public spaces where smoking is banned). It thus seems safe to assume those conclusions were written ahead of time, at least mentally. Put another way, the study of these people and their artifacts appears to have been conducted for the purpose of rationalizing a recommendation that harmed the research subjects and their community, since it served no other apparent purpose. Regardless of any flowchart, this introduces very serious ethical concerns. It is difficult to imagine there would not be an outcry and ethics investigation if a research team that was clearly intent on recommending restrictions on LGBT or muslim freedoms entered one those communities’ safest spaces and surreptitiously studied them.

(*As anyone following this topic as no excuse for not knowing, there is no reason to believe that the presence of droplets of whatever size pose a hazard absent any evidence of important levels chemical toxicants. Suspended droplets are not soot, they are fog, which is only a health hazard if you are trying to drive in it. As an aside, this brings up an entirely different ethical problem of making misleading health claims and drawing conclusions that do not follow from the scientific evidence, an even more so making policy recommendations that could not follow from any observations in the absence of a policy and political philosophy analysis. This is an almost ubiquitous problem in tobacco control research and throughout public health research. I will stay focused on the methods and related behavior of the researchers here, but it is worth noting that there is a bright-line ethical failure at different level also.)

There is a general feeling in the vaping community — presumably including those who were present at the studied event to the extent they are aware — that this research methodology was quite invasive. It produces a “how dare they?!” feeling. Of course a lot of actions that provoke “how dare they?!” feelings are perfectly legal. But that does not make them ethical, particularly for social scientists. (And make no mistake, any researcher pursuing a public health agenda is doing social science, even when they are mostly counting droplets. If the project is ultimately motivated by social engineering goals or behavioral recommendations, it is transparently disingenuous to claim “hey, I was just following the natural science results.” Natural science results only bear on policies or behaviors via a social science analysis, even if that analysis is never presented.)

When study subjects, upon learning about a study, strenuously object to having been studied, it is an ethical concern. This is, by itself, a good indication that the research project should have been subject to serious ethical review — or, better still, explicit consultation with the stakeholders. No flowchart can change that. The flowchart discharges legal mandates, but not ethical considerations.

I am in no way suggesting that an IRB filing would be necessary or sufficient for dealing with this. Indeed, in my experience, the medical IRBs that public health people usually use do more harm than good when ruling on non-medical research (e.g., objecting to utterly harmless voluntary surveys while allowing the provision of harmful disinformation and abusively deceptive actions to no apparent end). The role of an IRB is supposed to be to supplement the ethical judgment of researchers, not substitute for it. When researchers are seeking to harm those they are studying, and are enraging those they are studying, they ought to be able to figure out for themselves that there is an ethical problem. If they feel the need to go to lengths to deceive those they are studying, to conduct their research in secret and without seeking the courtesy of permission, because they anticipate they will get pushback if they are honest, they should be able to figure out there is an ethical problem. If they cannot, then there are much bigger problems afoot.

I welcome your response to any or all of this. Thank you for your attention.

——–

I received a somewhat productive reply to this letter, though it did nothing to change my mind about any of my concerns. I will not disclose any details of that, per the promise in my letter. I told my correspondents that I would be publishing this, and suggested ways this could become a dialogue rather than a monologue.

(to be continued) [Update: Or not. If it is, it will not be by me, due to this.]