Scientific claims in the FDA deeming regulation (part 1 of ???)

by Carl V Phillips

[Update — this series continues with:

• Part 2 (mostly about flavoring, p.19-20)
• Part 3 (mostly about usage patterns, p.21-22)
• Part 4 (mostly about claims of what good the regulations would do p.24-26)

Some closely related posts include:

• The three-part series about a recent collection of papers by FDA that starts here
• Analysis of the naivety on this topic displayed in a letter by FDA Commissioner Hamburg here
• My comments on behalf of CASAA about the inappropriateness of the economic impact statement in the context of the Paperwork Reduction Act, linked from here

]

As presumably every regular reader knows, the FDA CTP has published their draft “deeming” regulations that will bring e-cigarettes, dissolvable nicotine products, and some hypothetical other THR products (as well as some combustible products) under their jurisdiction.  You can find more about that, including our ongoing analysis of our overall assessment of the regulations, at the CASAA blog.

This post begins a series of unknown length that will analyze the scientific claims being made in the draft regulation[Update: That link is now dead, so here is the file; I prefer it because I was already using the page numbering, and it is much easier to read], the associated economic impact statement, and any other partially-scientific documents that are produced. While there is substantial unhappiness about the proposed regulations, and protests that it is just not the right thing to do, it is unlikely that these will have a large influence on FDA’s final decision or any adjudication of it.  Third party reviewers of FDA’s actions (i.e., courts) will probably be much more concerned with details of whether the science supports the claims and whether the claims support the policies.  This series focuses on those questions and attempts to avoid any normative analysis (other than about epistemology).

In a recent three–part series, I assessed the FDA CTP’s understanding of the science, and of scientific reasoning in general, as evidenced by a series of papers they recently published.  I observed that they seemed to be acting based on the disastrous epistemic strategy of assuming that just because someone wrote something, it must be right.  More specifically, the authors of those papers demonstrated clear distress at the prospect of having to deal with a world in which all knowledge is not collected into a small number of indexed papers (i.e., the real world of e-cigarette science).  They responded by ignoring that world and hinting that they would support making the world simplify down to their epistemic capabilities, rather than trying to expand their capacity to deal with reality.

The approach to science in the draft regulation, therefore, comes as no surprise.  It demonstrates an apparent ignorance of the enormous knowledge base that exists outside of ivory-tower journals.  FDA is not a court of law, which can conveniently limit itself to knowledge sources that are “officially on the record”.  They are obliged to consider all available knowledge when making scientific claims.  But they have not done so.

The possible explanations for why FDA ignored most of the available knowledge and were uncritical about what they did believe are:

1. Genuine freshman-level naivety, with authors actually believing that anything that appears in a journal must be correct.

2. Similar naive belief that all of human knowledge, even in the modern information age, is contained in bound volumes.

3. Desire for 2 to be true, because it makes life much easier, and an intention to change reality to fit this desire rather than trying to accommodate reality.

4. Cynical attempt to make up “rules of evidence” that tends to favor the outcome the regulators prefer.

While it is conceivable that some staff researchers at CTP actually believe 1 and 2 — putting them in the category of “dangerous but innocent” — it is obvious that the institution as a whole does not have such childlike (or childish?) views.  They do seem to favor 3.  But it is apparent that 4 is a strong presence in regulation.  As will become apparent from this analysis, the naive interpretation of the data follows a definite pattern of blindly accepting only those claims that tend to support aggressive regulation, while ignoring evidence (from whatever source) that is reassuring or implying doubt about it — often far beyond what is reasonable.  Put another way, the scientific claims were presented more as an indictment than as an attempt to understand or provide perspective.

This series assumes understanding beyond a childlike faith in 1 or 2, does not accept 3 or some other naive “rules of evidence”, and, of course, considers 4 unacceptable.  That is, this is written from the perspective of truth-seeking and the assumption that this should have been the goal of the scientific content of the FDA documents.

With that introduction, I will anchor what is to come with some of the highly-caveated statements that appear in the draft regulations.  Page numbers are from this version of the draft regulations.  It will become apparent that few of the claims in the document are made with such extreme epistemic modesty, and that there is the obvious pattern described above.

p.42:

…some researchers have stated that substituting e-cigarettes for tobacco cigarettes “may substantially reduce exposure to tobacco-specific toxicants” (Ref. 47 [Goniewicz et al. 2013]).

These particular authors stated this.  But citing a single source, in which they actually understated the claim (“may”??) tends to suggest to the reader that this is a bold rare position.  More accurate would be “every qualified commentator who has ever addressed the point has stated…” or “all of the many studies of exposure show…”.  Better still, of course, would be the simple statement of fact that this is true.  Notice that this obfuscating and hedging is not even about the net health effects; it is about exposure to tobacco-specific toxicants.

Another such example is the statement:

Some have advanced views that certain new tobacco products that are noncombustible (such as e-cigarettes) may be less hazardous than combustible products….

This exact statement appears six times in the document.  While not as extreme a downplaying of overwhelming scientific evidence as the previous statement, it is still extreme.  Many of those anonymous view-advancers have, of course, not merely “suggested” it is merely “less hazardous”, but have estimated and emphatically argued that the hazards to be two orders of magnitude less than those of smoking (i.e., in the order of 1/100th).  Moreover, I am aware of no case of anyone presenting any analysis that lends plausibility to the contrary claim, namely that noncombustibles are no less hazardous.  (Of course, some people have asserted without analysis or support that it is true, but the same can be said for basically any claim that can be made about anything.)

These statements, which imply that keystone pieces of knowledge about e-cigarettes and other smoke-free tobacco products are in doubt, have several possible explanations.  The knee-jerk interpretation of them — that the authors are trying to mislead about the benefits of smoke-free alternatives — is popular in the early commentaries about this.  But to legitimately reach that conclusion it is necessary to compare the epistemic attitude toward other claims.  Specifically, it would be possible for the FDA to treat all scientific claims with the same level of epistemic modesty.

That, however, is not what was done, as will be evident as this series continues.