Tag Archives: biomarkers

Why we absolutely positively must avoid pro-THR lies (and honest errors that look like lies)

Posted on 9 September 2012 by | 21 comments

posted by Carl V Phillips

This blog is only two weeks old but it is not too soon for an aside that addresses a critical tangential point, the accuracy and credibility of pro-THR claims.

Almost all anti-THR discourse is dominated by lies.  It has to be, because the truth has such an obvious pro-THR bias.  Most pro-THR claims are solidly based on evidence and honest communication of it, as well as established ethical norms and the very popular political view that people should have freedom to make their most intimate choices without interference by those in power.  Thus it is not surprising that, with the exception of a few unscrupulous merchants and Chinese spammers writing about e-cigarettes (scourges which it is impossible to avoid or eliminate when there is free communication), pro-THR discussions are almost never anchored in misinterpretations of the evidence.

Almost never.

And there is the problem.  The example to hand is the recent widely repeated claim that a study in Greece showed that “e-cigarettes do not damage the heart“.  The study showed nothing remotely that general.  It showed that under in particular circumstances, the brief use of an e-cigarette does not produce measurable acute (immediate) changes in a few particular biomarkers of cardiac functioning.  Is this good news for e-cigarette users and THR?  Of course.  It is the best news that could come from that study, which was only looking at the immediate effect of one vaping session on the particular short-term biomarkers that were measured.  But I trust that it is obvious why this is not sufficient to justify the headline that was repeated by many e-cigarette advocates.  The study obviously could not address what matters for health outcomes, the long-term effects of long-term use (the natural interpretation of the headline of that press release).  Moreover, it measured only a few of the many possible short-term effects.  It is a result that should be added in to the body of technical knowledge that is useful for experts who are compiling all existing information into conclusions; it should not be interpreted as having clear practical implications and should not have been touted to the general public as saying there is no risk of damage.

Imagine that an anti-harm-reduction “researcher” did a lab study in which they asked a few smokers questions that reflect their desire to have a smoke later that day, asking before and after they used an e-cigarette, and found that the measured desire increased afterward.  Headline: “E-cigarette use increases smoking”.  Of course, the measure was very limited and artificial, and tells us very little about long-term effects.  In that, it is very much like the e-cigarette cardiac study.  In fairness to the latter, the cardiac study could have been legitimately reported in useful and accurate terms like, “certain cardiac functions that show acute negative effects with smoking do not show the same effects with e-cigarettes”, whereas the imaginary anti study has no apparent legitimate interpretation.  But the accuracy of the more general declaration is similar in the two cases.

Of course, you do not have to imagine.  Readers of this blog will undoubtedly have noticed that this heart study sounds remarkably like the lung effects study that we wrote about for the last three days, which was press released at about the same time.  Both studies looked at short-term biomarkers of acute effect that could not possibly tell us anything about real health effects (unless the results were catastrophically bad, which we knew in advance that they would not be).  Both studies could add a bit of useful technical information to what we know about e-cigarettes.  Both were from Greece, though I think that is probably not relevant to anything.  And, unfortunately, both were reported via press releases that absurdly overstated their implications.

There were important contrasts:  The actual methods and useful results of the cardiac study were contained in the press release (to the extent possible in a few score of words), as opposed to the obviously misleading non-presentation of the science in the Christina Gratziou lung study.  The cardiac study author’s stated conclusions in the text of the press release were quite modest (one might say even more timid than the facts support) about whether THR was a good idea, and did not lean too much toward claiming that his results were particularly important, whereas Gratziou was basically shouting “my results prove that you might as well smoke!”   But still, the false claims in the headlines were remarkably similar, as was the willingness of news sources with the particular bias to uncritically repeat the claim in the headline.

Anti-THR activists cannot afford to not lie, because the truth is not on their side.

THR supporters cannot afford to make the mistake of blindly repeating overzealous pro-THR claims.

But, wait, a common response goes, you just pointed out that anti-THR people do exactly that, pretty much all the time.  They do a lot worse too: full-on campaigns of lies and research misconduct.  They lie constantly.

But that does not justify us doing it.  In addition, and probably more important in the minds of most THR supporters, it does not make it tactically sensible.  We have to police our own claims much more carefully, for the following reasons:

1.  We do not want to be like them.  Right?  Nuff said.

1.5  And if we are like them then ,well, live by the favorable tiny little biomarker study of limited scientific value, die by the much larger number of unfavorable little biomarker studies of limited scientific value.  If that cardiac study is evidence that e-cigarettes do not harm the heart, then the lung study is evidence that they do harm the lungs.

2.  One false claim from our side — even if it is basically consistent with all the evidence (as is the claim that we do not think there is any major cardiac risk), and even if it is presented with the best of intentions, as an honest mistake rather than a crafted lie — will be used to excuse every single lie from the other side.  We do not need to make false claims, but they do, so why give them the gift of being able to respond “well perhaps our claim of …, based on one finding from one study, overstates the case a bit; but the THR people claim that one little study of a few biomarkers showed that e-cigarettes pose absolutely no threat of cardiovascular disease, so they are really worse.”  (Note, by the way, that this is true: claiming “no damage” based on any single study is more inaccurate than claiming “damage” based on a single study, because no little study can provide much evidence for the universal absence of a something, but it can show its existence.)

It does not matter if their claim in the “…” is more egregious.  It does not matter that they have a legion of paid operatives who are intentionally trying to mislead, while we are running volunteer operations and trying to make the best sense we can of the science with limited resources, and so occasionally err by accident.  And most of all, it does not matter that they tell a thousand lies that can fill in the “…”; every last one of those thousand will be declared to be justified by just a single example that can be presented on the other side.

3.  It is not just the anti-THR people who will do this.  There is a silly notion in the American press to seek what they call “balance” in anything they report, and this has spilled over into a lot of public Anglophone discourse (I cannot say for sure whether it is so common beyond that).  A slight caricature of this is Krugman’s observation that if the mainstream press was running a story on the Earth’s shape, they would find one person who says it is flat and run the headline “Shape of the Earth: views differ”.  Their silly urge to “balance” is even stronger when pointing out that someone is lying; no matter how blatant a lie is, the mainstream media’s reporting of this fact is almost always accompanied by a hunt for some error from “the other side” to “balance” the information.  It just would not be [insert affected accent and fan self dramatically like a character from Gone With The Wind] proper to seem to be biased against a liar who represents an established and powerful institution, after all.

So, when we get mainstream discourse that points out that anti-THR claims are based on lies — which is what is something we really need to generate — it will most likely juxtapose that with pro-THR “lies”.  If all they can find are a few shady merchants making absurd claims, it will not look so bad (so long as they do not lie and suggest that everyone on our side is making such statements).  But “does not damage the heart”, widely reported by non-merchant THR supporters, is just the perfect balance to show that “both sides are making exaggerated claims.”

It may not be fair, but we have to deal with the way things are.

3.5  But it gets even worse and more unfair than that:  The mainstream discourse is actually going to favor anti-THR lies over our pro-THR truth, since it almost always defers to lies from powerful institutions.  So it will take a very long list of very obvious lies to hurt their credibility as much as one clear exaggeration hurts ours.  The “public health” movement is a small-minority special interest group that has abandoned society’s norms of ethics and scientific conduct, but it is a rich and powerful special interest group and used to be respectable.  The mainstream media, as well as most of the liberal-leaning new media and — most important — the average person on the street, still treat “public health” as if they are honest and public-spirited.  The “public health” people know this, of course, which is why they do not hesitate to lie.

But more importantly for present purposes, it is also why they can get away with misleading fixation on unrepresentative individual actions of opponents that distract from the real truths of the matter (e.g., fixating on one unfortunately-phrased ad for smokeless tobacco from the 1970s as a reason for condemning all smokeless tobacco forever, or on the claims of a few fly-by-night e-cigarette merchants, or that whole “tobacco industry documents” obsession).  The public is still ready to believe any bad thing that is claimed about THR products or those who support them (who will be labeled “the evil big tobacco industry”, of course).  We cannot give them the openings.

Again, it is not “fair”, but the total dishonesty and unearned credibility of the Drug Warriors-types and other anti-harm-reduction activists in “public health” forces us to be even more scrupulously honest than might be needed otherwise.

4.  If we start making every claim that seems to be pro-e-cigarette, treating anything that is pro as right and attacking everything that is perceived as anti as wrong, we will learn a lot less.  Learning is good.  Perhaps the marketers want everyone to believe that there is only good news, but those of us who care about more than pumping sales want to find the bad news when it exists.  This is not just because of Point 1., and the potential loss of credibility if an overblown claim turns out to be wrong.  It is also because consumers are better off if they know the risks, and only if we know about problems we can try to fix them.  E-cigarette users should be the absolute last ones to want there to be claims of “no risk” when we do not actually know that.

This issue reminds me, to a very disturbing degree, of work I did in the 1990s, trying (unsuccessfully) to persuade vegetarian advocates to stick to the true and scientifically valid arguments that supported their position, rather than glomming onto every blatantly irrelevant or junk science claim that seems to point in the right direction.  The bad information basically drove out the good (both the good information and most of the good people).  The situations are quite different in many ways — e.g., there are relatively few non-experts writing books and junk science websites about THR (yet!), while such people dominated (and still dominate) the discourse about vegetarianism.  But there are also some clear similarities.

There are honest and accurate pro-vegetarian arguments that would appeal to the substantial part of the population that is concerned about animals or the environment, and to a lesser extent their health, and yet vegetarianism was widely perceived mostly as a nutty cult.  The true claims that might have persuaded people were (and still are) hopelessly lost amongst the easily-debunked false claims which were more prominent.  So, almost two decades later and even as lots of people have stopped eating much meat, vegetarianism’s negative image has hardly changed.  For example, it has been mentioned to me that my credibility in the work I do to promote animal well-being is enhanced by the fact that I am no longer vegetarian (I am pretty close, but not entirely).  I believe that had influential pro-vegetarian advocates stopped acting like a bunch of unscrupulous marketers and doe-eyed cultists back then, things would look very different now.

THR is not nearly so vulnerable to being marginalized (a lot more people practice THR than practice vegetarianism in the West), but there is a powerful opposition that wants to marginalize it.  If we do not police ourselves and discourage THR advocates from embracing any claim that seems to support the cause, we run the risk of become an insular fringe that opponents can just ridicule and thereby ignore.  Quick: name an organization that advocates vegetarianism that you would turn to for credible information about the topic?  I didn’t think many of you would have an answer, though if you have ever thought about the subject, you probably thought of several that you would not trust.  There are some good resources out there, but you pretty much have to already be an expert to sort them out from the marketing and cultish propaganda.  Do we want to risk THR and e-cigarette advocacy looking like that in a few years?

21 Comments

Posted in Lies

Tagged ,

Gratziou ERS press release – where is the science?

Posted on 8 September 2012 by | 4 comments

posted by Carl V Phillips

In the previous posts we pointed out how Christina Gratziou’s anti-THR politics-disguised-as-science could be recognized as lies (even by someone who did not understand the science) and explained why her concluding about real disease outcomes from a short-term biomarker study were inherently flawed.  This did not even address the what she reported about the research, which I will do today.

Here is the content (all of it):

The study included 8 people who had never smoked and 24 smokers, 11 with normal lung function and 13 people with either chronic obstructive pulmonary disease (COPD) or asthma.

Each person used an electronic cigarette for 10 minutes. The researchers then measured their airway resistance using a number of tests, including a spirometry test.

The results showed that for all people included in the study, the e-cigarette caused an immediate increase in airway resistance, lasting for 10 minutes. In healthy subjects (never smokers) there was a statistically significant increase in airway resistance from a mean average of 182% to 206%.

In smokers with normal spirometry there was a statistically significant increase from a mean average of 176% to 220%. In COPD and asthma patients the use of one e-cigarette seemed to have no immediate effect to airway resistance.

So, what does this mean (other than the obvious conclusion from the first paragraph: that this was a ridiculously small study to be drawing any conclusions from)?  One good answer is “nothing”.  That is, this was sent out to newspaper reporters who passed it on to the general public, and what it meant to them was absolutely nothing.  So those readers skipped past the science and read only the lying assertions that were tacked onto it, assuming that those complicated bits supported the assertions; the lab results were simply there to camouflage the fact that the press release was little more than a political statement.

How can there be any honest motive for sending this information to the general public?  I am in well into the 99th percentile of the population in terms of ability to understand this science — easily more like 99.99th  — and I could not make much sense of it.  Fortunately, I know people in the 99.9999th, so I could ask a few question.

Also, before I started writing about this I asked a few questions of Gratziou, via the press contact listed on her press release.  The contact assured me that she would get the questions to Gratziou and I would hear back.  I did not hear back.  Real scientists generally like to explain their work to other scientists; liars, on the other hand, do not like follow-up questions.  (In case it is not obvious, my questions were simple and technical (and thus in no way impolite), and only about the scientific bits.)

What did the researchers do and find?  That is not clear even to the expert readers.  “Spirometry” is not an adequate description of the measurement method.  It is also not clear even to experts what the units of measure mean.  Absolute measures would have been more useful, especially lacking any definition for the “100%” that the percentages are based on.  It could be that the first of each pair of percentages was the baseline and the second was the result after the exposure, or it could be that the baseline was called 100% and these are the ranges of the final results.  The former is the natural way to read the sentences, but would mean that the smokers had clearer airways to start with than the nonsmokers, which seems unlikely.

So even for experts, the information about results presented to the world was meaningless.  That does not, however, mean there is not useful information about what was wrong with what they did.

As noted in the previous post, the biggest limitation (even worse than the size of the study) is that it was based on just the immediate reaction to a first exposure.  Moreover, the subjects were exclusively people who were not used to vaping, and were given the exposure under contrived and medicalized (and thus disconcerting) circumstances.  Would an experienced vaper get the same effect?  There is reason to believe not, since vapers quite often report that they had some airway irritation (which would cause the observed effect) when they first started vaping, but it seemed to disappear.  This would have been useful to report, though it is not clear whether Gratziou failed to report it because she was hiding it or because her knowledge is so limited that she does not even know it.  It is similarly not clear whether these lab-technicians-posing-as-scientists intentionally chose only subjects that would produce uninformative results, or did so out of ignorance.

That “lab technicians” remark is not a slight against techs — we would not get anywhere in lab and clinical sciences without good techs.  But scientists need to think beyond, “I have a machine, I have a person, let’s see what happens.”  In particular, in a case like this, they need some calibration (aka “controls”).  A scientist knows (contrary to popular belief) that it is not always necessary to have a null exposure control group or trial (“placebo control”), but sometimes, like in this case, the results will be impossible to make sense of without it.

Airway resistance measurement is finicky, sensitive to exactly what is being done and subjects’ behavior, as well as anxiety and other factors.  We really do not know what the methodology was, in terms of exactly what tools were used and how they were administered, or how they dealt with the challenge of finicky results.  (This has led some optimistic commentators to say that we need to wait for the paper to come out; I will wager that we will not know even then — any takers?)  But we do know that a lot of things could have produced the results, and there is no comparison group or trial to help deal with that.

A comparison trial where the subjects just sucked slightly flavored air through a tube — ideally without knowing they were merely doing that, but at the very least even without such blinding — and then were measured would have been a useful comparison.  Also useful would have been a comparison where the subjects inhaled just steam (water vapor), which creates short-term airway irritation for many people.  This would have provided a meaningful comparison for the quantitative results.  Instead of reporting utterly meaningless numbers, they could have reported, for example, that the effects were basically the same as someone vaping pure water.

A more subtle point is hidden in the phrase “a number of tests”.  Dishonest researchers who are trying to get a particular result — and we have established that this describes the present case — frequently take multiple measures and report only the results they like.  In toxicology, this consists of doing lots of different trials with minor variations of the exposure and reporting the one that finally produces the “right” results.  In epidemiology or econometrics this consists of running multiple statistical models on the data and only reporting one of them.  In this case, it sounds like multiple lab tests were run, but only one of them was reported.  The mathematical analysis of the results of such cheating is complicated and usually oversimplified, but with only intuitive math it should be easy to see that any method that creates a scattering of results and reports only the “best” one will bias the results in the direction of the dishonest author’s political preferences.

So, to summarize, we do not merely have a case where the authors made dramatic claims that were not supported by the reported results.  Rather, we basically have no results, since they are meaningless as presented and highly suspect also.  The only thing that we know for sure is that the methodology was never designed to produce useful information, or at least not useful at more than the pilot “let’s see if we can even do these lab measurements before designing some useful research that uses them” level.  This means that the most fundamental lie in all of this is not the bald political assertions about there being “harm” or that you might as well smoke which were not supported by the science; rather, it is the implicit claim that any science at all was being reported.

4 Comments

Posted in Lies

Tagged , , ,

The biomarkers lie

Posted on 7 September 2012 by | 3 comments

posted by Carl V Phillips

Our colleagues in the UK have predicted that there will be a flurry of new studies on e-cigarettes in the run up to the UK government considering how to regulate them, and perhaps this press release from the Greek research team of Christina Gratziou at the European Respiratory Society (ERS) meetings, which we started discussing yesterday, is an example.  Apparently those authors were just trying to get out a quick attack on e-cigarettes.  Others might want to just get in on this new area of research to bolster their career.  Still others might (gasp!) be genuinely interested in finding out the truth about e-cigarettes (a category that we hope includes most everyone who is pro-THR — the cause, and especially the product users, do not benefit from people doing junk science that is biased to overstate the benefits of THR).

Whichever category someone falls into, they basically have three options for research methods.  They can look at the chemistry of the vapor and compare that to existing knowledge about the effects of the chemicals in the relevant quantities (which has been done honestly by quite a few people — the dishonest ones that have already been covered in this blog are not the norm).  They can pointlessly torture innocent animals do toxicology, as was discussed here a few days ago, though it is unlikely that much of that will happen (no one is likely to learn much new about the effects of the simple chemicals in e-cigarettes).  Or they can look at “biomarkers”.  We would really like to have epidemiology, but that is not an option.

(Aside:  It is often said that the reason we cannot have the epidemiology is that it would take a long time to get.  That is certainly true, but it is much worse than that:  The current products will not ever be used by anyone long enough to get epidemiology on them.  The tech is changing too fast, so there will never be people who use the same novel THR products, exclusively enough and for long enough, to ever have the epidemiology.  Indeed, there is a similar problem for many types of short-term research too, like measuring the exact level of a particular contaminant in a particular e-cigarette.  Who cares?  It will not be the same in the products that most people are using next year.  I tried to explain this to the people who were excited about doing e-cigarette research at last year’s SRNT meeting, but it seemed to just baffle them.  I might as well have been speaking Greek(!) to them — they are so used to studying a non-varying product (cigarettes) that they have no idea that most scientists have to deal with a changing world.  And it is playing out just as I expected, with people doing hyper-detailed studies of something that will not exist in a year.)

A biomarker is any measurable effect of an exposure on people’s bodies.  Typically the term excludes anything that would be called a disease in itself, though this is not an important distinction.  A biomarker is anything like heart rate, immune cell count, lung capacity —  anything that can be measured about a body’s state or functioning.  The advantage of looking at biomarkers is that we can look for changes that occur long before a disease event might occur.  For example, we could compare a collection of these biological measures from people who have been vaping for a year to similar people who quit smoking a year ago without any substitute product, and see if there are differences; we would not know for sure what any differences we observed might mean for eventual outcomes that really matter (disease), but we could see the differences.

Do not read too much into the title I chose for this post.  Biomarker results are obviously not inherently lies.  Unlike toxicology, they give us an actual measure of the effect a real exposure has on a real person.  But like toxicology, we do not know exactly what the results mean in terms of disease risk, and it is a lie to claim otherwise.  Indeed, almost any press release about a short-term biomarker study will contain lies, because a press release is going to have to make overblown claims about the implications of the study to attract any attention.  Biomarker studies (done well and reported honestly) add to our total body of information, and so can add up to a lot of knowledge, but each result has extremely limited implications that, if reported honestly, would never be of interest to a newspaper.

Also, in a perfect world, we would be doing a lot more epidemiology of the biomarkers of tobacco/nicotine product use — that is, we would be doing long-term epidemiology (study of real disease outcomes) on people whose biomarkers have been measured to see which of those are really associated with eventual disease.  That would potentially be very useful (which is why I have been pushing the players who could actually make it happen to initiate it) but the payoff is far in the future.

So, to turn this background back to the case at hand:  Gratziou et al. measured a very short-term biomarker.  They had someone puff on an e-cigarette a few times and measured airway resistance.  Even ignoring the details of what they did, reported, and claimed, this tells us very little.  In particular, it does not support the lie that they led their press release with (emphasis added):

New research has shown that despite electronic cigarettes being marketed as a potentially safer alternative to normal cigarettes, they are still causing harm to the lungs.

No such claim can be made about these results.  Whatever the possible value of an acute (very-short-term) biomarker, it is difficult to extrapolate that to actual disease (or even “harm”, which can only be interpreted as a change that increases the risk of eventual disease).  It could be that a lifetime of vaping does increase the risk of lung disease, but there is nothing this study could have found — short of catastrophic problems that we already know do not occur — that would have shown that.  Similarly, if the study had found no measurable change, it would have contributed almost nothing to our confidence that vaping does not hurt the lungs in the long run.

A better biomarker study like the one I suggested a few paragraphs back would tell us more, but measuring immediate impacts tells us very little unless they are truly disastrous (which these were not).  We need to encourage the researchers who are really trying to learn something useful to do the more useful biomarker studies on experienced vapers compared to non-users, rather than just the quickie lab encounters which are cheap and easy (and which tell us something we can add to our body of knowledge, but not much).  These will still only be biomarker studies, not full-on epidemiology, but they can be done now and they will tell us a bit more that is epidemiology-like.

Of course, this was all a long way of getting to the point that other commentators (see my link from yesterday’s post for a good one) led with:  The study did not demonstrate any harm to the lungs.  But I think there is some added value in showing the intermediate steps, as it were, when pointing out why something was a lie.

The research was incapable of producing any terribly useful information, but it could have been a slightly useful contribution to our overall understanding if it had been honestly analyzed and reported.  But it was not, and so it contributed only to misunderstanding.  More on why the reporting itself (even apart from the simple lies) was dishonest as this series continues.

…to be continued…

3 Comments

Posted in Lies

Tagged , ,