Tag Archives: UAthens

Gratziou ERS press release – where is the science?

Posted on 8 September 2012 by | 4 comments

posted by Carl V Phillips

In the previous posts we pointed out how Christina Gratziou’s anti-THR politics-disguised-as-science could be recognized as lies (even by someone who did not understand the science) and explained why her concluding about real disease outcomes from a short-term biomarker study were inherently flawed.  This did not even address the what she reported about the research, which I will do today.

Here is the content (all of it):

The study included 8 people who had never smoked and 24 smokers, 11 with normal lung function and 13 people with either chronic obstructive pulmonary disease (COPD) or asthma.

Each person used an electronic cigarette for 10 minutes. The researchers then measured their airway resistance using a number of tests, including a spirometry test.

The results showed that for all people included in the study, the e-cigarette caused an immediate increase in airway resistance, lasting for 10 minutes. In healthy subjects (never smokers) there was a statistically significant increase in airway resistance from a mean average of 182% to 206%.

In smokers with normal spirometry there was a statistically significant increase from a mean average of 176% to 220%. In COPD and asthma patients the use of one e-cigarette seemed to have no immediate effect to airway resistance.

So, what does this mean (other than the obvious conclusion from the first paragraph: that this was a ridiculously small study to be drawing any conclusions from)?  One good answer is “nothing”.  That is, this was sent out to newspaper reporters who passed it on to the general public, and what it meant to them was absolutely nothing.  So those readers skipped past the science and read only the lying assertions that were tacked onto it, assuming that those complicated bits supported the assertions; the lab results were simply there to camouflage the fact that the press release was little more than a political statement.

How can there be any honest motive for sending this information to the general public?  I am in well into the 99th percentile of the population in terms of ability to understand this science — easily more like 99.99th  — and I could not make much sense of it.  Fortunately, I know people in the 99.9999th, so I could ask a few question.

Also, before I started writing about this I asked a few questions of Gratziou, via the press contact listed on her press release.  The contact assured me that she would get the questions to Gratziou and I would hear back.  I did not hear back.  Real scientists generally like to explain their work to other scientists; liars, on the other hand, do not like follow-up questions.  (In case it is not obvious, my questions were simple and technical (and thus in no way impolite), and only about the scientific bits.)

What did the researchers do and find?  That is not clear even to the expert readers.  “Spirometry” is not an adequate description of the measurement method.  It is also not clear even to experts what the units of measure mean.  Absolute measures would have been more useful, especially lacking any definition for the “100%” that the percentages are based on.  It could be that the first of each pair of percentages was the baseline and the second was the result after the exposure, or it could be that the baseline was called 100% and these are the ranges of the final results.  The former is the natural way to read the sentences, but would mean that the smokers had clearer airways to start with than the nonsmokers, which seems unlikely.

So even for experts, the information about results presented to the world was meaningless.  That does not, however, mean there is not useful information about what was wrong with what they did.

As noted in the previous post, the biggest limitation (even worse than the size of the study) is that it was based on just the immediate reaction to a first exposure.  Moreover, the subjects were exclusively people who were not used to vaping, and were given the exposure under contrived and medicalized (and thus disconcerting) circumstances.  Would an experienced vaper get the same effect?  There is reason to believe not, since vapers quite often report that they had some airway irritation (which would cause the observed effect) when they first started vaping, but it seemed to disappear.  This would have been useful to report, though it is not clear whether Gratziou failed to report it because she was hiding it or because her knowledge is so limited that she does not even know it.  It is similarly not clear whether these lab-technicians-posing-as-scientists intentionally chose only subjects that would produce uninformative results, or did so out of ignorance.

That “lab technicians” remark is not a slight against techs — we would not get anywhere in lab and clinical sciences without good techs.  But scientists need to think beyond, “I have a machine, I have a person, let’s see what happens.”  In particular, in a case like this, they need some calibration (aka “controls”).  A scientist knows (contrary to popular belief) that it is not always necessary to have a null exposure control group or trial (“placebo control”), but sometimes, like in this case, the results will be impossible to make sense of without it.

Airway resistance measurement is finicky, sensitive to exactly what is being done and subjects’ behavior, as well as anxiety and other factors.  We really do not know what the methodology was, in terms of exactly what tools were used and how they were administered, or how they dealt with the challenge of finicky results.  (This has led some optimistic commentators to say that we need to wait for the paper to come out; I will wager that we will not know even then — any takers?)  But we do know that a lot of things could have produced the results, and there is no comparison group or trial to help deal with that.

A comparison trial where the subjects just sucked slightly flavored air through a tube — ideally without knowing they were merely doing that, but at the very least even without such blinding — and then were measured would have been a useful comparison.  Also useful would have been a comparison where the subjects inhaled just steam (water vapor), which creates short-term airway irritation for many people.  This would have provided a meaningful comparison for the quantitative results.  Instead of reporting utterly meaningless numbers, they could have reported, for example, that the effects were basically the same as someone vaping pure water.

A more subtle point is hidden in the phrase “a number of tests”.  Dishonest researchers who are trying to get a particular result — and we have established that this describes the present case — frequently take multiple measures and report only the results they like.  In toxicology, this consists of doing lots of different trials with minor variations of the exposure and reporting the one that finally produces the “right” results.  In epidemiology or econometrics this consists of running multiple statistical models on the data and only reporting one of them.  In this case, it sounds like multiple lab tests were run, but only one of them was reported.  The mathematical analysis of the results of such cheating is complicated and usually oversimplified, but with only intuitive math it should be easy to see that any method that creates a scattering of results and reports only the “best” one will bias the results in the direction of the dishonest author’s political preferences.

So, to summarize, we do not merely have a case where the authors made dramatic claims that were not supported by the reported results.  Rather, we basically have no results, since they are meaningless as presented and highly suspect also.  The only thing that we know for sure is that the methodology was never designed to produce useful information, or at least not useful at more than the pilot “let’s see if we can even do these lab measurements before designing some useful research that uses them” level.  This means that the most fundamental lie in all of this is not the bald political assertions about there being “harm” or that you might as well smoke which were not supported by the science; rather, it is the implicit claim that any science at all was being reported.

4 Comments

Posted in Lies

Tagged , , ,

The biomarkers lie

Posted on 7 September 2012 by | 3 comments

posted by Carl V Phillips

Our colleagues in the UK have predicted that there will be a flurry of new studies on e-cigarettes in the run up to the UK government considering how to regulate them, and perhaps this press release from the Greek research team of Christina Gratziou at the European Respiratory Society (ERS) meetings, which we started discussing yesterday, is an example.  Apparently those authors were just trying to get out a quick attack on e-cigarettes.  Others might want to just get in on this new area of research to bolster their career.  Still others might (gasp!) be genuinely interested in finding out the truth about e-cigarettes (a category that we hope includes most everyone who is pro-THR — the cause, and especially the product users, do not benefit from people doing junk science that is biased to overstate the benefits of THR).

Whichever category someone falls into, they basically have three options for research methods.  They can look at the chemistry of the vapor and compare that to existing knowledge about the effects of the chemicals in the relevant quantities (which has been done honestly by quite a few people — the dishonest ones that have already been covered in this blog are not the norm).  They can pointlessly torture innocent animals do toxicology, as was discussed here a few days ago, though it is unlikely that much of that will happen (no one is likely to learn much new about the effects of the simple chemicals in e-cigarettes).  Or they can look at “biomarkers”.  We would really like to have epidemiology, but that is not an option.

(Aside:  It is often said that the reason we cannot have the epidemiology is that it would take a long time to get.  That is certainly true, but it is much worse than that:  The current products will not ever be used by anyone long enough to get epidemiology on them.  The tech is changing too fast, so there will never be people who use the same novel THR products, exclusively enough and for long enough, to ever have the epidemiology.  Indeed, there is a similar problem for many types of short-term research too, like measuring the exact level of a particular contaminant in a particular e-cigarette.  Who cares?  It will not be the same in the products that most people are using next year.  I tried to explain this to the people who were excited about doing e-cigarette research at last year’s SRNT meeting, but it seemed to just baffle them.  I might as well have been speaking Greek(!) to them — they are so used to studying a non-varying product (cigarettes) that they have no idea that most scientists have to deal with a changing world.  And it is playing out just as I expected, with people doing hyper-detailed studies of something that will not exist in a year.)

A biomarker is any measurable effect of an exposure on people’s bodies.  Typically the term excludes anything that would be called a disease in itself, though this is not an important distinction.  A biomarker is anything like heart rate, immune cell count, lung capacity —  anything that can be measured about a body’s state or functioning.  The advantage of looking at biomarkers is that we can look for changes that occur long before a disease event might occur.  For example, we could compare a collection of these biological measures from people who have been vaping for a year to similar people who quit smoking a year ago without any substitute product, and see if there are differences; we would not know for sure what any differences we observed might mean for eventual outcomes that really matter (disease), but we could see the differences.

Do not read too much into the title I chose for this post.  Biomarker results are obviously not inherently lies.  Unlike toxicology, they give us an actual measure of the effect a real exposure has on a real person.  But like toxicology, we do not know exactly what the results mean in terms of disease risk, and it is a lie to claim otherwise.  Indeed, almost any press release about a short-term biomarker study will contain lies, because a press release is going to have to make overblown claims about the implications of the study to attract any attention.  Biomarker studies (done well and reported honestly) add to our total body of information, and so can add up to a lot of knowledge, but each result has extremely limited implications that, if reported honestly, would never be of interest to a newspaper.

Also, in a perfect world, we would be doing a lot more epidemiology of the biomarkers of tobacco/nicotine product use — that is, we would be doing long-term epidemiology (study of real disease outcomes) on people whose biomarkers have been measured to see which of those are really associated with eventual disease.  That would potentially be very useful (which is why I have been pushing the players who could actually make it happen to initiate it) but the payoff is far in the future.

So, to turn this background back to the case at hand:  Gratziou et al. measured a very short-term biomarker.  They had someone puff on an e-cigarette a few times and measured airway resistance.  Even ignoring the details of what they did, reported, and claimed, this tells us very little.  In particular, it does not support the lie that they led their press release with (emphasis added):

New research has shown that despite electronic cigarettes being marketed as a potentially safer alternative to normal cigarettes, they are still causing harm to the lungs.

No such claim can be made about these results.  Whatever the possible value of an acute (very-short-term) biomarker, it is difficult to extrapolate that to actual disease (or even “harm”, which can only be interpreted as a change that increases the risk of eventual disease).  It could be that a lifetime of vaping does increase the risk of lung disease, but there is nothing this study could have found — short of catastrophic problems that we already know do not occur — that would have shown that.  Similarly, if the study had found no measurable change, it would have contributed almost nothing to our confidence that vaping does not hurt the lungs in the long run.

A better biomarker study like the one I suggested a few paragraphs back would tell us more, but measuring immediate impacts tells us very little unless they are truly disastrous (which these were not).  We need to encourage the researchers who are really trying to learn something useful to do the more useful biomarker studies on experienced vapers compared to non-users, rather than just the quickie lab encounters which are cheap and easy (and which tell us something we can add to our body of knowledge, but not much).  These will still only be biomarker studies, not full-on epidemiology, but they can be done now and they will tell us a bit more that is epidemiology-like.

Of course, this was all a long way of getting to the point that other commentators (see my link from yesterday’s post for a good one) led with:  The study did not demonstrate any harm to the lungs.  But I think there is some added value in showing the intermediate steps, as it were, when pointing out why something was a lie.

The research was incapable of producing any terribly useful information, but it could have been a slightly useful contribution to our overall understanding if it had been honestly analyzed and reported.  But it was not, and so it contributed only to misunderstanding.  More on why the reporting itself (even apart from the simple lies) was dishonest as this series continues.

…to be continued…

3 Comments

Posted in Lies

Tagged , ,

Christina Gratziou is a liar

Posted on 6 September 2012 by | 22 comments

posted by Carl V Phillips

Today I will start an analysis of the buzz-generating press release by Christina Gratziou (University of Athens) et al., based on their presentation at the recent European Respiratory Society (ERS) meetings.  Much has been written about this non-result that was blown up into the unsupported claim that e-cigarettes damage the lungs.  I call your attention in particular to a post by Katherine Devlin (ECITA) which does an excellent job of analyzing the situation, as well as presenting the analysis with verve that I cannot hope to improve on (see her letters to newspapers that naively reported on this, which are linked from the post, for more detail).

While it would be tempting to just yield this topic to Katherine (especially since she even used “lies” as the first word of her title), I think I have a few views on this that have not been covered and a few useful clarifications.  Also, I hope to make this blog a catalog of both anti-THR lies and issues related to anti-THR lying, and there are some points to be covered on that here.  With that, I start with the question of “why should the lay reader (or a reporter who does not understand the subject but wants to transcribe the press release) know that Gratziou is a liar?”

Start with the official abstract of her presentation:

Electronic-cigarette is marketed as potentially safer tobacco exposure product, but there are not enough data concerning its impact on the respiratory system. Therefore, we set out to investigate the acute effects of an e-cigarette on respiratory functions in healthy subjects and in smokers with and without chronic airway obstruction (COPD and asthma).

The basic statement is not too terrible.  To make it accurate would require only the minor change to saying e-cigarettes are a safer product.  Alternatively, to hedge and not make unnecessary claims in the introduction, that they are potentially safer or that they are widely believed to be safer.  Perhaps the poor wording just reflects the author not being a native English speaker; after all, “tobacco exposure product” is clearly garbled.

But, wait: “marketed”?  Why would a scientist care how they are marketed, even if she is inclined to mention how they are perceived. The only time anyone ever uses that nefarious-sounding word, marketed, is when they are actively setting out to dispute the marketing claim.  But someone doing vaguely-informative biomarker research should not be in the business of addressing such claims.  Someone writing a summary of all the knowledge on a topic can engage in such pursuits, but a single bit of lab research can only ever be about that single bit of lab research.

Now consider the press release (reporting on the exact same results covered in the abstract):

New research has shown that despite electronic cigarettes being marketed as a potentially safer alternative to normal cigarettes, they are still causing harm to the lungs.

This includes similar downplaying of the existing knowledge that they are safer.  But, wait: “despite”?  That word conveys the message that the belief is wrong because the research provides evidence of that fact.  But this is obviously a lie; even someone who somehow honestly believes that we lack sufficient evidence that e-cigarettes are less harmful than smoking, and whose caveats and other wording meant that she was just being a hyper-cautious about drawing any such conclusion, would know that finding one specific harm from e-cigarettes would not justify that “despite”.  Not unless the discovered harm were somehow worse than the combined harms from smoking, which this one obviously was not.  (Indeed, it was not apparently much of anything, but I will come back to that in a later post.)

But Gratziou is not hyper-cautious, nor a careful scientist.  Consider, for example, this interview she gave at the conference (h/t “elpatron83” at ECF for the link)

We don’t suggest anybody to take and use e-cigarette as an alternative tool for smoking [broken English in the original — not a criticism, just trying to quote accurately]

(Aside:  Viewing the video, and even reading her text, one cannot help but be struck by her apparent glee when claiming that a tool that is helpful to so many people is bad for you.  She was not sad to be the bearer this unfortunate news; she reveled in it and clearly thought it was wonderful to report it.  Such glee — which can be observed in many anti-THR and anti-tobacco activists — is not the mark of a mere liar, but of something much much worse.)

The hyper-careful scientist possibility goes out the window as soon as she starts making unequivocal policy/behavior recommendations.  Even apart from the fact that she was telling people “you might as well smoke”, the fact that she was telling them what to do at all makes her an advocate.

There is nothing wrong with being an advocate, of course, nor with doing research when you are also an advocate.  But if you are actively supportive of a particular relevant viewpoint, you need to make that conflict of interest clear.  A researcher who gives no hint that she has that conflict of interest is implicitly communicating that she does not care which way her research comes out, which is clearly not the case here.

Moreover, if you are an advocate-researcher, then if your research tends to support your bias, you then need to be extra careful vigilant about presenting it in a way such that an honest expert reader of it would conclude, “I understand that the author preferred to get the answer that she got, but when looking at what she reported, unless she was simply lying about the data, she does indeed seem to have supported her conclusions.”

Gratziou failed both of these tests.  She covered up her conflict of interest, and she presented nothing but vague descriptions of the research and assertions about the results.

As a contrast, those of you familiar with the bits of my research whose results argue in favor of THR might have noticed that I make clear my views (stating them, that is; giving unintentional clues about bias via weasel words obviously does not count to one’s favor) and do everything I can to make sure that every bit of information that could possibly be reported is reported.  I have read plenty of whining about my research by people who wish it was not right, but I have never once read a challenge to my research by an anti-THR advocate or other ANTZ that identified any flaws in the research or reasoning, or that suggested there was something being hidden.  Ironically, I did publish a correction to one paper based on an error spotted by someone who is pro-THR; for most anti-THR papers, spotting such errors, which are clearly rife, would be impossible because the authors just make assertions rather than showing their work.

It is also interesting to consider the previous posts in this blog from the perspective of the obligations of the advocate-researcher:  Stephen Hecht has consistently covered up the fact that he is an anti-THR activist, and pretends to be doing unbiased research.  Ellen Hahn does not actually do research, so there is no mistaking that she is merely acting as an anti-THR activist — so by this one measure, she is the least dishonest of the individuals we have discussed.

Returning to Gratziou, she is Chair of the ERS Tobacco Control Committee, which is apparently responsible for the ERS’s aggressively anti-THR statement on e-cigarettes; undoubtedly she approved it and quite probably she authored it.  This moves her from advocate to activist.  (Incidentally, that statement might well be the dumbest, most uneducated position statement I have ever read from a theoretically-respectable organization — I am not counting the American Pediatric Association or ANTZ groups in that category, of course.  So it will be the topic of one or two future posts here.)

I will go on to analyze the science of her report — the few sentences worth of it that she actually reported — in the next couple of posts.  But today’s post should demonstrate how even someone who knows basically nothing about science (like, say, a typical health reporter for a newspaper), someone who cannot even figure out that the transitory effect that she measured cannot legitimately be called “harm to the lungs”, should know that she is a liar:  Her background statements made clear her activist agenda.  Yet she never admitted to being an activist, let alone made sure her claims were credible in spite of her bias.  Her grand sweeping conclusions about e-cigarettes could not possibly be supported by her research; even a health reporter should be able to figure out that unless she discovered an effect that might kill a third of all long-term users (which she obviously did not, nor did she even claim), then even if her results were completely definitive, she would not have support for her claim that you might as well smoke.  Yet she aggressively used weasel wording to imply that her little result contradicted the conventional wisdom.

Given the clear evidence that she is a liar and intent on presenting biased results without admitting her biases, her assertions should not be accepted (let alone published in newspapers), and the substance of her report (such as it is) should be analyzed with that in mind.  As I will do.

…to be continued….

22 Comments

Posted in Lies

Tagged , , , ,