Tag Archives: NCI

The failures of peer review do not begin with the journal – more on the Popova-Ling fiasco

by Brian L Carter

[Editor’s note: This is a guest post by CASAA advisor Brian Carter, who is working with me on research on peer review. He took the lead on part of the behind-the-scenes effort to get the Popova and Ling article retracted, an effort whose failure (and the irony of that failure) I covered in the previous post. –CVP]

Those of us who admire the elegance and clarity of thought contained in good scientific reasoning no doubt found Popova and Ling’s report of a study on warning labels severely lacking. The article is here and Carl V Phillips’s and Clive Bates’s devastating critiques of it are hereherehere, and in comments attached to the article at the journal here. People who manage to publish worthless junk out of sheer scientific ignorance are worrisome enough. But special condemnation is called for when people manage to combine their cluelessness with malicious intent. They use the language and trappings of science like a facade, all to support their decidedly unscientific personal policy goals.

At first, it’s difficult to understand how such an ill-conceived, poorly executed, and scientifically vacuous study could have ever been conducted, much less see the light of day in a respected journal like BMC Public Health. The journal peer review process, which we count on to at least identify utter junk science, was a colossal failure from start to finish, as documented here. However, this most basic failure was simply the last in a long line of peer review failures, aided by willful institutionalized ignorance and prejudice.

Beginning with the release of the article, we can work backwards, like crime scene investigators, to trace the various malicious acts back to the original fraud that formed the ideological genesis for this article. Popova and Ling note in the article that the National Cancer Institute (an organ of the National Institutes of Health), funded their study through a grant awarded to Pamela Ling. This means the very ideas behind the study, the background, the logic, the rationale for doing it, had to be blessed by an expert panel of scientific grant reviewers. These reviewers supposedly make their decisions on the basis of good scientific judgment. If you don’t make your case at this stage, you don’t get the money: Ling had to make a compelling argument for doing what she proposed, and she had to do it better than about 95% of the other applicants because there’s usually only enough money to dole out to the very highest scoring grant applications.

The National Institute of Health publishes information about every awarded grant on its Research Portfolio Online Reporting Tools (RePORT). One of the many pieces of information contained on this site is the applicant’s summary description of the proposed research. Ling’s is here.

Although we are not privy to the study details she proposed, this description clearly supports the thinking and methodology she used and subsequently published. The description is a checklist of the standard University of California San Francisco lies, evasions, and fallacies. Here are the lowlights:

 “Tobacco use is responsible for 35% of cancer deaths, and new smokeless tobacco marketing efforts threaten both to increase cancers caused by smokeless use . . .”

“new smokeless products are marketed as line extensions of major cigarette brands (Marlboro and Camel) to promote ‘dual use.’”

“These changes in smokeless tobacco marketing may blunt the effects of smokefree environments and the health benefits of smokers’ cutting down and quitting.”

“test new counter-marketing messages to block initiation of smokeless tobacco use among novices and the dual use of smokeless tobacco and cigarettes as an alternative to smoking cessation.”

“Findings will be relevant to guide development of policies on smokeless marketing and advertising.” 

That this grant was reviewed and given a top score tells us something about the review committee charged with evaluating its scientific rigor. I have sat on several grant review committees, and each grant has three (sometimes two) primary reviewers (just as journal articles do). With as many as 60-80 grants to review, each member is a primary reviewer on 3-4 grants and usually defers to the primary reviewers of the other grants when submitting a vote on quality score. In this way, the grant review system is embarrassingly similar to the journal review system. The only major difference is that with grant review there are 20 or so potential reviewers, and they all are supposed to be highly qualified to sit on the committee. Most have to have been awarded a grant themselves, be highly published and fairly well known in the field related to the RFA, and have the demonstrated expertise to evaluate the grants they are assigned. However, this academic record is obviously no guarantee committee members are blessed with deep critical thinking skills or untainted by strong political bias.

Clearly, the primary reviewers for Ling’s grant were fellow travelers able to ignore (or, through confirmatory bias, simply not see) the clear evidence that smokeless tobacco use has trivial cancer risk, if any, a well researched scientific finding that makes the entire premise of the grant specious. None of the other 15-20 committee members (who have access to all grants under review, and usually have read the summary descriptions) apparently had any serious objections either. Any one of them could have raised the point of smokeless tobacco’s trivial risks and demanded a debate on the topic, a discussion that could have significantly lowered Ling’s score. So Ling’s grant sailed to the top of the score list in much the same way it sailed onto the pages of BMC Public Health.

How does a grant review committee so ignorant of smokeless tobacco pass muster on Ling’s grant? For this clue we have to dig a little further. In the “details” section of Ling’s RePORT page we discover she had submitted her grant under a specific Request For Applications (RFA). The NIH frequently publicizes RFAs when it wants some narrowly focused research applications to address a particular topic area–in this case, an RFA titled, “Measures and Determinants of Smokeless Tobacco Use, Prevention, and Cessation.” The RFA is a special invitation to submit a grant tailored to it. Full text here: http://grants.nih.gov/grants/guide/rfa-files/RFA-CA-08-024.html.

Most of the RFA contains instructions on how to apply, but the most important piece of information is listed in the executive summary under “purpose.” This text describes what the issuers of the RFA want in an application and helps explain the rubber stamp of the review committee.

“The overall goal is to develop an evidence base to inform smokeless tobacco control efforts, and to develop effective ways to limit the spread and promote cessation of smokeless tobacco use.”

There you have it. From the very beginning the federal government, in the form of the National Cancer Institute, deliberately solicited applications for the express purpose of figuring out how to “limit the spread” of smokeless tobacco, especially for smokers who might fall into the fictitious trap of dual use in an attempt to switch. They formed a review committee that would stick to the flawed central premise of the RFA. The premise was right in Popova and Ling’s wheelhouse, and they naturally proposed an experiment going further than merely misleading smokeless tobacco users with false labeling, but adding some graphic and disturbing images to boot. You can fault Popova and Ling’s ignorance of good scientific practice, and their shameless attempt to use their wreck of a study in a shabby attempt to influence FDA policy. But you can’t accuse them of failing to deliver exactly what the government wanted.

Advertisements

NCI lies about the effectiveness of e-cigarettes, via deletion and explicitly

by Carl V Phillips

It has been documented recently on twitter that the National Cancer Institute (NCI), at their smokefree.gov site, has been deleting stories of people quitting smoking by switching to e-cigarettes.  (Apologies to whoever first broke the story — I cannot figure out who you are to give proper credit.  We retweeted what was probably a retweet of a retweet.)  At this page, NCI solicits stories of successful smoking cessation.  They have like 200 of them!  They would have a lot more, except that when someone reports quitting using e-cigarettes, they systematically delete it.

[UPDATE: I have been told that this forum discussion may be the reason so many were submitted and where the deletions were first reported.  There is at least one call for posts that predates that, via CASAA, but the other is the earliest report of deletions I have found. Note that those are almost three months old, but complaints about the deletions did not go viral until this week.]

The U.S. government historically was, and possibly still is, the world’s most influential anti-THR liar, particularly including the Surgeon General, the National Institutes of Health (which includes NCI), and the CDC.  These lies have undoubtedly killed tens of thousands of smokers, and conceivably hundreds of thousands given their reach.  U.S. government agencies turned down their anti-THR lying a bit for a few years, but have picked up recently with a vengeance.

The lies at the NCI website are particularly egregious since it is very explicitly — in both its name and content — about stopping smoking.  It does not claim to be generally anti-tobacco.  That is great, since the NCI has no business opposing the use of forms of tobacco that cause trivial or no risk for cancer.  (Indeed, it is not entirely clear what business NCI, a research institute, has in creating propaganda even against smoking, but that is a different story.)

I have seen at least a dozen reports from vapers who reported that NCI deleted their stories.  There will probably be dozens more tomorrow since right now the most recent 15 entries, and another 14 of the next 20, are about e-cigarettes (thanks, presumably, to the twitter comments).  The 16th comment and at least three older ones are by people observing that their entries keep getting deleted. (You might find those at the above link; click the “text only” button unless you want to mess with their silly “click on the map” thing (why, exactly, would anyone care where the people live??).  Of course, chances those will be deleted by the time you go there, so I printed the page as it existed at the time of this writing if you want to see it.)

But unacknowledged deletion is not the only anti-THR lie on this site.  

The site is relatively silent about smokeless tobacco, which is interesting because NCI was historically a leading anti-ST liar.  The side does link to couple of documents that contain the usual anti-ST lies, including their own anti-smoking publication which, as an aside, lies:

If you also use smokeless forms of tobacco like chewing tobacco or snuff, you are still putting yourself at risk for oral health problems and cancer. Just because these products do not involve smoke, doesn’t mean they are safe. Smokeless tobacco is addicting and isn’t a healthier substitute for cigarettes.

Of course, the evidence does not support the claim that ST causes cancer or any other oral health problems (indeed, if you avoid the highly sugary versions, it plausibly has oral health benefits on net).  Most of the government has learned to lie with plausible deniability, such that they would phrase that lie that cigarettes are no worse for you than ST as “not a healthy substitute”.  This is a literally true statement that communicates the same lie to the reader, but lets them pretend to be telling the truth.  (Though since, as I have pointed out, the act of carefully choosing words in order to lie with literal truths is arguably more unethical than just stating the lie, perhaps we should give them credit for this.)

The site does not, however, outsource the lies about e-cigarettes.  At a tab on the page, NCI lies:

E-cigs aren’t regulated:
E-cigs contain other chemicals besides nicotine, which also get inhaled. Since e-cigs aren’t regulated yet, there’s no way of knowing how much nicotine is in them or what other chemicals they contain. These two things make the safety of e-cigs unclear.

Yawn.  Not regulated — other than the thousands, perhaps tens of thousands, of regulations that apply to them.  No way of knowing how much nicotine — except by doing what we do for every other product in the market, trusting the manufacturers to be telling the truth about the ingredients.  No way of knowing what other chemicals they contain — other than the many tests that have been done on what chemicals they contain.

But good news!  Since these apparently are the two things that make the safety unclear to NCI, and we know that they are wrong about them, the safety apparently is clear.

E-cigs haven’t been shown to be effective
There haven’t been any scientific studies that prove e-cigs actually help people to quit smoking. There is also concern that using e-cigs may lead kids to start smoking regular cigarettes.

That is the real kicker, since the very evidence that does show them to be effective is exactly what NCI is systematically deleting from the website.  You would like to think that people at our national health science research institute would be familiar with what “scientific” means, but no such luck.  (Your tax dollars at work!)  They seem unaware that each and every one of those stories they are deleting is a scientific study in itself.

As for that “concern”:  I am really worried that e-cigarettes represent a secret plot by a sleeper cell of space aliens to prepare for their eventual takeover of the Earth.  (I mean have you seen some of the mods out there?  If those are not evidence of alien design influence, I don’t know what they are.)   So there is concern that e-cigarettes will lead to the enslavement of humanity, since all that word means is “someone once said they are worried about this.”

In fairness, their other bullet point is true:

E-cigs contain nicotine:
An e-cig is a battery operated (disposable or rechargeable) device that contains nicotine. The nicotine is turned into a vapor in the e-cig and then inhaled. The vapor looks similar to smoke. E-cigs come in all sorts of sizes and sometimes have flavored nicotine cartridges.

I think they are trying to suggest that this is a strike against e-cigarettes.  I think that perhaps this means they are really clueless.

The bottom line is that we just don’t know enough about e-cigs, so we don’t recommend that you use them. There are other quit aids, with or without nicotine, that have been proven to be safe and effective at helping people quit smoking. But if you do choose to use an e-cig, we recommend that you be very careful!

A remarkably mild conclusion, actually (setting aside the lie that there are medicines that have been proven to be safe and effective).  Ok, NCI, fine.  Since you seem unaware of the vast body of evidence about e-cigarettes, it is fine if you choose not to recommend them.  E-cigarettes do not need the recommendation of an agency that is not supposed to be in the recommendation business anyway.  But that is no excuse for lying about something that you openly admit you do not know as much about as we do.  It is most definitely no excuse for the government to censor citizens’ perfectly legitimate, honest, useful, topical comments at a website that they are paying for.

[UPDATE 2:  A few CASAA members — at least I assume you are CASAA members!! — have already reported filing the suggested FOIA request (see comments).  No use being redundant about this, so please forward whatever responses you get to board@casaa.org when you get them.]