This post directly expands on the previous one, and will not make much sense if you have not read that first.
So after musing about this more — there is so much to say — I realized that if someone had just emailed me the text of the editorial, I might have thought it was a parody. I mean, even in the unlikely event that someone thought the various substantive claims and normative assertions were reasonable, would they actually think that they could harangue their critics into ceasing to criticize (except via their heavily-censored officially approved channel)? It is difficult to imagine that the authors would be quite so insulated from normal human behavior that they would not know that they were basically just begging for more ridicule and criticism on forums they cannot censor.
Indeed, the Altmetric report of Twitter mentions of the editorial, at the time of this writing, cites 63. That does not include what must be a couple of hundred that link to my first post about it, but that do not link to the editorial directly. By my quick count, one of those said something positive, four were neutral announcements of the editorial’s existence, and the rest were criticizing or ridiculing it in one form or another. Perhaps the authors were going with “any publicity is good publicity” when they asked for this inevitable reaction. It is, after all, just a boring technical-note type article, and yet is already in the top 5% of Altmetric scores.
But probably that was not the goal. The more I think about it, the more it seems that the real purposes of this were (a) to force Ruth Malone to stop tweeting, as I discussed in the previous post, and (b) to deceive ignorant readers into thinking the journal’s editors actually pay attention to substantive criticism.
After I wrote the previous paragraph, Junican commented on my post that we should consider the possibility of: (c) They were trying to signal to their friends that they also should adhere to what is already the practice of this cabal, to not engage in debate or discussion in any forum tobacco controllers cannot censor. A possible weakness of that theory is that most to all of their friends also already avoid engaging in scientific debate. But perhaps it can be seen as an attempt to declare that criticism outside their own forums is “officially” nonexistent. Something like that. It is worth considering. In some sense this is a specific angle on purpose (b), in which they attempt to mislead readers into thinking this is anything new.
The next comment, from Dr. Sussman, suggested: (d) This is part of a grand scheme to trick outsiders into believing that journals like this are legitimate scientific journals, rather than activist house organs. And, more important, they are trying to trick them into believing that bloggers who do not participate in the journal process are similar to those that question, say, research in physics, microbiology, or cultural anthropology. In many fields, any legitimate criticism is welcome within the big tent of the “official” channels for the science, and so anything that is forced to the outside is crankdom. But approximately the opposite is true in the case of tobacco control and other “public health” activism. Of course, in some fields, like economics, most of best scientific debate takes place in the blogosphere. But the average non-scientist (the target audience for this trick: physicians, politicians, etc.) is not aware of that, and just remembers learning as a freshman that journals are legitimate and other forums are not. Anyway, that deserves a post of its own.
Circling back to my point (b), consider this bit from the editorial again:
As a result of discussion about these issues, the Tobacco Control editorial team has now established a policy that editors will not respond to external blog posts or social media messages about specific studies.
My readers know that they already do not respond to blog posts and social media. With the exception of the hapless Malone tweets, I am not sure they have ever done it at all. As I noted, I suspect they realize that any response from them to their expert critics would merely emphasize just how correct the criticisms are. But readers of Tobacco Control are going to interpret this as saying “we are so weary of all the effort we put into deftly deflating all the criticisms of the papers we publish that we have, alas, decided we cannot keep up the pace.” Using literally true statements to imply something that is false is their thing — they are tobacco controllers after all.
In addition there is:
Occasionally, an individual who has written a postpublication critique has declined invitations to review similar papers prepublication.
I noted that they have never once asked me to review a paper, and assured them that I would do it if they asked. I suspect both of those are true of Brad Rodu also. I seriously doubt that they would ask Chris Snowdon or Clive Bates; even though both are excellent scientific thinkers, they lack the credentials (doctorates, professorships, a curriculum vitae full of research publications) that ivory tower journals usually demand. That is even more true for most other bloggers. So who are they talking about? I am going to give them the benefit of the doubt and assume there is some antecedent for this claim, that they did not just concoct it from whole cloth. That seems to leave only Michael Siegel (after all, in his post about the editorial, he seemed to think it was all about him — *smirk* [update: to explain that, note that MS’s theory is that they were responding to a particular one of his posts; the timing was such that seven authors would have had to formulate their position, draft the text, edit and agree to the text, and then get through the journal’s copyediting and tech process, all in less than two weeks — um, yeah.]. He is their kind of people, even though he criticizes a few of their proclivities. His criticisms almost always fiddle around the political edges of a paper, failing to identify core problems with the methodology (or claiming problems that are not really problems), so he would probably be fairly safe to get comments from. As far as I know, he might turn down such invitations, thereby completing their claim.
But even with that benefit of the doubt, the statement still appears intended to mislead the reader. The truth is probably “once [maybe a few times] a single one of these blogosphere critics was invited to review a paper for us [perhaps one he later criticized, but maybe a different one — even that is ambiguous given the word “similar”], but declined.” But the statement implies that they actually invite the serious critics to review for the journal, often, and that those critics refuse.
On a different point, I still cannot get over this:
Ultimately, the author is the guarantor of his or her work and is entitled to be aware of and respond to critiques of that work, particularly when those critiques question accuracy or scientific integrity.
I find it amazing that not one of the editorial authors had a decent liberal education, or is merely well read, and thus never came across the lesson that an author does not get to decide what people later think and say about her work. The work — whether a novel, blog post, or journal paper — acquires its own identity upon publication, and commentators are then interacting with it, not the authors. An author has the option of joining the conversations about her work, of course, and could provide unique insight. But she does not get to control the conversation. It baffles me that someone would not realize this.
In addition this seems rather snowflakey. If someone cannot stand the thought that a paper they published (as author or as publisher) is being criticized out there somewhere without their knowledge, maybe they are in the wrong line of work.
Of course, we are talking about a field where authors of research reports routinely tell readers what they are allowed to think about a study result. This takes the form of bald assertions about causation, statements of what particular changes in the world would change the results, and assertions of what should be changed in the world. This is a perennial serious problem with Tobacco Control and its ilk. It is a problem that, despite the protests in the editorial of “oh, we just depend on the individual reviewers”, lies with the journal itself. The editors could stop authors from doing that, including when the inevitably superficial reviews fail to point out it has occurred. But they do not, and that makes it their fault.
This goes deeper than mere fact that political conclusions in Tobacco Control papers almost never follow from the research (recall my observation from the previous post on their use of the word “thus”). That is also a problem that the journal editors are responsible for. But the deeper problem goes to the heart of why anyone would actually exhibit the parody-level hubris in this editorial (whatever the tactical motives were). The editors of Tobacco Control are not part of a scientific enterprise. They are cogs in a political activist movement that dabbles in science-ish writing when it is convenient. The purpose of that writing is to advance the activism, and thus it does not even cross their minds that they should stop authors from asserting unsupported political conclusions or from telling readers what they should do with the research results. Similarly, they see no problem with declaring that they will ignore scientific criticism of the papers they publish, and that others should also. Scientific debate is not part of their actual mission.
Anyway, they asked for rapid response submissions and I said I would give them a go moving forward (and, recall, I suggested that you, dear readers, mine the archives to do it retrospectively). So I will submit the following to them and let you know what happens:
The authors of this editorial assert that a journal article’s authors are “entitled to be aware of and respond to critiques”, and imply that this is only possible if critiques appear in a forum attached to the journal. Setting aside the fact that authors can easily become aware of and respond to critiques on other forums, I am curious if the authors could offer some basis for claiming such an entitlement? It seems quite contrary to all existing laws, principles of ethics, cultural norms, and standard practices that relate to commentary about published work. Moreover the behavior of many of these very authors suggests they are willing to go to great lengths to avoid being made aware of critiques.
It seems safe to interpret the statement as saying that at least these particular authors would like responses to their work to appear on this page. And so, I am fulfilling their request. (Assuming this is allowed to appear, that is. I say that not because I believe there is anything in this comment that would warrant censorship, but to emphasize the blindness of this process. That is, the commentator really has no idea what will be allowed to appear.) I call the authors’ attention to two blog posts I have written critiquing this editorial to ensure they have the requested opportunity to be aware: LINKS. In those posts I expand a bit on what appears in this submission. I welcome responses to anything in them, either here, in the blog’s comments sections (I promise their comments will not be censored), or wherever else. However, no familiarity with those posts is necessary to respond to the following questions.
The authors declare “a policy that editors will not respond to external blog posts or social media messages about specific studies.” This statement implies that in the past they have provided such responses. However, I am quite familiar with the scholarly blogs (my own and others’) that often criticize papers that appear in Tobacco Control, and cannot recall a single occasion in which an editor of this journal responded. With the exception of the editorial’s last author occasionally engaging in Twitter conversations about articles — a format which precludes serious debate — I am not aware of any social media engagement. Thus I would like to ask the authors to support their implication by characterizing how often actions that are precluded by this policy actually occurred in the past, and to provide a few examples.
The authors state, “Occasionally, an individual who has written a postpublication critique has declined invitations to review similar papers prepublication.” I am one author of such critiques, and highly qualified to review research papers, but have never once been invited to review for Tobacco Control. I am in close communication with other such individuals, and would be surprised to learn that they have received any such invitation. So I would like to ask for clarification: Is the claim here that on a single occasion, Tobacco Control asked someone to review a paper, but s/he declined, and then wrote a critique after it was published? Or did that happen twice, three times, or more? Or does this merely mean that someone who was once invited to review *some* paper at the journal, and declined, later write a critique of another paper the journal published (thus the use of the word “similar”)?
The authors state: “As noted above, the Rapid Response process provides a forum for exploring such issues. In contrast, placing personal blog posts or social media messages complaining about a study, alleging flaws in the review process, or making ad hominem attacks on authors or editors do not advance the field or allow an appropriate scientific dialogue and debate.” I have several questions about this:
Should we interpret this to mean that the Rapid Response process will censor any attempt to post something that “complains” about a paper or identifies flaws in the review process? Taken on its face, this seems to preclude literally any important criticism. If a commenter observed, say, that a causal inference suffers from enormous residual confounding, which was not acknowledged by the authors, and which renders the conclusions in the paper unsupported, how is that not a “complaint” about the paper? If the identified flaw is apparent to the reader, how is that not also an allegation of a flaw in the review process that allowed the paper to be published with that flaw? Some clarification is needed.
Are the authors of the editorial simply saying they object to *explicit* statements about the failures of the review process, and are saying that these are forbidden from this page? And thus the implicit indictment of the review process from noting there is a major flaw in a paper is acceptable? But would noting a major flaw still constitute a “complaint”? If not, what does?
I am also curious about what ad hominem attacks the authors are referring to. Those of us who criticize tobacco control are quite familiar with the experience ad hominem attacks on our analyses (or, more often, as rationalizations for simply ignoring our analyses). Indeed, such attacks are far more common than substantive criticisms of our work. By contrast, I cannot recall any cases of scholarly blogging critics of a paper in Tobacco Control or other tobacco control papers have descended to ad hominem attacks. I would like to ask the authors to provide examples to support this allegation. (I will offer the reminder that drawing conclusions about an author or journal based on a paper is, roughly speaking, the opposite of an ad hominem attack. An ad hominem attack would consist of criticizing or dismissing a paper based on the identity or characteristics of the authors or journal.)
Finally, the authors state: “Our role is to facilitate the processes of peer review, transparency and accountability which underpin the legitimacy and independence of academic research.” I am curious about what transparency they are claiming. It appears to me that the journal (in keeping with common practice in this field) sends out papers to reviewers who are chosen based on a non-transparent basis, keeps those reviewers anonymous and the reviews secret, and then makes a decision to publish based on non-transparent criteria. Yes, there are some published statements about what is considered in this process, but they are sufficiently vague that they seem to preclude or guarantee nothing. Am I wrong about this? If not, what transparency is the editorial referring to?
More immediately relevant, there is no apparent transparency in the decision about whether to publish a particular Rapid Response submission. Again, the stated guidelines seem sufficiently vague that they would allow an ad hoc decision in either direction about most any submission. It seems rather unreasonable to ask commentators to take the time and effort to submit to a system with vague requirements, particularly given the suggestion that merely “complaining” about a paper is grounds for censorship. Again, clarification is needed if, as the editorial claims, this page is a legitimate forum for serious debate.
I will suggest that a genuinely transparent rule would take a form like the following: Should a reader wish to post a comment on my blog, it will appear, unedited, so long as it is on topic. I suppose I would refuse to post a comment that was utterly outlandish — that, say, ranted about the sexuality of a paper’s author, or alleged criminal behavior — but I have never been forced to make such a decision. I will further note (as I have stated previously) that if authors or editors of a paper that I am criticizing wish to comment, I will allow them to say literally anything they want. I suspect the same transparent rules apply to my fellow scholarly bloggers.
[Update: The submission appeared, with the links (though, mysteriously, without my name) on the Rapid Response page. This is not too surprising, given that it would be the height of honesty on their part to refuse to publish it, if you see what I mean. Less surprising is the fact that these editors, despite all their talk about this being about engagement, did not respond to it at all.]