Today I’ll review one interesting sentence in Climategate Letter 1080257056 on March 22, 2004, in which Jones tells Santer
She [Heike] sent me an email to review a paper two weeks ago. Said I didn’t have time until May.
Innocuous enough on the surface. What makes this sentence interesting (and I noticed it because I looked for something like this) is that, in my opinion, the sentence is sufficient to identify the paper in question. Further, there is convincing evidence that Jones did in fact carry out the requested review (after May, as he says here) and, even though the review is not in the Climategate documents, it is nonetheless accessible and, together with other Climategate Letters, leads on to many backstories.
Heike
Heike of the Climategate Letter 1080257056 can conclusively be identified as Heike Langenberg of Nature – enabling us to conclude that, around March 9, 2004, Jones was asked to review a submission to Nature.
On that very date (March 9, 2004), I received two emails from Nature – one from Heike Langenberg saying that MBH would be asked to issue a Corrigendum ( a long story in itself); the other from Rosalind Cotter, another Nature editor, who said that two referees of our January 2004 submission were “essentially in favour of publication in some form” (see here, reviews included.)
We also sent the exchange to two referees, whose comments are enclosed. You will see from this advice that, while our referees are essentially in favour of publication of the comment and the reply in some form, they make important criticisms that we would like you to address before we reach a final decision on publication. Please accompany your revised manuscript with a short letter explaining the changes you have made in response to the comments from the authors and the referees.
The two referees are now known to be Jolliffe and Zorita – both highly qualified on the matter at hand and neither closely associated with CRU and the Team. At the time, I suspected that Nature was more than a little surprised that our submission had received favourable reviews.
We resubmitted our revise-and-resubmit a couple of weeks later, but didn’t receive a decision until August 2004. It took more than twice as long for the revise-and-resubmit to be reviewed as the original article. We learned that a third referee had been added – a referee with definite animus towards us. His review was as follows:
It seems interesting that in the comment not only the original publication (MBH98), but also, MBH99, a rebuttal by Mann et al. available from a CRU website (ref.3), an “unreported MBH calculation” available from a University of Virginia website (ref.5), another rebuttal (corrigendum) by Mann et al. now published in Nature (ref. 9), and a detailed critique of MBH by McIntyre and McKitrick published in Environment and Energy (ref. 14) are cited. Additionally, the paper by Jones and Mann published in Reviews of Geophysics (ref. 4, response) already touches this issue. [Received 20 October 2003; revised 4 February 2004; accepted 17 February 2004; published 6 May 2004.]
Besides numerous technical and data-related issues, McIntyre and McKitrick also address a possible CO2 effect on southwestern US strip-bark trees that was “corrected” using high-latitude tree-ring data. Whether it was at all useful to use these data or to apply this correction, seems not highly relevant, since MBH never hid this issue, but described it in detail. More relevant and pleasing would be if someone would find a way to assess the possible CO2 fertilization effects that potentially influence growth at many sites. Additionally, the observation that some of the chronologies used in MBH98 and MBH99 have quite low sample replication during their early periods is also not new and was mentioned in a recent paper published in EOS.
To judge that the criticism by McIntyre and McKitrick is valid would require downloading all data and applying the seemingly differing approaches. Further, judgments would be needed on methodological decisions that were made by both McIntyre and McKitrick and by Mann et al. as two possibilities within the whole spectrum of methodological decisions on which chronologies to use, the calibration and computation of PC’s over different time periods, special treatments to series, and so on. It could be seen as interesting, that the calculations as done by another operator with other perhaps reasonable alternative methodologies can have such a large effect on the resulting reconstruction.
Unfortunately, I have the impression that preconceived notions affect the potential “audit” by McIntyre and McKitrick. That would, of course, not mean that their assessment is necessarily wrong, but might explain the rather harsh and tricky wording used here and at other places by both parties, and I generally do not believe that this sort of an “audit” and rebuttal will lead to a better understanding of past climate variations.
Generally, I believe that the technical issues addressed in the comment and the reply are quite difficult to understand and not necessarily of interest to the wide readership of the Brief Communications section of Nature. I do not see a way to make this communication much clearer, particularly with the space requirements, as this comment is largely related to technical details.
I also find it relevant that McIntyre and McKitrick already published a critique on MHB98 including some arguments similar to what is outlined in the current manuscript (ref. 14).
There’s another strand of evidence suggesting that Jones was the added reviewer. Elsewhere, we’ve seen Jones’ tendency in reviews to self-cite. The added reviewer cited Jones and Mann (2004) on matters M&M – an article that was not even published until May 6, 2004 – after our re-submission to Nature in late March 2004.
Right now the evidence is circumstantial. (The question could be easily settled by either the University of East Anglia or Nature.) I suppose that it is remotely possible that, in March 2004, Nature asked Jones to review another paper and asked someone else to review our submission. But that seems a bit farfetched. For now, let’s work with the assumption that Jones was the added reviewer (and I’ll refer to the review by the added reviewer for the rest of the post as the “Jones Review” ).
Amusingly, the “Jones Review” used the word “tricky” – a word that Jones notoriously used elsewhere (as “trick”) in his es’ email about a “trick… to hide the decline”.
Backstories
Academic Check-Kiting: The citation of Jones and Mann (2004) in the “Jones Review” extended an academic check-kiting incident that I’ve mentioned passim at CA before. Jones and Mann 2004 check-kited claims about M&M from Mann et al., “submitted to Climate Change, 2003”. This article was never published.
The MBH98 Corrigendum (July 2004) also check-kited Mann et al (Clim Chg submitted) (see here) In passing, the MBH Corrigendum was not externally peer reviewed – a point directly confirmed by Marcel Crok. It was merely edited -presumably by Heike Langenberg. In addition, according to an email to me from Nature, not only was the MBH Corrigendum not peer reviewed, it wasn’t even reviewed by a Nature editor.
The handling of the MBH submission to Climatic Change features heavily in Climategate Letters in early 2004 – Jones and Santer were both on the editorial board of Stephen Schneider’s Climatic Change and lobbied hard to prevent Mann from having to provide computer code to a reviewer who had requested it for the purpose of carrying out peer review. (A story for another day.)
Wahl and Ammann: Another backstory from the Nature 2004 submission and response hasn’t been discussed much yet. The battleground arguments in Wahl and Ammann 2007 (increasing the number of retained North American PCs from 2 to 5; the no-PC case etc.) all originated in the Mann et al 2004 response to our Nature submission (and were re-stated in early realclimate posts in late 2004 and early 2005.)
Although the key arguments of Wahl and Ammann are first developed in Mann’s 2004 Nature article, Wahl and Ammann do not acknowledge Mann’s Nature response (the first response had even been published online at Stephen Schneider’s website) or the realclimate posts published in late 2004 and early 2005- both of which were cited in MM (E&E 2005). For that matter, Wahl and Ammann didn’t even acknowledge Michael Mann, though they acknowledged Doug Nychka.
It seems surprising, to say the least, that Wahl and Amman developed precisely the same arguments sufficiently “independently” as not to require citation of Mann’s prior submissions on the same topic.
Wahl and Ammann is first mentioned in a Climategate Letter here of Dec 14, 2004 from Mann to Briffa entitled “paper in review in J. Climate (as a letter”. Note that this is before the publication of our 2005 articles. This may shed some light on something that is very frustrating about Wahl and Ammann (eventually Wahl and Ammann, 2007) – it’s hugely frustrating to deal with because it seldom dealt squarely with our 2005 papers. In particular, in MM 2005 (EE), we discussed the various cases in the MBH submission to Nature (relying on the realclimate posts as authority). Wahl and Ammann discussed the same cases without reconciling (or even citing) our analysis of the same cases. It was as though they had never even read our 2005 paper. (As a reviewer of Wahl and Ammann, I pointed out that they had failed to assess this literature as required under Climatic Change policies, but Wahl and Ammann were not required to do so and I was terminated as a reviewer.)
I haven’t seen the December 2004 version of Wahl and Ammann mentioned here. But I suspect that it might be surprisingly similar to the May 2005 submission to Climatic Change. If so, that would explain why Wahl and Ammann seemed to be more about our Nature submission in 2004, than our substantially re-stated 2005 publications in GRL and EE. Bishop Hill’s excellent essay Caspar and the Jesus Paper will be interesting to revisit in a Climategate context.
Conclusion
Lots of strands and lots of backstories. However, the point of today’s post was to merely preview the backstories and so let’s return to Nature’s actions.
Assuming that the above identifications are correct, Nature responded to the favorable revise-and-resubmit on March 9, 2004 by immediately adding a reviewer with a known adverse interest (to us). Jones had been Mann’s coauthor in Mann et al (Eos, 2003), Jones and Mann (GRL, 2003) and the pending Mann and Jones (Rev Geophys, 2004). Jones and his associates at CRU had published a webpage in later 2003 supporting MBH against MM2003 (also publishing their online response to MM2003). (This is quite aside from the Climategate Letters, which obviously much other evidence of a very close relationship between Jones and Mann.)
This adverse reviewer stated that the “technical issues” in our comment were “quite difficult to understand” – an embarrassing statement in a supposedly eminent science journal – and opined that these issues were “not necessarily of interest to the wide readership of the Brief Communications section of Nature”. If nothing else, this seems to have been a pretty serious misjudgement. Obviously there’s been no shortage of interest in the technical issues involved. It seems like an unusual thing for a reviewer to be opining on anyway.
The “Jones Review” argued that space limitations for Brief Communications stood in the way of the comment: “I do not see a way to make this communication much clearer, particularly with the space requirements, as this comment is largely related to technical details.“ This seems to tie back to earlier actions by Nature. We submitted our revision on March 21, 2004 and a few days later were told to shorten the submission to 800 words. In the final decision, we were told that the decision was rendered on the basis of an allowance of 500 words ( a limit much lower than contemporary comments on Emanuel (2005) for example) as follows:
we have regretfully decided that publication of this debate in our Brief Communications Arising section is not justified. This is principally because the discussion cannot be condensed into our 500-word/1 figure format (as you probably realise, supplementary information is only for review purposes because Brief Communications Arising are published online) and relies on technicalities that do not bring a clear resolution of the underlying issues.
Both at the time and in retrospect, this seemed like a very unsatisfactory disposition of the matter.
One wonders how much more rational the subsequent debate would have been if Nature had published our original comment.
Climate Audit 
193 Comments
I’m interested in readers discussing Climategate-related issues and Jones’ review.
Please do not discuss Nature word limits on comments – a policy that has been deservedly criticized elsewhere. Also please do not re-litigate the Zorita and Jolliffe reviews or the articles themselves – which have been extensively discussed and debated.
Try to discuss whether you think (1) that I’ve demonstrated that Jones was the added reviewer (2) whether Nature intentionally added a reviewer with a known conflict of interest; (3) whether Jones’ adverse conflict of interest interfered with an objective review.
yes.
yes, yes, yes
.
Is there really that much to discuss?
Steve: I’ve just started.
The practice of sending a paper for review to a party in the dispute is so strange. Imagine if the newspaper editor sent a food critic’s article about a restaurant to that restaurant for “review” before publishing, or if Spielberg got to review the critiques of his movies and could vote them down. Who would allow that? Even in 2003 I think it was known that Jones and Mann were allies (and already coauthors?).
Yes to all three points. The peer review process operated by Nature seems to be no longer fit for purpose.
Sure, guessing it was Jones is a reasonable suspicion.
Not sure about conflict of interest – conflicting interest for sure. Conflict is not an unreasonable thing… so long as it’s reasoned. Adversarial processes are a time honored method of getting to truths.
I think I’m going to go reread Othello the suspicions are really piling up.
Steve:
Your identification of Jones as the third reviewer is, as you say, purely circumstantial. The import of the third reviewer’s comments in and of themselves does not differ, IMHO, significantly from those of the other two reviewers. I can see no obvious indication that it is Jones and, given the universe of possible reviewers, I would say that in order to support the identification of Jones as the third reviewer additional information is needed from Nature or CRU or some forensic assessment of Jones’ writing style.
If he answer to your Q1 is no, then Q2 and Q3 are largely moot.
Bernie why would the second question be moot? Shouldn’t Nature avoid submitting to anyone with a conflict of interest Jones or anyone? In that line question 3 is also not moot.
If the question is should Nature’s editors ask individuals with conflicts of interests as significant as the one that Jones has to be reviewers then surely this question answers itself. The question of whether someone with a significant conflict of interest can be objective also answers itself.
I see the questions as moot because in asking Q2 and Q3 Steve refers specifically to Jones.
If the reviewer is not Jones, then one would need to know the nature and extent of his/her conflict of interest.
Not sure that I agree with your assessment of the word “tricky” in this context. More interesting is the use of the “audit” in quotation marks. A brief check of the CRU emails reveal several similar instances. It indicates the reviewer was probably a member of the team. However, I’m not familiar enough with the history to determine if it’s use in that manner was common or uncommon.
There are people who purport to be able to determine who wrote what based on their writing style. Last year, a well known radical from the 1960’s was accused of writing a well known book by a well known politician. Similar work was done on Shakespeare. I can’t speak to validity of their methods, but if this question (about Jones being the reviewer) is important enough, it might be worthwhile to consult one of those folks.
I am an intermittent reader of M&M – Climate Audit since ~2005 and interested in the various, inadequately addressed, solar related energy terms. I was aware of the fading CO2 IR absorption band since grad school in 1977 to possibly, *very weakly* fight global cooling as well as common abuses of computer modeling such as the AGW proponents revel in.
Responses: (1) The article’s discussion here raises the suspicion. Steve is far more finely tuned and sensitive to textual characteristics and issues than a casual reader. More hyperlinks with more detailed reasoning, previous stories or history, word analysis, etc would be useful. (2) Depends on Nature’s rules and reviews. If both reviewers, Z+J, originally recommended publication with minor edits (needs clarification), a third reviewer seems biased to “revolutionary (AGW) orthodoxy”. (3) In such a controversial topic, Nature reviews should have not used a recent co-author. Given Jones’ previous email and public statements, adversarial COI now seems clear to me but should have multiple hyperlinked words to more detail for the casual reader.
The same email exchange has Santer whining about his in-ability to get his paper published in nature. I think all that’s demonstrated is that Nature’s ‘parameters for publishing’ are somewhat more capricious then they would have the world believe.
Just as the parameters for publishing on the NY Times opinion page is a somewhat capricious process despite the various denials of the NY Times editorial board.
What will be or not be published will be governed by
A) The prevailing views of the editorial board
B) The editorial boards perceived need to publish views that differ in order to demonstrate balance or objectivity.
Steve: Again, I want readers to focus on issues of conflict of interest even if you think that the question is moot because of capriciousness.
“….. and I generally do not believe that this sort of an “audit” and rebuttal will lead to a better understanding of past climate variations.”
Maybe not, but it should have lead to a reduction in mis-understanding of past climate variations.
I suppose those are not quite equivalent, but the reviewer seems to have chosen his words especially carefully.
I don’t disagree with the decision to allow Jones to comment. There is some logic to preventing the publication of a blatantly incorrect comment, and an involved reviewer would be more likely to point out a gaping flaw in the argument. Where the process seems to have gone astray is in permitting this reviewer to (a) delay the process of debate, and (b) taking any notice in of the objections which were not to point out a trivial error. The process seems to have been contrived to achieve a particularly non-scientific outcome, but presumably one which was to Nature’s benefit as a business (at least in the short term)
It would seem that the issue, rightly considered, is whether jones was allowed to be an anonymous commenter, and given equal or greater weight than the two approving commenters. Had they openly allowed Jones to reply, it would have been a different question.
In legal terms, absent an explanation from Nature, your points appear to meet the “more-likely-than-not” standard. I think it’d be pushing it to claim points one and two demonstrated “beyond a reasonable doubt.”
Re: Wahl and Amman
Steve, did you see this e-mail ?
From: “Wahl, Eugene R”
To: “Keith Briffa”
Subject: RE: confidential
Date: Fri, 21 Jul 2006 04:23:24 -0400
Hi Keith:
…I have done my best this evening to digest the issues you asked me to look at, and to give perspective on them. Here is what I can offer at this point.
1) Thoughts and perspective concerning the reviewer’s comments per se. These are coded in blue and are in the “Notes” column between pages 103 and 122 inclusive. It got to the point that I could not be exhaustive, given the very lengthy set of review thoughts, so I am also attaching a review article Caspar and I plan to submit to Climatic Change in the next few days. [The idea is that this would accompany the Wahl-Ammann article, to summarize and amplify on it — given all the proper and non-proper interpretation WA has received and the need for subsequent analysis that WA only lightly touches on. Steve Schneider is aware that it is coming.]…
…how over-strong and one-sided are the arguments Steven McIntyre puts forth in this area. … He has done with the IPCC just what he did in reviewing the Wahl-Ammann paper–and indeed in all his efforts–write volumes of very strongly worded, one-sided critiques, which can take a lot of time to see through and then respond to…
From: Keith Briffa [mailto:k.briffa@xxxxxxxxx.xxx]
Sent: Tue 7/18/2006 10:20 AM
To: Wahl, Eugene R
Subject: confidential
Gene
I am taking the liberty (confidentially) to send you a copy of the
reviewers comments (please keep these to yourself) of the last IPCC
draft chapter. I am concerned that I am not as objective as perhaps I
should be and would appreciate your take on the comments from number
6-737 onwards , that relate to your reassessment of the Mann et al
work. I have to consider whether the current text is fair or whether
I should change things in the light of the sceptic comments. In
practise this brief version has evolved and there is little scope for
additional text , but I must put on record responses to these
comments – any confidential help , opinions are appreciated . I have
only days now to complete this revision and response.
note that the sub heading 6.6 the last 2000 years
is page 27 line35 on the original (commented) draft.
Cheers
Keith
Jim:
I see nothing more than “consistent with” – “more likely than not” surely requires some listing of other possible reviewers and their exclusion. For example, why couldn’t the reviewer be Briffa or Santer or Wilson or any other prominent dendrochronologist? One might exclude Santer because, as you note, he had a pending article. IMHO, Steve really needs some additional relevant data to attribute the review to Jones. The reviewer’s comments, of course, desrve to be criticized regardless of the author.
Steve: Bernie, surely the timing in MArch 2004 is strongly suggestive of Jones.
Well, I like Steve’s added fact that the reviewer was citing a Jones article that wasn’t published, yet.
I agree with you that somebody else could have been the reviewer of record, but it seems that that person would have been ‘in cahoots’ with Jones.
We have other e-mails showing patterns of both Jones going to town in reviews, and of Jones suggesting reviewers who “knew what to say”.
It seems like a prima facie argument to me, and I did put in a very weak proviso, above. [“…absent an explanation from Nature…”]
1) Does it really matter if the referee was Jones or not?
2) The behaviour of Nature is very strange. The letter from Cotter makes no mention of sending it to a third referee. This is not normally done unless the first two are in disagreement, which does not apply in this case. It seems that someone at Nature sent your paper to referee 3 in order to have an excuse to reject it.
Steve: I don’t have enough experience with academic protocols to comment on this last point. That’s why I asked the question – it seemed odd to me. Have any readers had a similar experience? A third reviewer added when the first two reviewers are in agreement??
“A third reviewer added when the first two reviewers are in agreement?”
No, not in my >30 years’s experience in publishing in biochemical/biological journals
“A third reviewer added when the first two reviewers are in agreement?”
I have not seen this in my 20 years submitting and reviewing papers, even for Nature.
“I want readers to focus on issues of conflict of interest..”
I think it’s reasonable for an editor to use one of the parties in a dispute to review a critical paper, but I would expect the editor to take into account the obvious interest that party would have in rejection. The best use of the conflicting reviewer would be as someone who is probably more familiar with the material than the other, non-partisan reviewers – giving that person the opportunity to spot errors that others might miss. However, the “JonesReview” used an old hat method of arguing fit-to-journal rather than specific scientific critiques. In this case, the Editors should have ignored the argument.
I have not seen this behavior either. It appears to be “unprecedented”.
Top journals in economics appear to send (original) papers out sequentially to reviewers. They appear to do this sometimes to get the answer they want. But also if a technical/difficult paper gets positive reviews on the first round it might make sense to send it to someone you would not bother otherwise in order to really evaluate it (which the heavyweights actually do in economics).
It does not seem to be inappropriate to send the paper to a co-author of the original paper, but it is bad style and bad method not to treat the review suspiciously. For example, Jones may have been in a good position to spot an error in the reply that the first set of reviewers missed, the kind of flaw that once spotted, can be agreed upon by all. But instead the review shows a complete lack of expertise to evaluate the points made (that even my undergrad students can grasp). To then apparently rely on that review from a person who is not arms-length to reject is appalling.
Like many people here, I’m not surprised that they were doing this kind of thing. It is unusually that we get to see them in action ex post. But it is surprising and appalling that so-called prestigious journals and Nobel prize winners like the IPCC went along with this behavior.
To clarify a point – please don’t take the above post as complaining. Nothing happened at Nature that I didn’t expect.
I’m trying to get a foothold on a different point – how should journals and reviewers deal with conflicts of interest arising out of close association on the one hand and adverse intellectual interest on the other. This sort of conflict is clearly recognized in medical journals. The World Association of Medical Editors policy on
Conflict of Interest in Peer-Reviewed Medical Journals states:
Academics are so inured to these factors that their reaction is simply to swim on and try another journal. And I understand that 99.9% of the time, this is feasible and practical.
But surely we can think a little bit – just for fun.
Conflicts of interest often arise in business. In all cases, it must be disclosed and in many cases, a party is prohibited from acting in a conflict of interest situation.
If a conflict of interest arises in a peer review, it seems to me that they should be disclosed to both authors and readers so that both can assess how much weight they wish to place on the peer review. In a small field, this may make it hard for peer review to be anonymous. If so, shouldn’t the tradeoffs be discussed openly?
Steve:
I agree that the dates and the tendency to self cite provide a sufficient circumstantial case for you to press the issue with Nature and/or CRU. However, you posed a specific question and I believe that more data is needed before you have “demonstrated that Jones was the added reviewer.”
Steve:
Sorry the above comment got tagged to the wrong comment.
In principle, it is hard to conceive of any journal review process that would not require a genuine effort to avoid using reviewers with conflicts of interest. At the same time, if a reviewer has a conflict of interest then they should either recuse themselves or openly acknowledge the conflict and its relevance to the paper being reviewed. If Jones was asked, he should have declined.
Bernie,
Isn’t that the whole point, Jones didn’t decline (if he was asked that is) because it is clear from the other emails that the team spent a good deal of time trying to muzzle articles that were critical/had different conclusions to their agenda. I would agree with you that without the participation of Nature in this discussion toconfirm or otherewise that Jones was a reviewer no jury would find that Jones was the reviewer. However the fact that the reviewer cited an unpublished Jones paper, along with his habit of self-aggrandisment in reviewing would lead any prosecutor to believe he had and try to get proof.
My personal belief is that the balance of probability given the timing, Nature’s seeming political support for the AGW agenda, Jones’ and the Team’s determination to stifle discussion, the citing of an unpublished paper, and the previous habit of self-referencing, is that it was Jones.
Re: Steve McIntyre (Jan 5 13:49),
The US NIH has a coi policy: http://grants.nih.gov/grants/policy/coi/
I know a little bit. I believe that researchers at the NIH are allowed to have extramural activities for pay. These NIH people, having PH.D or M.D. degrees, for example, can make extra income in, hopefully, a structured, public (‘up front’) way. A controversy would be over something like the NIH research being intended as a check on the private research, just as one quick hypothetical.
One other quick example, coi-wise, is at Universities, professors may have income from the ‘private sector’ in addition to “peer-reviewed”, extramural, grants. In what way does one influence the other?
One obvious check is some disclosure of those arrangements. For academics, their colleges and universities need to have coi polcies. Just moving on to another journal would have a check.
Many granting agencies have clear coi rules that require the disclosure of all relevant relationships, co-authors, etc. It is part of the application.
Journals should have similar policies and editors could have an easy way to do a check in order to inform their decision with respect to reviewers. Reviews could still be anonymous in a normal situation but, should a controversy arise, there would be a mechanism that might finally include disclosure.
It is not difficult to fulfill the disclosure rule. Once that first list is created, one simply adds new students and collaborators as time goes by.
I think it is the Editor’s discretion to decide when there is a problematic conflict of interest. Clearly, science is all about conflicting ideas and thus, conflicting interests. Problem is not all conflicts of interest are Conflicts Of Interest, and it can be difficult for an Editor to discern which is occurring. In fields other than climate science acrimonious debate (of the type Osborn wished to avoid) can carry on for years through the literature. Conflicting ideas. Conflicting interests. Hey, that’s science.
Re: bender (Jan 9 12:47),
It is the Editor’s responsibility but mere discretion seems to be part of the problem. The idea of having policies in place as a check can make things less difficult. “Acrimonious debate” is a good description for the way a policy might be finalized. When applied to “through the literature”, the phrase is too bitter for me.
When a conflict of interest exists the parties involved must make a complete disclosure of its existence. The party must also cease to participate (recuse) in the activity or relationship from which to conflict arises. Parties to a conflict can voluntarily waive it unless the waiver will harm the interests of 3d parties or the public.
Jones, Nature, Mann et al appear to have kept the existence of the conflict of interests secret and/or denied one exists so that they can violate and/or avoid confirmation with the accepted standards for fair and impartial peer review.
For anyone who finds the Climate Audit posts a bit cramped and hard to read here is a useful add on hat make the post easy on the eye with just a click.
http://lab.arc90.com/experiments/readability/
I notice a real cultural divide on this sort of conflict of interest issue between academic and business/legal communities. Academics typically don’t understand how strict conflict of interest rules are elsewhere.
IT seems to me that academics tend to argue that the conflict doesn’t necessarily “matter” because an unbiased person might have been able to get to the same decision. In a commercial dispute, (according to my understanding of the relevant law and someone will correct me if I’m wrong), the opposite party is pretty much entitled to assume the worst of a party acting with an undisclosed conflict of interest. The case law goes VERY badly for such people. It’s diametrically opposite to the rationalization by academics.
Also from a commercial point of view, the trading of favorable reviews to which academics have become inured really seems tainted. As long as the articles themselves don’t “matter” to anyone other than the academics, no one cares. Some of the disconnect between academic and public reaction to the Climategate Letters probably arises because the Climategate Letters afford a window on an underside of a more general academic practice, that the public was unaware of but doesn’t like very much.
Seems to me that it’s a time-scale thing. Academics work in a world where, assuming the peer-review process in a field becomes completely munged from time-to-time with wagon-circling cronyism, you simply wait fifty or so years until they’ve all died off, and someone will come along and do the obvious work. In this particular scenario, though, for once the results need to be true in the short-term, as well as in some number of centuries from now.
In my line of work we cannot even have the appearance of a conflict of interest.
From my perspective – far away from academia, I see nothing wrong with going to a third reviewer, particularly if there is controversy with the subject matter. I also see nothing wrong going to an obvious non-advocate – getting away from group think is good and there is often intelligence added to the debate as a result. I also see nothing wrong with going back to the author with reviewers comments – the authors druthers are obvious so it is easy to put in context, and you would not evaluate those comments as you would an unbiased referee. However, if they went back to a third reviewer that they knew had a conflict of interest, and they used that reviewer as an unbiased referee, then they got what they asked for – bad science. That would be an awful legacy for Nature.
Rules against conflict of interest seem to have originated in the field of law. A lawyer is supposed to be an intellectual whore for his client; if he knows the client to be a murderer, he is still supposed to do his best to get him off, within the rather broad limits imposed by other aspects of lawyers’ codes of ethics. In such circumstances, prohibiting conflict of interest is an absolute necessity: a lawyer who tries to serve opposing clients simultaneously can not possibly be a complete intellectual whore for both. In other fields, however, where people are supposed to have some intellectual integrity, conflict of interest is to some extent accepted and managed. The whole idea of democracy, for instance, is a massive conflict of interest: people deciding their own destinies. (Nor are other forms of government less conflicted in this respect, except for the government described in The Hitchhiker’s Guide to the Galaxy, which consisted of submitting all decisions to a man living in an isolated shack on a deserted planet.) With scientists acting in good faith, conflict of interest can mean that, as someone once expressed to me, “Because of conflict-of-interest rules, I wasn’t able to comment on any proposals [for research funding] that I actually knew anything about.” Alternatively, in the absence of good faith, much harsher measures than the mere imposition of conflict-of-interest rules are necessary.
Yep, the idea of science relates more to the principle of a common interest, than the principle of competing and conflicting interests.
Yes, and the reason is that conflict of ideas and interests is an integral part of the scientific process. Academics are used to the idea of engaging in conflict when ideas and interests start to clash. Of course, the material concerns are usually very small. It’s over ideas and data that the most intense conflicts typically arise.
Here’s an economist’s opinion on the ethics of this. It’s actually desirable to send papers to hostile referees, in order to get a more thorough review. The editor, however, should take such reports with a grain of salt, and not hide behind them. He definitely should not tell the referee what to write.
It is the editor’s perogative to reject a paper as being uninteresting or unimportant, as seems to be the excuse here. He should come out and say that himself, though, not pretend that it is the referee’s decision. Referees should be encouraged to provide input on this, but really it’s the kind of thing an editor should be able to judge almost from the title and abstract.
In this case, it looks like the editor wanted to reject the paper but was too cowardly to take responsibility, so he sent it out for refereeing and when that didn’t work he made extra sure to get a hostile reviewer. The hostile reviewer couldn’t, it seems, find anything wrong with the paper (he surely would have said so if he had!), and thus the editor should have known it was correct. Thus, the reviewer went to the last line of defense for rejection: “This just isn’t very important.” That is completely implausible with regard to this particular paper, of course.
at is completely implausible with regard to this particular paper, of course.
Doubly so given the two rounds of reviews. Puts me in mind of the old joke about the differential equations professor who skipped a step in a proof, saying it was “obvious”. When challenged, he looks at the proof on the board, starts to say something, stops, looks at his teaching notes, then retires to his office for a half-hour to consult some other sources. He then returns to the class and says “yes, it’s obvious.”
Whether or not it was Jones: dismissing a ‘comment’ with “this comment is largely related to technical details” leaves me feeling that the reviewer wasn’t qualified for the job – almost all science (Science proper, that is) consists of masses of technical details. As I’ve sung for some years now, many of these Climate Scientists are duds. Hoodlums too, perhaps, but certainly duds. As for his identity – your evidence might not do for a criminal trial, but it might suffice for a civil action. However, IANAL, as they say.
I think it is likely, but unproven, Jones is the reviewer. It seems odd to add a third reviewer after the first two agree. What would motivate such a decision unless Nature wanted a reason to decline the paper? If this is the motivating factor, then Nature must have been counting on the obvious conflict of interest to drive the negative review.
I think authors who have a conflict of interest should only be allowed to review without the cloak of anonymity. It is unfair for readers to read such a strongly-worded review without knowing about the COI.
COI in any contest has to be disclosed and failure must come with some penalty. In other industries this is obligatory and so if people are going to be holding themselves up as final arbiters in this still unfolding science of climate change then they must be humble enough to declare their interest. That way their reviews can be better appreciated.
Steve you talk of a small field but by their actions they are even making the field smaller by keeping every so called dissenter at bay
Something I’ve asked before: How is this different from having Steve McIntyre as a reviewer for Wahl and Ammann?
I’d suggest that it’s somewhat different.
As has been noted above/below, a reviewer who disagrees with the paper or article he’s reviewing isn’t the issue. Providing their disagreement is valid, ie pointing out a mathematical error, invalid use of a statistical tool.
This more like a case of the reviewer just opposes the views of the reviewed and this is, in his opinion, sufficient to invalidate the paper and thus it should be rejected.
What’s different is the ad hoc process by which the third reviewer was called in to trump the positive reviews of Jolliffe and Zorita. Very, very unusual. Show me a case where M or M or M were called in to trump two positive reviews of a an alarmist team article.
It seems odd (make that extremely unusual) to me to request a third review at all, though, when the first two reviews are in substantial agreement. Whatever the reason, the author absolutely should have been notified.
With respect to choosing a particular reviewer, I’ll assume an adversarial reviewer was chosen. There are then three possibilities: (1) the editor did not realize the reviewer was adversarial, which would imply lack of diligence not uncommon for busy editors; (2) the editor sought an adversarial review to get a ‘worst-case’ review, in which case the actual review is so mild to my eye that it would not hinder publication (as in ‘is that the worst the reviewer can come up with?’); (3) the editor sought an adversarial review to game the process. Possibilities 1 and 2 seem most likely; if there had been evil intent, the editor would likely not have provided a favorable report on the first two reviews until the third came in.
In my (engineering) discipline, it is quite common for journal editors to seek a further review if one of the original reviewers notes that the conclusions of a paper are in direct conflict with a previous author. Often comments are sought directly from the conflicting author. I see no problem with this, providing the journal editors are doing this with the (sole) aim of ensuring accuracy in the paper and not with any intent to control an undeclared agenda. The real problem is that Climate Science seems to be out on its own in setting new standards for bad behaviour.
The space limitation excuse is familiar. Unlike Craig, I wouldn’t have a problem with someone on the team I was criticizing seeing my work before publication: I’d want to be sure there wasn’t something I missed. Sometimes their replies reveal another avenue to explore too. It’s akin to a journalist sending out their copy for comment before publication. The reviewer comment though amounts to “I’m not capable of making a technical objection and I’m bone idle too so here’s a simpler trick you might use for rejection”. And Nature editors delayed it until they found a better excuse. All I can say is that it wasn’t the Mann himself because there was no mention of oil industry shills.
This is about how Science journal works, written by Bray at Hans von Storch’s website.
Conflicts of interest there as well?
http://klimazwiebel.blogspot.com/2010/01/tale-of-two-consensus.html?showComment=1262720960072_AIe9_BEbOIg_B6O0J4uadzmnbNoVEk7X-5zIkE1ASqxpk2aSxRxMI4CK25cQ7-R2i_Itw3Tuv8j8qZxGGHTp21OIPC4mw8I3ONCRjUB_x6p0XOkzbDqPnSDPY9uk0_8QuhOY1Yz2Xt_GS0Sbxu17wCDW3IGnsx8ybBLloyW6uTWvlr7Y3w4Y2gqcRswPupioOi087TRt_Lf59_NfuwpkNO9Zs0h9g6LI2s_17k4cZLxb7GN69_uqTvY#c1511633893592949200
A couple of comments:
1.) In the context of the e-mail, would you have expected Jones to have referred to McIntyre & McKitrick by name, rather than just as “a paper”?
2.) In terms of adversarial reviews / conflict of interest, were you not a reviewer of Amman and Wahl’s Climatic Change paper that purported to rebutt your paper? Is this really any different?
Steve: 1) Dunno. 2) Yes, they asked me to review Ammann and Wahl. I thought that the whole idea of asking a person adverse in interest to act as a reviewer was very odd – even I was the one being asked. ( What makes more sense is giving them the right to reply. ) Having said that, as a reviewer adverse in interest, I tried hard to ensure that my review comments adhered to policies. Also, I had no particular objection to Ammann and Wahl publishing a comment – what I objected to was their misrepresentation of our paper, withholding of adverse results, etc. For me in my capacity as a reviewer, their refusal to disclose verification r2 statistics was objectively objectionable, as was their failure to disclose the rejection of their GRL comment.
The objectives of the editor seem to be different. In the Nature case, the editor arguably asked Jones to review in order to get a reject review (sort of like Briffa asking Cook to hammer a dendro paper.) In the Wahl and Ammann case, Schneider asked me entirely as window-dressing. He totally ignored all my review comments regardless of their validity – and every one was valid – and terminated me as a reviewer.
At the time, I didn’t have much experience in academic reviewing. I might do it differently now.
He totally ignored your comments? He didn’t go back to Wahl and Ammann for a response?
Looking at the ClimateGate mails, it looks like it took them a while to get published. Didn’t your review have anything to do with this?
On a side note, did Nature really take 4 years to decide on your paper?
Looks like Wahl and Ammann did see (and respond to) Steve’s Comments:
http://www.eastangliaemails.com/emails.php?eid=805&filename=1182361058.txt
-Chip
Steve: That’s a very long story. See the Wahl and Ammann category. They did not respond to the comments per se. As a reviewer, I insisted that they provide verification r2 results (they had issued a press release saying that our claims were invalid.) They refused. I had lunch with Ammann at AGU and again asked him to report verification r2. Again he refused. I filed an academic misconduct complaint at UCAR. Then they reported the ghastly verification r2 results in an Appendix to Wahl and Ammann 2007.
This letter from Wahl is absolutely bizarre. Having a scientific paper challenged is grounds for “slander” and is “unspeakable”. There must be some way to get a “cease and desist” etc. The megalomania of these people is astonishing.
Re: pcknappenberger (Jan 6 13:15),
I can think of only one alternative to what occurred, and the information in these recent threads than this:
Journals are in the business of producing papers, the papers are based on grants and scientists further the reputation of their institutes and training grounds. All in all this is a commercial relationship — though not usually expressed as one and journal advertising and obligations are not necessarily obvious.
Perhaps you were offered a probationary membership in “The Club” and did not realize it. If you had made a few corrections on the use of semicolons, added a few comments, pointed out how to better use a few words, then you likely would have received another opportunity to review — and eventually publish.
Offering the truth, upsetting a publishing schedule, ticking off grant providers etc doesn’t necessarily endear you to the various parties. Maybe that has more to do with what you are seeing than anything else. The compulsion to “go along to get along” is pretty darn strong.
My opinion: You have been offered an opportunity to join “The League of Extraordinary Gentlemen” — Climactic Science Chapter and did not recognize it for what it was. Should you have accepted you might have a lot more “respectability” and “credibility” but liked it a lot less. Maybe you would have a better opportunity to effect change — maybe not.
I have received a few NRC grants, and lent my name (such as it is) to a few research proposals and grants (NSERC). and have seen this process at work. I’m not particularly comfortable with it. Some people just can’t handle the truth. 🙂 ..and it gets some of us in trouble.
In truth I do not believe that anyone really understood how due diligence — or the lack — would affect the science community and mankind in general. I suspect that everyone wanted to remain on the :”business as usual” footing not knowing how theses decisions would affect the deployment of trillions of dollars. Perhaps they only thought of keeping the “Grant Engine” running smoothly. Wish I knew the answers.
..and I think I have said enough on this issue.
Not at all. To resolve a conflict of ideas there must be a dialogue involving some kind of conflict. The Editor may choose to set up a very lively, clashing, short-lived discussion that gets settled quickly, or a muted, lingering, back and forth that seems to never resolve. The Editor exercises this right and responsibility through their choice of reviewers and style of oversight in the review process. Editors are powerful people.
Dear Mr McIntyre
Stop trying to get even, get mad?
It is not a cultural divide, you give to much credit to this group of self named climate scientists.
Normally people whether they be doctors, engineers, scientists or dare I say statisticians behave in a professional manner, there are of course exceptions.
What would the man on the Clapham Common bus think about all this. Much to my shame Jones and Schmidt are British.
I’m not a Writing Style Analyst, but the review is remarkably similar to texts written by Jones. His style is usually clear, concise, grammatically correct and unambiguous. His e-mails stands apart from most others because of their efficient use of language.
The onus must be on the editor to ensure no conflict of interest with reviewers. Of course, who makes sure the journal/editor has no COI is another matter .
There are pleanty of e-mails by Jones out there. A good forensic expert could compare and make a call.
I see he spelled “fertilization” the American way – with “z” and not an “s”. But the dates are written in standard British: 20 October 2003; revised 4 February 2004; accepted 17 February 2004; published 6 May 2004.]
I’d say the comment was likely written by a British man who has picked up some American English.
Or who like me, is stuck with a US spell-checker;)
Oddly enough Jones and his spellchecker prefer both spellings.
At 05:25 PM 6/20/2003 +0100, Phil Jones wrote:
Mike,
Malcolm has just called Keith. He’s been with Ray. Apart from probably being a little miffed off he’s not on the article, he says that the W. US series in Figure 2 is wrong. He says it looks the first PC (which I said it was), but that this isn’t the corrected one (for CO2 growth effects). Can you check whether it is the right one? Malcolm says that Idso (who was on E&E) will say that the increase in that series is not climatic but due to fertilization. This would not look good obviously. Idso was on a paper with Don Graybill re fertilisation effects on bristlecones.
If you need to send a revised series for this top series in Fig 2 then send it to Tim. Tim has done this plot so can make the alterations if another series is needed. If you think that the series is OK then we’ll leave it. If you do change it will affect Fig 2 of the GRL also but probably not to any noticeable effect – at least at the size the plot will be.
Tim will send round the copyright forms to all and reprint forms. Tell Tim if you want any. Seems like the pdf will do.
Cheers
Phil
PS Tell Lorraine I’m not always emailing you – but Malcolm thought the above was important.
I assumed you would have sent the corrected one you used in GRL in 1999.
Interesting, in this email he spelled both ways–fertilization and fertilisation! ( See the last two sentences of the first paragraph).
If you have a lot of contact with another culture, you start picking uo some of their ways and customs. I’m an American living in Europe and have adopted the British way of spelling over the years.
The same has likely happened to Jones, whose British English has now become contaminated with US English. A little forensics would identify the author. They found the Unabomber by examining his manifesto.
Just to say don’t read too much into the US/UK spellings. I am in the UK and by coincidence have spent part of today trying to stop Outlook switching to US autocorrection, apparently on its own initiative. I think it changes back every time I reply to an email whose author used US settings… and of course it does it silently so if you blink you miss it.
“contaminated with US English”:
Here’s Fowler’s entry on “-ize”:
“In the vast majority of the verbs that end in -ize or -ise and are pronounced -iz, the ultimate source of the ending is the Greek -izo …. Most English printers, taking their cue from Kent in King Lear, ‘Thou whoreson zed! Thou unnecessary letter!’, follow the French practice of changing the -ise to -ize. But the Oxford University Press, the Cambridge University Press, The Times, and American usage, in all of which -ize is the accepted form, carry authority enough to outweigh superior numbers. The OED’s judgement may be quoted …. [paragraph clipped]
Many Americans write their dates that way. It’s also a U.S. military format. Good catch on the spelling difference though.
I see he spelled “fertilization” the American way – with “z” and not an “s”
The other possibility is that it’s a cut and paste from another communication (i.e. an email that is not captured in the climategate list – such as for example the one to Warwick Hughes).
These dates, which you say are “standard british”, were added by (presumably) Steve McIntyre. They are not part of the review itself. Now that we have proven that Steve is british, this leaves for the reviewer, according to you, only “the American way”. 🙂
<– this was added by someone else, probably Steve McIntyre
By George you’re right!
Good chance this wasn’t written by Jones. I’m now a sceptic.
If it was typed or published on a modern word processing application, it was likely automatically formatted the “american” way by the app. BTW Steve is Canadian, and AFAIK, Canadians are as likely to format dates the british way, just as they use “zed” instead of zero or ‘oh’.
A reductio ad absurdum, so to speak.
It seems quite likely Jones was the reviewer.
Regardless, what is most depressing is that none of the reviewers were apparently competent statisticians, and they all show little concern for science.
Especially depressing is the comment
Generally, I believe that the technical issues addressed in the comment and the reply are quite difficult to understand and not necessarily of interest to the wide readership of the Brief Communications section of Nature.
(1) Could Nature not have asked for a review from a competent statistician?
(2) The issue is of immense consequence, how could it not be of interest.
I sure would like to know if the reviewer you allude to:
‘Jones and Santer were both on the editorial board of Stephen Schneider’s Climatic Change and lobbied hard to prevent Mann from having to provide computer code to a reviewer who had requested it for the purpose of carrying out peer review. (A story for another day.)’
was someone we might describe as ‘outside’ either The Team or CA orbits, because Schneider (in ‘Science as a Contact Sport’) discusses ‘abusive’ demands for code.
If ‘ordinary’ reviewers were being stonewalled on code, then his argument falls apart.
Steve: The “reviewer” was me and Schneider at the time treated it as a serious question and referred it to his editorial board.
As a possible perspective to the question(s) at hand, the following may offer some insights to predisposed mindsets at play…
The Divine Right of Intellectuals, by David Hogberg
http://article.nationalreview.com/print/?q=MzczMzhmNjZhNzRiMWI3ZjMyOWZjMGViZDc4ODY2NjA=
It would appear that the extremely small subset world of Dendroclimatologists were possibly allowed to set the agendas and were deferred to as being the arbiters of fact…
the identification of jones as reviewer is obviously speculative.
the issue of a third reviewer seems entirely within normal practice; journals frequently send out to multiple reviewers, and only a few respond. As to whether Nature had malicious intent in sending out to (putatively) jones, that conclusion would require considerably more knowledge than is on the table. It seems unavoidable that Jones is a senior academic in the relevant field, and an appropriate reviewer.
Generally, the number of academics within a given specialist field, and publishing at high level, is small. This is further limited by the personal knowledge that nature’s editors have of referees. So small circles of reviewers and reviewed are unavoidable. If you can think of a way of avoiding this, patent it 🙂
There is a practical difficulty. If you start saying that personal knowledge, or co-authorship, is a conflict that prevents (or seriously degrades) someone from peer-reviewing, you are going to have to re-configure peer review. The whole point of much science is people meeting, getting relevant ideas from other workers, collaborating; all key process ideas in the funding of science.
There are options available such as naming particular persons that are not to review an article. That gets difficult if you claim large numbers are conflicted.
per
Regarding your comment that a third reviewer is common: this may be so, but there were not originally three. There were two, and both found the article acceptable, subject to revision. This revised version was then sent to three. Surely this is not common, unless the two original reviewers were at odds?
Per,
Multiple reviewers are no problem. What is a problem is sending the submittal to an obviously hostile reviewer and then acting on the suggestions of the single hostile reviewer.
Aircraft typically have 2 back up computers. If the pilot believes that the main computer is not operating correctly he can compare it to the other 2. Would you like the pilot flying the next airliner in which you are a passenger to always use the one computer of 3 which was not in agreement?
Wow, nice breakdown Steve. From any angle you look at it, Nature added another reviewer to insure a non-publication. Sure it sounds like a Jones review, limited discussion of the technical issues (as though he doesn’t understand) and a bunch of hand waiving.
We’re going to have to refer to you as Sherlock McIntyre, someone should send you a magnifying glass and a cool Scotland yard hat – to impress the ladies of course.
Here’s Nature’s Mission Statement (revised in 2000):
“First, to serve scientists through prompt publication of significant advances in any branch of science, and to provide a forum for the reporting and discussion of news and issues concerning science. Second, to ensure that the results of science are rapidly disseminated to the public throughout the world, in a fashion that conveys their significance for knowledge, culture and daily life.”
The last part of the statement implies that the role of the magazine is not just scientific, but a forum for “news and issues” means that it lies somewhere at the interface of science and culture and probably, although unstated, also politics and policy (“daily life”). Obviously, editorials in Nature and Scientific American take a certain political direction.
So it’s not really surprising that in a hybrid journal of this type, judgments and actions of the type described here take place, is it? It’s not dissimilar to editorial board opinions in papers like the New York Times subtly leaking over to the news desks….it’s not anything overtly malicious, but goes to what is selected or rejected, what is emphasized or downplayed. However, the result of this bias is much more important in gray areas than in black or white science.
I don’t think it’s extremely productive to dwell on this type of thing – it exists and it won’t change. Developing alternative methods of distributing competing viewpoints that are highly credible is probably a better source of progress…if it’s impossible to develop any support for these viewpoints it may be an indication that they are lacking in substance at this point in time. The web alternatives to conventional journals make it possible to have somewhat of a voice and that’s a good thing. By definition journals like Nature are “mainstream” and will largely reflect the “mainstream” viewpoint by definition.
Would the Nature editors in question have known (or cared) there was a conflict of interest, given that the other reviewers were on the Team?
Would it not be possible to ask (letter) Nature if this is true. The reply/non-reply might help.
Motion to ask Nature seconded.
Re Jones: preponderance of the evidence, yes. Beyond a reasonable doubt, no.
The problem with the review process focusing on academics within a specific field is that within a given article several disciplines may be involved, many outside the bounds of pure ‘climate science’ (whatever that is.)
We can certainly see that many climate scientists who write code are not professional-quality software architects.
The traditional ‘peer review’ process seems incapable of accommodating the need for professional evaluation of the material by each specific discipline.
I must admit that I don’t understand why Nature called in a third referee. Generally this only happens when a referee states that part of the manuscript is beyond their understanding (typically outside their field) or them to objectively pass judgment. This does happen in my field and normally the referee suggests a colleague and asks the editor if they can also have a look.
I can’t see why the Editor would do get another referee at a very late stage.
It would be interesting to try using stylometric analysis to identify the author of the review text. I’m not sure whether the text is long enough though.
Steve: I don’t think you have “demonstrated” that Jones was the second reviewer, but you have provided enough circumstantial evidence to cause someone on the editorial board to investigate. Whoever wrote the third review, Nature should be embarrassed. IMO, the quality of review is as relevant to the subject of Climategatekeeping as the possibility that Jones was the author, but you may choose to snip the following as off-topic if you wish.
More than half of the third reviewer’s comments (paragraphs 1,2&6) discuss the large number of papers already published on this subject. Despite the list of citations, the reviewer doesn’t tell the editor whether the papers he is now reviewing do or do not make an important addition to the existing literature.
“To judge that the criticism by McIntyre and McKitrick is valid would require downloading all data and applying the seemingly differing approaches.”
This is exactly why Nature should require all authors to submit all data, analysis and programs as Supplemental Information!
“Further, judgments would be needed on methodological decisions that were made by both McIntyre and McKitrick and by Mann et al. …”
A reviewer’s main job is to offer “judgments about methodological decisions”. (If that job is too difficult, the controversy should be published so the scientific community can decide.)
“It could be seen as interesting, that the calculations as done by another operator with other perhaps reasonable alternative methodologies can have such a large effect on the resulting reconstruction.”
This sentence actually sounds too open-minded to have been written by a Team member. It provides the perfect rational for publishing in a prestigious journal like Nature: The reviewer basically says that climate reconstruction, as practiced by Mann and others, doesn’t produce reliable conclusions! Paleoclimatology clearly needed to improve its analytical rigor. The reviewer somehow ignores this logic and reaches the opposite conclusion:
“I generally do not believe that this sort of an “audit” and rebuttal will lead to a better understanding of past climate variations.”
In light of Climategate, Nature should be the interested the process that produced this review no matter who wrote it.
Real Climate (http://www.realclimate.org/index.php?p=11) writes on 4 Dec w2004 about this subject:
“False claims of the existence of errors in the Mann et al (1998) reconstruction can also be traced to spurious allegations made by two individuals, McIntyre and McKitrick (McIntyre works in the mining industry, while McKitrick is an economist). The false claims were first made in an article (McIntyre and McKitrick, 2003) published in a non-scientific (social science) journal “Energy and Environment” and later, in a separate “Communications Arising” comment that was rejected by Nature based on negative appraisals by reviewers and editor [as a side note, we find it peculiar that the authors have argued elsewhere that their submission was rejected due to ‘lack of space’. Nature makes their policy on such submissions quite clear: “The Brief Communications editor will decide how to proceed on the basis of whether the central conclusion of the earlier paper is brought into question; of the length of time since the original publication; and of whether a comment or exchange of views is likely to seem of interest to nonspecialist readers. Because Nature receives so many comments, those that do not meet these criteria are referred to the specialist literature.” Since Nature chose to send the comment out for review in the first place, the “time since the original publication” was clearly not deemed a problematic factor. One is logically left to conclude that the grounds for rejection were the deficiencies in the authors’ arguments explicitly noted by the reviewers]. The rejected criticism has nonetheless been posted on the internet by the authors, and promoted in certain other non-peer-reviewed venues (see this nice discussion by science journalist David Appell of a scurrilous parroting of their claims by Richard Muller in an on-line opinion piece).
See also Climategate email 1102956436.txt in thisn respect.
This comment, given the context Steve has provided, illuminates an example of RC’s untrustworthy “spin technique” in operation. It makes one wonder how many other similarly authoritative-sounding comments of theirs are similarly “spun”?
Steve,
Although I am not an academic, I do not have a particular problem with a journal using hostile reviewers, IF they consistantly use hostile reviewers for all peer reviews. The problem that I see with Nature is that sent your submission to a hostile reviewer, but send submittals by Mann, Jones, et al to friendly or perhaps neutral reviewers. Furthermore, if a reviewer of a Mann, Jones, et al submittal takes a hostile view, Nature terminates the reviewer.
This sort of behavoir is agenda journalism not scientific journalism.
Steve,
I hate to say this, but this article – and your questions – fall a long way short of your normal objective style. I would strongly recommend that you leave this issue alone. The answers to your questions will not change the world in any way. Whether Jones was the “third man” or not in this story, Jones is going to get his comeuppance for a variety of reasons, but letting go of this particular issue may be good for your mental health! The only rational answers to the questions you posed are: 1) No. The evidence is circumstancial and would never constitute legal proof. 2) We cannot deduce unambiguously the motives of the decision-makers in Nature on the information available and 3) is a leading question which presupposes that Jones was the “third man”.
snip
Back on topic, I agree with you it probably was Jones, and Nature probably called him in so they could block the paper, and we’ll probably never know for certain. The only way we’ll get to the truth is for someone on the inside to tell us what was going on. But it’s SteveMc’s blog and he’s entitled to ponder whatever he wishes on it.
Jones email to Santer might imply he was refusing the review work, that he did not have time at present. It was most certainly not explicit, and Jones is careful with his words. I suspect at this point he had not decided whether to be reviewer 3.
If there was a time-scale for publication, an alternative reviewer ought have been sought. Judging by the delayed decision, the implication is that they took their time deciding what comments to seek and consider. Therefore it is quite possible that they would wait for Jones if they wanted his expertise, his particular “perspective”.
If Langenberg had proceeded to get the adjustments made that referees 1 & 2 had requested and then proceeded to publish, it is likely that this comment would have been contemporary with (or before) Jones and Mann 2004 published 6th May. In that context the delay in publication appears classic gate-keeping for this field.
I was surprised that referee 2 suggested “and that they make their original reconstruction (Fig 4 , bottom) available to MBH, so that these authors can also compute validation statistics with the original MM reconstruction.”
That doesn’t to seem to be “normal” climate science.
There is so much about “peer review” in climate science that reeks, it’s hard to be sure whether something really stinks, or just has that inherent makes-you-heave climate science “taint”.
I’m undecided on the conflict of interest. I like hostile review. However, with the standard number of reviews in the bag (2) to bring on a 3rd reviewer known to be hostile does present a problem. Plus we have the issue that we know Jones is prone to flauting the rules ( see his mail to mann where he discusses a review that Osborne is doing and jones wants obsborne to take on his [jones] view, while osborne wants to do the review cleanly.
As someone who passed several dozen peer reviews in leading journals (and failed a few), I concur. What seems like conflict of interest in business can be normal exchange of ideas in peer review. The journal peer review system should be able to cope with self-interested and even misbehaving reviewers. What points to an unacceptable situation is the editor’s actions, starting as you say with the solicitation of a review expected to be hostile following receipt of two solid thumbs up. Context, indeed. The misdeeds of Jones are interesting mainly as a sign of what the Team expected of Nature, and why their expectations were realistic.
We humans are subject to individual eccentricities, whether we are journal editors, reviewers or writers. It’s a dry gully to seek to unravel the eccentricities, let alone motives, unless you are working with the lawyer/politician advice “Don’t ask a question unless you already know the answer.”
My inclination would be to write to both “Nature” & Jones and ask them. Expect nothing, be happy if you get something.
Next step is for all of us to use measures – if they exist – to reduce the incidence of undesirable procedures as an altruistic contribution to the community including science. In this case, that would mean a replacement of the Board of “Nature” or a profound change to their philosophy; or the rise to prominence of a competitor Journal, which market forces might well do, but in a long time.
The inadequacy of peer review and editorial processes is important and needs publicity, but I feel that your best examples come from the hard mathematical area, not from softer hypotheses about personal motives.
However, it’s your blog and so you choose the subjects. I suspect that many other readers like me are in overload approaching shock at the scope of bad climate science and that further examples are on the diminishing return curve.
That said, the background that have have added from your early involvement and IPCC review work adds a great deal of context to the public disclosure record and is fascinating. Thank you.
My personal interest is starting to turn to the Mr Bigs behind all this. Was Heike Langenberg acting voluntarily, or was it an order? How many layers are in the pyramid, calling the shots?
Conflicts of FINANCIAL interest are more worrisome than run of the mill conflicts.
I have nothing to add to discussion direction that Steve M has encouraged here as I obviously have no hard evidence to offer. I could suppose, but I will not.
I would think a better direction to this discussion would to estimate the probabilities that a prestigious journal could impose a third reviewer to kill the publication of an article that their prejudices dictated. The academicians, who publish and review, that are replying here, I think, have answered that in the affirmative. Some have indicated that that is how the system works and that a prejudice motive cannot be separated from the editor’s intent to insure that all avenues of possible constructive criticism are exhausted.
So what is left and particularly for us laypeople with little or no experience in publishing or reviewing papers? Well, for one thing, I can read and understand the third reviewer’s comments. And I can, in my judgment, say that it was vague and obtuse and added little to the reviewing process. I am not sure that I know whether it was the third reviewer’s or the first two reviewers’ reviews that killed the publication.
But that is not the question that I am most interested in. That question is whether the peer review system can guard against some rather bad proxy papers and allow a reasonable response to the problems with these papers. My view, thanks to what Steve M has been willing to offer from the behind the scenes of the review process, is that bad papers can get published and reasonable responses can be killed.
I see some posters here who will say that Steve M has to completely join the fray and publish or attempt to publish with a fervor and that eventually the peer review system will work and if he is correct in his analyses and methods that he will prevail.
If that eventuality referred to some three center bonding involving boron, I can see that if that eventuality took a long time the consequences should not be great, but what does that eventuality mean in terms of the consequences of climate science.
Re: Kenneth Fritsch (Jan 5 19:12),
Personally I think it will all come out as part of a judicial proceeding; though given the way our legal system works these days (in the US) I’m not sure “justice” will be done. But the truth at least will come out. The question is who wants to foot the bill for filing a suit against someone. And that depends on who has standing and who stands to lose. Probably it will require something like a mandatory cap and trade system to create an incentive to go to court.
Meanwhile I’m happy to keep the warmers worried.
If you count as bad papers those that contribute nothing of note to the body of scientific knowledge, then probably more than half of what gets published is bad. I believe most authors have long ago accepted that a properly functioning peer review system will put obstacles in the way of bad papers but will not succeed in extinguishing them. The real threat to science comes when legitimate replies can’t pass the gatekeepers: that needs fixing.
I’ll play the 1-2-3 game as an entry fee to the thread!
1) Yes, it’s a decent inductive guess that this is Jones. The writing style looks similar to the naked eye, but that’s not really on the way to a proof yes. The mentioning of the Mann-Jones paper is maybe some further slim evidence – perhaps particularly the way it gets a sentence to itself.
But i don’t think the timeline is too significant. (i) The review was presumably written after the Mann-Jones paper had been published. (ii) I imagine that the Mann-Jones paper would have been widely circulated within the climate science community at the very latest by the date that it was accepted (and to some of people well before). So it’s reasonable that more or less anyone could have easily have (recently) read that paper immediately beofre writing the report.
(2) I don’t understand what went on here. Steve, you were corresponding with Rosalind Cotter as editor responsible for your submission, right? So how does Heike Langenberg get into the picture, supposedly asking for referees for it? In my world it’s one editor per paper (otherwise we’ld all get totally confused:) ).
In this kind of instance, as plenty of people have pointed out, it’s fine to have someone adversarial who will know the science (to have a motivated, sharp eye to cut out real mistakes) *provided* the editor is competent/feels powerful enough to discount obsfurcating reports.
In this case I’m simply not in a position to judge whether they were.
(3) Taking “Jones” out of the question for the moment: Did the reviewer’s point of view influence his opinion on publication?
Yes, it looks like it. But that’s part of the referee’s job. Reviews aren’t objective – in the end one is (usually, > 90% of the time) asked to say whether one recommends publication or not, given the high standards of the journal, the long list of people who want to be published & dum de-dum de-dum.
After we’ve gone through the science, if we have no substantive objectives (btw, we ask, =demand, the authors fix the insubstantive ones in the notes to them – and then send them back to us via the journal & editor for checking) we simply have to weigh our view of the work against the usual quality of what the journal publishes and what we thinks is desirable for science.
Did you see there was *lots* of relying on us all to be gentlemen / ladies and “play the game” there? Of course the system is routinely abused by sh*ts small and big (and it’s always stunning to see how pettily vicious some of the “world leaders” will be – but thankfully not all), but we just stick to the values we have (and share with plenty of others).
Too long, didn’t read: summary: it’s perfectly valid to say that the science is right but not appropriate to publish here – with short shrift to “what is appropriate”. Without knowing exactly who reviewed the M&M article, and why, I can’t answer “yes” to this part of the Q.
I think that this sort of thing – assuming that the story follows Steve’s narration – is more common than he realizes. In many sub-fields of science, there are only so many people with the knowledge to say whether a particular paper makes sense. The world of slug systematics is only so large, and the participants will be familiar with each other, one way or another. There is an assumption of good will in science that doesn’t exist in the law, so conflict of interest functions on a different level in science publishing.
Whether this particular editor sent the comment to a reviewer with the intent to get it killed is another matter. I know that my Ph.D adviser got papers to review that were written by collegues/friends, and he reviewed them as well as he could. Could a skeptical/cynical eye see hanky-panky? Sure. But good scientists like to think that the cynics are wrong.
Having paid the price of admission with my epically uninteresting views on Steve’s questions I just want to respond, perhaps tangentially, to Kenneth Fritsch’s comment abour sacred peer reviewed articles.
First point: (peer) review is essential in science.
Second point: anyone who makes competent comments is a “peer”.
Third point: not everyone is competent.
(Corollary: sometimes, with the best will in the world, we get this judgement wrong: stuff happens.)
Main point: the world climate is changing super-fast – academically. Look, here:
http://terrytao.wordpress.com/
(Updates on my research and expository papers, discussion of open problems, and other maths-related topics. By Terence Tao)
http://gowers.wordpress.com/
(Tim Gowers’ weblog – mathematical discussions)
are two of the world’s topmathematicians, both very active. I can say that because they both have recent Fields medals (more chic than Nobel prizes), Gowers in 98 and Tao in 06, and in fact they really are. (Fields medals are only awarded to under 40s, so you can guesstimate their ages.)
If you look at the sites you’ll see that they are devoted to *doing mathematics collaboratively online*. Morevover they and the people who join in are proving good things. You can join in today as well .. if you have the ideas to contribute. The “peer review” isn’t something fossilized behind some veil of referees, editors, and rip-off publishing houses, it’s your real peers – anyone who can mount a coherent argument/proof/idea.
Contrast this with certain gatekeeping – and the level of the people doing it 😦
[Snipey consideration: Ok, so Gowers and Tao’s careers are already bulletproof and they can afford, in terms of kudos, to do this. Yes! Caveat: We’re not living in M.Pangloss’s ideal world yet: it isn’t so clear how to integrate career structure for those not secured /still developping/risk-averse into the plan. This is structural work still to be done!]
It seems to me that the most important issue is not whether Nature passed on the comment to Jones, but rather, why the devil they resorted to a third reviewer to begin with. That seems inappropriate in and of itself. The initial reviews seem pretty straight forward. It actually appears that Nature’s editorial staff deliberately sand bagged the discussion until there was a procedural excuse to not publish it. My own bias in peer review is to prefer the _Current Anthropology_ model where reviews are not anonymous, and not only the reviews but response from authors are attached. Nature might do well to look into this as their reputation as a key technical publication is on the line.
I think the speculation regarding the identity of the third reviewer is below the usual standards of the site. Already some posters have decided who the reviewer is and are now speculating on the ed’s motive.
Regarding point two, I too find it unusual that a third reviewer was enlisted since there was agreement amongst the first two. However the assumption that the ed was deliberately sabotaging the comment is also speculation.
I think it entirely likely that the rigorous statistical treatment by MM intimidated the ed and reviewers alike and being ill equipped they called for backup. Despite the the opinions of some here, many scientists are not PhD statisticians.
Unless job requirements call for advanced mathematical chops many will be unsure of the most appropriate statistical analysis esp if there is no one clear choice.
IMHO the prob with the Mann etal is that they have not acknowledged the limitations of their reconstructions and allowed for even the possibility of alternate explantions. How they have come to believe that they have definitively reconstructed global temps with the scant data available is beyond me but their cock-sure pronouncments have invited the backlash.
i second all of that!
Steve, I think you demonstrated very well what went on. It might not hold up in a court of law, but perhaps it will make such deplorable behaviour less common in the future. Well done!
You wrote that “Jones and Mann (2004) on matters M&M – an article that was not even published until May 6, 2004”
And it seems you received the “third” review only in August 2008.
But in the meantime, anyone could have read Jones and Mann (2004).
Am I missing something?
Steve: The reviews were received in August 2004. Jones has a tendency to self-cite in his reviews.
(Perhaps you meant August *2004*, but that would still be several months of time in between.)
Steve, your reply seems to miss my primary question. It is whether by August, anyone could have read the article published in May.
Re: Norbert (Jan 6 11:35),
I am not following this.
Are you rebutting Steve by challenging the reading ability of scientists?
WillR, the question is whether Steve has any information that the review by the third reviewer was written before August. I didn’t notice any such information. To me, it would seem that anyone could have read the article, so that, yes, what he called “another strand of evidence”, would not be evidence at all.
By the way, Steve corrected “August 2008” to “August 2004”, the date following the phrase “but didn’t receive a
decision until” (without marking that fix of a typo in the text itself).
Steve: I noted the correction in the comments.
(Of course, if it was someone else, then it wouldn’t be “self-cite”.)
I was struck by this statement from the reviewer: “Generally, I believe that the technical issues addressed in the comment and the reply are quite difficult to understand and not necessarily of interest to the wide readership of the Brief Communications section of Nature.”
Determining what interests the “wide readership” is the providence of the editors and not the reviewers. The comment suggests a too familiar relationship between reviewer and editor. If such a cozy relationship exists, it’s not surprising, given Nature’s recent editorial castigating those who disagree with the ruling elite in climate science and their use of the pejorative “denialists.”
As a librarian, I follow this discussion with interest. After reviewing their online Mission Statement I would be compelled to cancel my subscription to Nature Magazine.
snip – overreaching
Might I suggest a simple “trick” (a clever method) that is often used to detect plagarism, or source of authorship in the university arena…
Note that the author in question uses certain words to begin, and qualify, his/her sentences, combines phrases in a particular manner, and tends to structure comments in a manner easily recognized, and hence easily tracked.
Starting with “Generally”, “Unfortunately”, “Further”, and “It seems” could be a smoking fingerprint. Like tree rings, writing patterns can reveal much… 🙂
The point is not to identify the author to crucify, but rather to put all the pieces of the puzzle together for a comprehensive total picture.
The fact that Nature took 4 years to get back to you shows a conflict of interest. – snip
Steve : It was August 2004, not August 2008. My typo (though the typo should have been obvious by the context,) Sorry about that. Fixed.
The email informing Steve appears to be dated August 03, 2004. I’d think Steve made a typo (or confused something) when he wrote “August 2008”. That would be a few months, then. (Unless the typo/error is in the email header.)
“To judge that the criticism by McIntyre and McKitrick is valid would require downloading all data and applying the seemingly differing approaches.”
Oh!
Would it require standard scientific scrutiny?
God forbid!
My guess is Steve is just keeping the pot simmering and that more important issues are around the corner. Conflicts of interest must ALWAYS be declared.
Roger Knows Now….
————————-
Return to the index page | Earlier Emails | Later Emails
From: Phil Jones
To: “Michael E. Mann”
Subject: Re: Fwd: RE: IJOC040512 review
Date: Fri Aug 13 13:38:32 2004
Mike,
I’d rather you didn’t. I think it should be sufficient to forward the para from Andrew
Conrie’s
email that says the paper has been rejected by all 3 reviewers. You can say that the
paper was an extended and updated version of that which appeared in CR.
Obviously, under no circumstances should any of this get back to Pielke.
Cheers
Phil
At 08:11 13/08/2004 -0400, you wrote:
Thanks a bunch Phil,
Along lines as my other email, would it be (?) for me to forward this to the chair of
our commitee confidentially, and for his internal purposes only, to help bolster the
case against MM??
let me know…
thanks,
mike
At 03:43 AM 8/13/2004, Phil Jones wrote:
http://www.eastangliaemails.com/emails.php?eid=424&filename=.txt
Steve: Remember to distinguish McKitrick and Michaels 2004.
Both -1ze and -ise are common, acceptable (in most quarters) British English. In theory a writer should stick to one or the other in a given piece but many of us flounder especially if prodded by a draconian spell-checker.
“…verbs containing the Greek suffix -ize. (…) These words must be spelled with -ize in American English. In British English, the spelling with -ize is traditional, and is still preferred in many conservative quarters, for example at the Oxford University Press. But the newer spelling in -ise is now widespread in Britain and is preferred in other quarters. ”
‘Mind the Gaff. The Penguin Guide to Common Errors in English’
January, 2010 ecopy of Physics Today…has an article on the “Hacked Climate Emails.”
….they report, “report of a conversation with a PR operator for one of the world’s leading environmental organizations who didn’t want to be quoted. He said he believed that they were on the verge of convincing the general public that they needed to make sacrifices in order to stop AGW, but all that ended on Nov. 20. “The e-mails represented a seminal moment in the climate debate of the last five years, and it was a moment that broke decisively against us. I think the [Climatic Research Unit (CRU)] leak is nothing less than catastrophic.””
Can someone with access to that issue of Physics Today get the URL reference and send it to the Wikipedia Talk page for Climategate? They refuse to acknowledge the possibility that this is a scandal, and it might help if one of their number, equally devout, could confirm that. Not that I have much hope. But it seems like a necessary thing to do.
Re: snowmaneasy (Jan 6 03:12),
Interesting to think of this now in 2013, although OT to the thread.
In regard to Steve’s 3 questions,
I say yes, yes, and yes. I’m not an academic. Such blatant conflicts of interest need to be openly declared and understood. In the clashes of ideas and argument, of course it is healthy to examine critical views, but it is also crucial to take account of how intellectual cliques and alliances can operate. I see nothing wrong with an editor wanting to understand the different sides more fully before a decision to publish, but using a highly interested (conflicted) party to derail a process that had previously seemed likely to result in publication (based upon Joliffe and Zorita reviews) seems abusive of sound process. Sound judgment includes knowing how to weigh evidence and arguments from a highly interested party such as Jones. Nature editors did not display sound judgment.
Whoever the reviewer is (my bet would be on Jones), isn’t the following an admission that he or she is unqualified to review the paper (although it’s put forward as a criticism of the paper) ? :
“Further, judgments would be needed on methodological decisions that were made by both McIntyre and McKitrick and by Mann et al. as two possibilities within the whole spectrum of methodological decisions on which chronologies to use, the calibration and computation of PC’s over different time periods, special treatments to series, and so on.”
The review could be summed up as “Sounds interesting, quite difficult subject matter, needs skilled reviewers to determine merit, some ideas have been mentioned before – too complicated for the simple readers of Nature? Thumbs down.”
Re: oneuniverse (Jan 6 05:10),
Of course, it’s absurd. The whole comment is about methodological decisions.
So it is either absurd, or it refers to the reader rather than the reviewer. Meaning the reader is not put in a position that allows a judgement (without downloading all the data), or even relate to the judgement made by the authors. In which case the comments are of a more subtle meaning than we would expect based on the picture painted of Jones. We would expect Jones to just point out the wrongness and misfit of those claims, wouldn’t we?
Norbert, you’re referring to a different criticism, and it’s one that the reviewer could have levelled at any paper not containing all the data under its consideration – ie. most papers involving data would fall foul of this ‘criticism’.
Steve, general, sorry OT.
I’m writing a piece for a UK monthly called Prospect on the importance of Climategate. My thesis is ONLY (in order to confine it) that CRU’s critical job is temperature record recreation and monitoring, and if we can’t trust that absolutely basic first pillar of AWG theory, then we are nowhere.
Who could I send it to for checking for basic errors (peer review) so I know it’s factually accurate?
After objectively reading the submission and the comments I’m now not at all surprised they asked for another reviewer. As the 3rd reviewer was even more vague, lazy and indecisive as as the first two I’m not surprised either about the final decision. All three reviewers seemed to just ignore the important game-changing, icon-status of the hockey-stick graph and therefore the great need for it to be scientifically accurate. That’s a dilemma I’d kick into touch too if I were an editor. Your submission really needed less lazy reviewers. That’s very common too though, since they aren’t remunerated for it. What a pathetic system!
Ladies and Gentlemen,
This is a call to arms (ok, perhaps a bit over-dramatic, but this is climate science). The people of Northern Ireland, due to the acquiescence of a weak media, are totally unaware of the ‘Climategate’ scandal. It’s almost as if it never happened.
I managed to get a letter published in the Belfast Telegraph (BT) today regarding ‘Climategate’. When one searches the BT website, the only search result for ‘Climategate’ is my letter. That, surely, is a disgrace.
Therefore, I would be very grateful if some of the regulars on here could also do a bit of commenting on the BT website. I just don’t want the people of NI to think that I am some sort of crazed, lone nutter I promise, I am not!
Your help will be greatly appreciated. Here is the link – http://www.belfasttelegraph.co.uk/opinion/letters/global-warming-argument-not-the-only-view-14623971.html
By the way, the comment box only allows 1000 characters, so keep it short and sweet, please.
Cheers,
Will Hawkes
Re: Will Hawkes (Jan 6 08:26), I commented a few hours ago, but nothing posted yet.
(1) that I’ve demonstrated that Jones was the added reviewer (2) whether Nature intentionally added a reviewer with a known conflict of interest; (3) whether Jones’ adverse conflict of interest interfered with an objective review.
1) There is a good circumstantial case.
2) Who knows…
3) Assuming it is Jones — COI is strong enough that it could preclude an objective review.
As for point 1) — some people think that circumstantial evidence is not enough to convict — there are probably enough people in jail based upon circumstantial evidence to cause people to re-think this one. Some of them, probably most, are guilty too…
And then there is point two… how many reviewers would dare be objective if they wanted to publish again?
Too many confounding ideas at work.
The review looked to me more like a group exercise. I can’t offer proof — it’s just a feel. It has too many ideas — it’s not coherent, and it works on the balance theory. i.e. throw as many ideas on the pile as possible and hope to overwhelm the opposing viewpoint.
Mr. McIntyre,
In answer to questions:
1) yes 2) yes 3) yes
Obviously, not much I can add rhetorically, but I am a relatively intelligent and educated individual and would like to give my moral support!! 🙂
Dan.
New Jersey, USA
Would Jones really refer to the Nature submission as ‘a paper’, and not give more detail?
I think he is referring to another paper.
The third reviewer may have gotten input from Jones or Mann when doing the review.
Maybe it’s Gavin.
Relevant issues here are:
1) Since the first two reviewers were in agreement there was no need to seek a third “tie-breaker” review. Purpose of recruiting third was only to dig up dirt.
2) Third reviewer was likely Jones but whoever it was could only muster peripheral, quasi-political comments. So the review should have been ignored for lack of substance.
FWIW:
Much to my frustration, esp. during years of doing translations in Switzerland, the Word spellchecker accepts both ise and ize even when U.S. (or U.K.) English is specifically requested. My husband often made the first draft and had the tendency to use British English.
While very rarely, I myself have used both endings!
Another thought on the following:
(2) whether Nature intentionally added a reviewer with a known conflict of interest;
Perhaps Steve’s viewpoint is the one that presents a COI (to the journal(s))! Consider the following quote:
“We know from the climate debate that when scientists publicly disagree it’s not good for governance, but when scientists agree, that’s a recipe for making change happen,” he added.
Form Here:
http://www.timesonline.co.uk/tol/news/environment/article6734078.ece
AGW generates a lot of journal articles and a lot of research dollars and a lot of classroom dollars. Debunking the AGW hypothesis is dangerous to scientists!
There is some logic there. I hardly know what to think of it.
“Further, judgments would be needed on methodological decisions that were made by both McIntyre and McKitrick and by Mann et al. as two possibilities within the whole spectrum of methodological decisions on which chronologies to use, the calibration and computation of PC’s over different time periods, special treatments to series, and so on.”
As a matter of peripheral interest, how are we supposed to interpret the above comment from the third reviewer? Many different methodological approaches can be properly used and they will all give the “correct” answer? Or judgments are needed to decide the best/correct approach and I, by not, providing one would indicate that I am not capable of making one? The first interpretation would appear to require the reviewer to be capable of comprehending all possible approaches while the second would call into question why this reviewer was reviewing this paper.
More vague to me than the review comment was Norbert’s comment below on the review comment. It could just be me and my POV, Norbert, but I have difficulty comprehending some of your comments here at CA – and I reread them.
“So it is either absurd, or it refers to the reader rather than the reviewer. Meaning the reader is not put in a position that allows a judgement (without downloading all the data), or even relate to the judgement made by the authors. In which case the comments are of a more subtle meaning than we would expect based on the picture painted of Jones. We would expect Jones to just point out the wrongness and misfit of those claims, wouldn’t we?”
Can I interpret this as meaning that both Mann and MM have written something that the reader is not in a position to judge and therefore Mann’s published writings stands without reply from MM?
Are you implying in last part of your comment that Jones has shown to be vis a vis the “emails” not capable of being vague and obtuse or nuanced- when the situation calls for it. I would disagree with that observation.
Re: Kenneth Fritsch (Jan 6 12:20),
If this is a call for simpler more direct language — I concur!
Asking rhetorical questions when a simple statement of the facts might do equally well takes time and bandwidth as well.
First, how come you comment is able to reference my comment Posted Jan 6, 2010 at 11:26 AM ? It seems yours was written earlier. Are you able to edit it later-on, or is there something strange with the “Posted” date and time ?
Why would you interpret my comment as having anything to do with Mann’s article? That is neither the review, nor the reviewed text, which we are discussing here.
To put it in different words, my comment said that if one doesn’t interpret the review as absurd, I’d take it to mean that the reviewed text doesn’t offer the reader an understanding of the judgements which went into the reviewed text. That might something to think about, and probably can’t be explained better without taking more “bandwidth”.
I disagree – the reviewer says, unambiguously, that in order to verify (not read) MM’s criticism as valid, the data would need to be downloaded and the necessary knowledge available to discern between good and bad methodologies.
eg. Mann’s centering of his proxies on a decentered low mean causes his method to tend to select hockey-stick shapes as PC1 of his PCA (I rely here on the work of MM and the verification thereof by Wegman’s committee, and others) : the ‘verifier’ should be able to determine that Mann’s use of this advised-against method is undesirable (advised by the literature concerning the use of ‘decentered’ PCA).
It doesn’t say “verify … as valid”, it says “judge … as valid”. I take this to refer to the reader not being able to make any (even somewhat superficial) judgement about whether or not this criticism might be an agreeable criticism. Actual “verification” would always require a more elaborate process. I.e., the reader wants to be able to say: “Yes, that sounds like it makes a lot of sense” vs. “Well, it might make sense” vs. “I wonder if that makes any sense”.
To me it appears that this is a general theme in the review, with an arch over to the sentence: “Generally, I believe that the technical issues addressed in the comment and the reply are quite difficult to understand and not necessarily of interest to the wide readership of the Brief Communications section of Nature.”, which of course directly refers to the reader.
Otherwise, those earlier comments about judgements wouldn’t make sense to me. It would be too absurd to imagine the reviewer discussing his/her own limits of understanding or lack of time to download data under the label “Remarks to the Author”.
Re: Norbert (Jan 6 20:47),
Norbert:
I agree with you — really! Steve’s review was pretty heavy going — I work in NP problems (touch on stats a bit). Also, I found the third review equally heavy going. Is Steve’s review of “General Interest”? It should be. It probably isn’t — that would be the problem in my mind.
Wegman made the comment that there should be more Statistical Mathematicians working with the Climate Crowd. Steve has made the point very well – in various ways.
Yes it would be nice to have a simple explanation — as in: “This is the deficiency, It’s shown by this statistic, this is what the problem must be in the data etc…”
Perhaps the real issue is that a lot more statistical expertise is required in the design of experiments, and the design of data collection programs. This was beaten into me in the undergrad programs and the scars have never healed. 🙂 …Then when you collect your data — test your tests, then modify data collection or your testing program as required, or perhaps deal with why the answers don’t match preconceptions. In other words — the scientific method.
etc.
Norbert: “It doesn’t say “verify … as valid”, it says “judge … as valid” ”
The method the reviewer describes for the reader to judge the validity sounds like a verification to me : replicating the work using the same data and approaches & evaluating the soundness of the methodologies.
For our contemporary reaction to the Nature reviews, see Ross here
Steve:
This is an interesting additional piece of information. At the time, who did you think were the reviewers? The contemperaneous identification of Jones by either or both you and Ross would certainly be additional support for your current argument.
Steve: We thought that it was probably someone at CRU. Indeed, that was my hypothesis when I looked at Climategate emails for contemporary evidence and was one of the reasons why I interpreted the email in the post as I did.
Well IMHO even though it is ex post facto that suggests a one tailed test and a greater confidence that your surmise is correct – Climategate letters confirm an existing hypothesis.
Reading the second round of referee comments, linked from Ross’s page, it is clear that one of the first round referees (who you say was Zorita) is less enthusiastic about the second version (“Considering the changes relative to the first version of MM04, it seems to me that the case presented by MM04 has weakened considerably”). So the August rejection was not based entirely on the third (“Jones”) referee.
I think you just have to accept that getting anything published in Nature is very difficult and something of a lottery.
Steve: As I said in the first comment, I’m not trying to re-litigate the process but to examine the Jones Review. (For what it’s worth, Zorita subsequently expressed his regret for his 2nd review, explaining that he had not fully understood the Mann situation at the time.) Once again, I discern somewhat of a cultural difference to the handling of conflicts of interest. Academics attempting to justify acting in a conflict seem to look for whether it was possible to get to the same answer in a different route. And in many cases it is. As I noted before, in a business/legal situation, this sort of argument isn’t permitted. In the case at hand, Nature cited the reasoning of the “Jones Review” in their rejection and thus it is reasonable to deduce that the Jones Review affected their decision. This isn’t to say that they couldn’t have got to the decision a different way; on the other hand, with just the Jolliffe and Zorita reviews, they could also have proceeded. One doesn’t know – because the Jones Review proved to be the one that they cited in their reasoning. While the business/legal framework is a bit foreign to academics, I urge that some thought be given to it, if it’s not natural to you, because conflicts of interest have been thought about and litigated much more in business/law than in academics.
Re: PaulM (Jan 6 13:00), Hear hear.
That was referring to Steve’s reply, for clarity.
OT, but Jeff at the Air Vent has an interesting comparison of urban warming bias based on CRU data… http://noconsensus.wordpress.com/2010/01/05/cru-3b-urban-warm-bias-in-ghcn
If it’s OT, then why post it?
Good question Mark but it is a darn interesting article and the kind of thing that should be done independantly with the original data.
Steve, while your readings of the situation are probably the most natural, there is an alternative interpretation which you have not considered, namely that your original version was sent to all three reviewers, but reviewer #3 did not reply at this stage. You revised version was then sent back to the original three reviewers, and now that reviewer #3 saw reviews 1 and 2 he decided that something had to be done. I have seen editors use phrasing similar to that used here, only mentioning the reviewers who have replied.
some interesting correspondence online in nature correspondence, from von storch and others.
“we welcome debate about the ethics of science…” is von Storch’s.
There is also another letter, which addresses the issue that CRU systematically attempted to prevent access to their data.
all good stuff !
per
The masks keep slipping off. Now IPCC Chairman Pachauri has been discovered in a massive scandal involving expenses and conflicts of interest.
http://eureferendum.blogspot.com/2010/01/pachauri-smoking-gun.html
http://eureferendum.blogspot.com/2009/12/pachauri-basques-in-glory.html
A couple of key links, but more than a dozen other stories about Pachauri and his vast nexus of businesses that profit directly from his recommendations as IPCC Chairman.
Best regards, and that you for your work on this blog!
I’m leary of the “conflict of interest” argument here. An hostile reviewer doesn’t necessarily constitute a conflict of interest. In the fields of expertise discussed, a sympathetic reviewer would seem to pose more of a conflict of interest. For instance if Mann had reviewed Wahl and Ammann there would be an obvious conflict as they have built on each others’ work.
There may be a conflict of interest between Jones and Nature as Nature seems to champion the cause of Jones (and Mann). So if Jones were the reviewer, it would be appropriate to consider why Nature would bring him on. However, I don’t see a conflict of interest if Jones reviews M&M in an impartial publication. And I suppose that’s partly what we are trying to discern, was Nature impartial?
In the Law, in which I have some experience, the question is whether there exists a conflict of Interest and Duty, not just a conflict of Interests. If a man owes no duty to prefer a particular interest then he can embrace as many conflicting interests as his capacity for anxiety allows. The important question is: who’s interests was it the duty of Nature, or the duty of the reviewer, to act in? And what interest did Nature have, or the reviewer have, which stood in conflict with that duty? That question is at the heart of scientific ethics.
Re: JT (Jan 6 20:12),
Very succinct and to the point.
I would say that if your duty is to the truth then you will be eternally at conflict.
Anyway — Wikipedia does have some good sections on this and would be helpful if you want to understand the legal concepts.
http://en.wikipedia.org/wiki/Conflict_of_interest
See Standard of Care and Fiduciary Duty as well.
This might help a lot of people understand why Steve is so perplexed with the confusion in the Scientific World. 🙂
I think it is a lot more clear in the business world. My $0.02
Steve: Good reference. I’m glad that you understand the context of how I’m trying to understand the issue.
Exactly.
Re: WillR (Jan 6 21:47),
Yes, the Wikipedia summary is pretty good.
There’s a huge difference between conflicting ideas and conflict of interest.
Any vigorous discussion involves conflicting ideas.
When a party on one side of a disagreement can take advantage of their position of influence to their personal or corporate (group) benefit… then they are in a COI position that should be revealed, recused, etc.
It is difficult: knowledgeable parties to an active area of disagreement will obviously benefit to some extent if their position prevails.
What’s clear in this case is that one Team of players is overly familiar, overly influential, with the editorial staff of these journals. To properly deal with a situation like this, in the business world:
a) We’d seek diligently for reviewers who are more disinterested;
b) We’d require disclosure of these potential/real COI’s;
c) We’d seriously question why such a vitally-interested party did not recuse themselves.
Seems to me that network-relationship analysis such as done by Wegman could become an ongoing database maintained for the purpose of identifying and making use of knowledgeable reviewers who are least-involved already in any given contentious issue.
In fact, NSF maintains such databases, and for exactly this purpose of determining where there is a CoI … in granting.
I know little about how “conflicts of interest” are handled in the academic world. And agree with Steve that the group seeking to depress his views were guilty of a conflict of interest. However, the problem, as I see it, stemmed less from the fact that different individuals (in this case, the editors and referees at Nature) acted in a way that “conflicted” with the “objectivity” of science (properly understood) but rather that Nature has failed to manage this conflict.
In my view, conflicts of interest define the human condition – and are a dominat element in any organization. The challenge is not to eliminate such conflicts but rather to manage them.
At CEI, we’ve long been interested in how the “conflict of interes” is being used to suppress debate, to limit the voices that are allowed to be heard on a specific policy issue. The same result as that suffered by Steve but from the opposite direction. That is, policies which seek to eliminate conflicts of interest can be damaging as Steve experienced.
I arrived at this conclusion from my experience in the policy world (where I do have expertise). In the composition of advisory boards, selection of witnesses for commissions or hearings, decisions on which groups should be consulted prior to a policy decision and in other areas, there is growing effort to argue that “conflicts of interest” are somehow “unnatural”, that objective truth comes only from “disinterested” parties. As one of the comments in this thread noted, the result would be that only those having no knowledge or interest in a subject could provide advice.
Of course, it is true that people rarely enter a debate with no priors. Moreover, it is very difficult to move an individual from those priors (see Thomas Kuhn, “The Structure of Scientific Revolutions”). The question is whether those priors (or “biases” if you will) preclude that voice from being heard – or whether those with bias use their positions of influence to block other voices.
To prevent that result, it is not necessary (indeed, I would argue, impossible) to eliminate conflicts of interests but rather to discipline the role they play in the decision process. These are less scientific than institutional issues. Every organization must manage the natural conflicts of interests within itself. For example, within a business, the marketing department may push for a wide variety of products to maximize sales; the production group for standardization to allow 24/7 production runs; the finance group may seek high quarter gains to reduce borrowing costs. It is the task of the CEO is to reconcile these “conflicts” to ensure that the joint effort is sustainable profitability. In economics, the “principal/agent” problem addresses this issue. Conflicts of interest arise both within the firm and in its dealings with others.
This same problem exists also (and at a greater extent) in the policy world. A government agency creates a review panel to examine a proposed regulation. it seeks advice and solicits “experts” for that purpose. So far so good but increasingly the view has arisen that only those with no economic links can provide such “objective” advice. Individuals working for a company or having served as consultants to one are viewed as “biased” and should either be excluded from the process or be forced to don a scarlet dollar sign. The goal of many raising the “conflict of interest” concern seems to be to drive the market out of the marketplace of ideas.
But, in our interest group democracy, there are two types of “interest groups” – economic and ideological. If one excludes or downgrades the advice of the first, then one receives advice largely from ideological groups alone. Only the disinterested academic, the intellectual can be trusted. The role that intellectuals have played over the last century (note the reference in this thread to the recent book by Thomas Sowell, Intellectuals and Society) suggest that this is a very bad idea. That tactic is suggested by Mann labeling Steve as “in the mining industry.”
Business, at least, has a clear goal in managing conflict of interest. it is less clear how groups such as Nature or EPA resolve this question. It is that aspect of this problem that should concern us.
CEI, as an analytic/advocacy group, has a clear point of view. we favor liberty with responsibility – the classical liberal approach. We, like all other groups of our type, must manage the enthusiasm of ourselves and our staff that can run ahead of the facts. We try and I hope we succeed but it is not an easy task. We’re disciplined – as are most non-economic groups – by our integrity (of course) but, perhaps, more importantly by the need to maintain our reputation as a principled free market group. Credibility to these values, after all, is the primary value we bring to the policy process. Others must judge whether we’ve been successful or not – as must others decide about the ways in which Nature has addressed this issue.
In summary, however, let me repeat: I do not think we should seek to eliminate conflicts of interest or, indeed, even to view them as somehow “wrong” but rather to examine the institutional setting within which these conflicts occur and examine whether the institutional is effective at managing them. EPA, too many universities and (as it appears) Nature are failing this challenge.
Fred Smith, CEI
You are exactly right. It is not the bias of a reviewer per se, but the type of bias that leads a reviewer to try to prevent publication of anything remotely critical of his work or that of his friends that is the problem. It is the job of the editor to manage this bias, but when the editors share this bias, then entire points of view are not allowed to be heard. This is our current situation, and it smells.
So if control can be gained of the literature, opposing points of view can be managed, and further, criticized for not being a part of the peer reviewed process.
I think that it is this latter part that is worse in the internet age. Steve’s made an excellent example of this blog: different points of view can be publicized – outside the literature. But, he can’t get around the fact that the majority of his work isn’t “peer reviewed”.
I do think that a better question is whether the editors have a bias. From my understanding of the history of science either time (death) catches up with biased gatekeepers or else a preponderance of data eventually changes minds. In order for this second to occur, either 1) there would need to be many more Steves 2) Steve would need an excellent marketing department or 3) temperatures and other climate signals continue to show cooling.
Along the lines of 2) Obtain a grant(s), start a journal, send out press releases. Simple, right?
Pursuant to Craig Loehle’s point (1/6/10 @ 10:05pm) —
It’s often very unusual for uninvolved parties to get an up-close view of how established procedures play out. This case concerns peer review–review by Nature of a contentious submission in a controversial field with looming importance to societal policies.
I do not think that many interested people would have much of an idea of what such peer review would look like, both when it performs well and when it does not.
The back-and-forth email correspondence on this matter (link in the main post) is revealing in that regard.
Advocates should emphasize the term “the peer reviewed literature” when arguing their case, as they do. It follows that they should welcome greater transparency as to what this process actually entails. Spectators–by which I really mean “citizens”–should have a clear and realistic view of its strengths and weaknesses.
Thus, it’s very useful to have this record discussed, and then archived on the internet.
(1) Yes
(2) Umm, that would be, yes
(3) Yes, actually
OT:
I propose a name for the whistle blower: Sore Throat.
(1) that I’ve demonstrated that Jones was the added reviewer
Yes, by a “preponderance of evidence”. I don’t know if it’s “beyond a reasonable doubt”, yet. Speaking as a juror, not as a lawyer.
(2) whether Nature intentionally added a reviewer with a known conflict of interest
Hindsight, including, for example, the social network analysis in the later “Wegman Report” shows a conflict of interest. “Intentionally”? “Known conflict”? One alternative could be “fell for a trick”. The “community” was pretty tricky about pretending to be just group of professionals who, of course, knew each other but were independent from each other, thus able to render unbiased reviews of each other’s work products. I’m not defending Nature.
(3) whether Jones’ adverse conflict of interest interfered with an objective review. Yes. see just above under (2)
The Heike Letter states: “Regarding confidentiality, you should please refrain from publicising the decision prior
to publication of the Correction (we are asking the same of Michael Mann and his co-authors).”
Did all parties refrain from “publicising the decision”?
Steve: I did. Mann breached the confidentiality terms by sending his Response to Stephen Schneider, who published it as his website. We formally objected to Nature about the breach of confidentiality but they didn’t care.
Re -ize/ise:
suffice to say my copy of OED refuses to even recognize the latter option.
I would say that the matter looks like Nature definitely wanted to reject the paper and ensured a suitable review to meet that purpose, so my answer to question (2) is yes.
The review itself is carefully written not to show any conflict of interest. It is meant to sound objective and taking concern about Nature policies. It does not disprove the results, it stresses that the comment is unsuitable for Nature. This is a very remarkable tactic likely practiced by experienced reviewers who must know what the Editor needs. I tend to agree with theduke’s Jan 5, 2010 at 10:02 PM vision.
We recently had a small comment published in Nature. There were three reviewers, two positive and one negative. This negative review took exactly the same strategy as the hostile “Jones review”, not objecting to the essence of our contribution but questioning the validity of our contribution for Nature:
“I am pleased to see a contribution from Makarieva et al., on the Head et al piece, given that the heart and soul of the snake paleothermometer are derived from Makarieva’s work. These authors seem in this contribution to be stepping back from their claim that there is a universal value of alpha (~.3) and instead encouraging Head et al., to use a value of 0.17 from the older allometric study of Chappell and Ellis. Using this other calibration leads to a change in estimated MAT, but not a fundamental change in the main conclusion. Since this amounts to a recalibration (which is always an ongoing aspect of progress in science) and not a major flaw or a major step forward I highly recommend this contribution for a specialized journal rather than this venue.”
In M&M’s case, Nature could have refrained from sending the first two reviews to the authors upon receiving them, but involved a third review immediately and ensured that to be negative, then presented the authors with the bunch of three reviews and a rejection decision (because, as we know, the competition for space in Nature is very high and even very good papers do not pass, unfortunately) — that would be “in order”. So I agree that something went wrong with this scheme, and the Editor was surprised to have received two positive reviews and obliged (perhaps by the formal rules) to send them to the authors.
Regardless of what Wahl and Ammann did or didn’t do, is it fair to say Schneider ignored your review? Did he not forward it, ask for a response, etc?
Steve: There’s a thread on our Wahl and Ammann review. Take a look at it and take a look at the paper. Then try to tell me that he didn’t ignore the review.
I’ve read the thread. I guess we define ‘ignore’ differently.
Just because your review didn’t effect the final draft, doesn’t mean the review was ignored.
Steve, it occurs to me that the good work done here by you and others SHOULD be of interest to the various people investigating Climategate at the CRU and Penn State.
Just in case they might be tempted to “miss” this valuable resource, would it be prudent to use some public forum to volunteer your help? At a minimum, you could offer to provide verifiable factual information developed from “raw” data.
Do you think they’ll appreciate the help?
Haven’t read all the comments, but it does seem that your post is completely wrong, and no-one’s pointed that out.
The chronology you’ve reconstructed seems to look like this.
Early March
Heike Langenberg asks Jones to review a paper; Jones says he is too busy until May
Rosalind Cotter asks McIntyre to revise and resubmit for publication in some form
Late March
McIntyre resubmits
Jones writes an email reaffirming that he is unwilling to review until May
Late March?
Nature (Cotter?) passes McIntyre’s resubmission to one extra referee
August
Cotter rejects McIntyre submission as not suitable for brief comments
What in this chronology suggests that Jones was the extra referee? I see two major reasons to think that Jones coudld NOT be the extra referee.
1) People. These are two different editors. It may be that Cotter and Langenberg work sufficiently closely that they request referees for each other, but you’d have to give some evidence.
2) Timing. Why would Langenberg make a review request in early March for a resubmission that hasn’t come in yet?
Cotter’s language “publication in some form” seems like a clear marker that she doesn’t think this piece works for Brief Communications. All three reviewers you link to say pretty much the same thing – highly technical issue of interest to specialists only.
You’re protesting too much on this. Seeing conspiracy theories when you just submitted to the wrong journal. Get over yourself.
Re: Phil Hand (Jan 7 10:40),
Phil Hand wrote: “it does seem that your [SMc’s] post is completely wrong”
You go on to say that Jones might be reviewer, but we don’t have the evidence to say for certain. Which is in agreement with the post you describe as “completely wrong”.
PH: “highly technical issue of interest to specialists only”
The ‘hockey stick’ was pushed onto the public by the IPCC and the attendant MSM, and used to inform national and international policy decisions. Therefore, while the detailed working of the MM paper may be highly technical and specialised, the results of the paper are of great general interest and relevance.
My comments here are not limited by a context of a juror receiving instructions from a judge on how I must decide in this case. What I like about this situation is that we have more transparency here then would normally be the case – by way of the “emails” and Steve M’s historical evidence provided here. We can all judge for ourselves where the evidence points and it need not pass some legal or even ethical test. We can view here a pontification of what the system is and what it should be, but what is important to me is what do these “real life” cases say about the system and whether the holy grail of the IPCC, in the form of the sacred role of peer review, holds any water, holy or otherwise, in documenting probabilities of AGW, its levels and effects.
What comes through to me is that that Nature was no doubt very leery of publishing an article, letter, reply or whatever of the controversial and minority status of the Mc and Mc submission. To get it wrong with a publication of an article like Mann’s that agrees with the consensus view is nowhere near the risk that getting it wrong on a controversial reply that could be construed as going against the consensus would be.
Nature, therefore might well have upped the requirement ante considerably for Mc and Mc. And is not that what they did by steps in the requirements with first some indecision on what category of publication to consider the Mc and Mc submission, then two reviewers went to three and the word limitation went to 800 and then to 500? The third reviewer, who ever it might have been and it is not important to my point here, obviously had nothing in the way of technical judgments to offer but did put forward the thought hinted at by the other reviewers that the subject was too complicated and technical for the now limited 500 word reply. Those were the magic words that Nature needed to get this monkey off their prestigious backs.
Since none of this is going to send any one to jail or even put a dent in their careers, I suppose one can conjecture about how those magic words appeared out of the mouth of the third reviewer, but I will not.
I can and will conjecture about the vague and abbreviated reviews I read from not only the third reviewer, which was particularly bad in my view, but all three reviewers. I was not at all impressed with the tenuous approach of the reviewers. It made them appear unsure of their supposed expertise. The tenuous review approach would not seem to me to have alleviated the leeriness of Nature must have had with Mc and Mc.
RE: Phil Hand
Posted Jan 7, 2010 at 10:40 AM
**Haven’t read all the comments, but it does seem that your post is completely wrong, and no-one’s pointed that out.**
You have not proven anything. Read it all first.
**2) Timing. Why would Langenberg make a review request in early March for a resubmission that hasn’t come in yet? **
Think Nature attitude – two favorable reviews.
**All three reviewers you link to say pretty much the same thing – highly technical issue of interest to specialists only.**
NO. The third reviewer was not in favor and had a problem understanding the statistics.
Kenneth, those are all good points.
And yes, quite aside from who did the reviews, let’s not lose sight of how flimsy the reviews are and how the reviewers complain that it’s impossible for them to decide who’s right and who isn’t. On many occasions, I’ve observed that journal peer review is a very limited form of due diligence – and nothing comparable to an audit or qualifying report where the reviewers would have to look at the data and decisions.
True, peer review can be a bit fluffy but eventually an accurate picture comes into focus. Remember when dinosurs lumbered around with their tails dragging behind? Now they run and have feathers, wasn’t due to bad science, just takes time to piece together from a scant record. The problem arises when the stakes are high and policy makers are told by overzealous advocates they must act on the information. Feel free to snip or delete
What I remember about dinosaurs is the time lag between a whack on the tail and recognition of that event by the brain.
Steve’s pointing to a whack on the tail of “Nature”. Its editorial brain seems slow to respond.
Kenneth Fritsch
“To get it wrong with a publication of an article like Mann’s that agrees with the consensus view is nowhere near the risk that getting it wrong on a controversial reply that could be construed as going against the consensus would be.”
I disagree. In regard to the established view of global temperatures over the last 2 millenia, MM2003 should really be seen as defending the consensus view and MBH as the radical new theory. The fact that MBH was pointed to as a “consensus” view has everything to do with politics and little to do with science. In normal fields, overturning such as well-established understanding takes many years and contributions from many independent groups.
Steve: We didn’t “defend” the consensus view. We said that Mann hadn’t proved what he claimed – different thing. But this is OT.
Steve – My point here is not that you and Ross set out to explicitly defend the consensus view, but that in terms of consonance with established understanding, and for the purpose of Nature doing due diligence, it was MBH that should have had a higher burden of proof.
Something that I like about the line Steve quoted is that it is ambiguous. Ironically and trivally, this can be viewed as adding to the credibility. If Jones had written more explicitly that he had an M&M manuscript to review, someone might suggest this was an email fabricated by the leaker. The overall authenticity of the emails is srengthened when plausible interpretations can be found even in obscure sentences.
Also from other emails, we can sense there were concerns that emails might someday be obtained via FOI. So there can be understated euphemisms employed, among the hockey team-mates. So when someone says he got a manuscript from Nature, maybe this is unimportant small talk. Or maybe he thinks it is too important to expressly elaborate upon.
Both Nature and Science have continuing problems with getting quality refereeing of interdisciplinary papers.
Here’s the latest bruhaha.
http://www.rsc.org/chemistryworld/News/2010/January/07011003.asp
Take this quote and substitute “statistics expert” for “synthetic chemist” and you have several climate papers all over again.
The funny thing is that both Science and Nature recognize they have a problem with peer review in other fields, just not in climate science for some reason.
Climategate issue is losing steam. I’m glad you are bringing it to attention and review the information for yourself.
Bernie -You are coming at him like you are the defense lawyer.
IMHO Nature is/was part of the team. As a part of the team, it was incumbent on a team member to find ways of refuting MM2003. Any way would do. So they sent the paper off to a team member and took his word for rejection. Does this hypothesis fit the facts?
To answer the four inquiries you’ve made:
1. I would say you’ve made a prima facie case that Jones is the author of the “Jones Review.” Still, while, on this evidence, that appears to be the most reasonably inference, there might be information about Nature’s administrative schedule that would undermine it. For example, perhaps there’s something about Nature’s proceedures that cause lots of people to get a lot of review-related business done on the same day of the month.
2. Even if the “Jones Review” isn’t Jones’ work, it pretty clearly is the work of a “team” member. It would be much harder, therefore, for Nature to show that they did not immediately solicit the opinion of a third reviewer with a known conflict of interest.
3. Whether doing so interfered with an impartial review is a bit of an ontological puzzle. By itself, soliciting a review from someone with a conflict of interest is not necessarily problematic. Your observation about the difference between the attitudes towards conflict of interest in academia and elsewhere is well taken, but, speaking as an attorney, even at law experts with conflicts of interest are permitted to give opinions, so long as the fact-finder is advised of them. In fact, it is my normal practice, when feasible, to go directly to an expert with an adverse interest, in order to get to the bottom of a technical dispute as efficiently as possible.
What is truly troubling, though, is Nature’s subsequent conduct. Apparently, they overturned the weight of opinion of two other competent experts based on the conflicted review, despite the fact that it offered essentially nothing in the way of valid technical criticism. Rather, it offered little more than advice regarding editorial focus–not something within the field of the reviewer’s expertise, and not something on which Nature ought to be expected to defer. It seems clear, therefore, that Nature was fishing for excuses to reject, and latched onto the best argument they could find.
I conclude, therefore, that there never was an impartial review process in the first place, with which the “Jones Review” might have interfered.
4. As to what this says about how journals ought to deal with such conflicts of interest, I think Nature’s procedures were sound, though, in this case, they blew the call. While I agree with the sentiment expressed elsewhere, that an important function of journals is to publicize legitimate threats to accepted theories, some effort to separate wheat from chaff is appropriate, and I think that would include soliciting comments from a champion of the orthodoxy. However, the purpose of doing so should be limited to giving the orthodoxy the best chance to expose technical defects in the work. In this case, the “Jones Review” failed to do so, and so Nature should have followed the advice of the other two reviewers.
Regarding Question #1:
If I were to apply the standards of criminal justice to your evidence (“beyond a reasonable doubt”), I would conclude that you have no case. If I were to apply the standards of civil justice (“preponderance of evidence”) to your evidence, I would again rule against you. If I were to apply the standards of scientific hypothesis evaluation, I would conclude that you have an interesting speculation that deserves further research, but does not at this time constitute a hypothesis that I would be willing to support.
You have certainly demonstrated that there is a probability that Jones was the reviewer. That probability, however, is not large enough to justify drawing so serious a conclusion.
Question #2 is predicated on the assumption that the answer to Question #1 is “yes”; since I have not so concluded, I must therefore conclude that the evidence does not support an affirmative answer to the Question #2. The same reasoning applies to Question #3.
Furthermore, I concur with an earlier commenter who pointed out your misconstrual of the meaning of “tricky” in this context.
One more thought on point (1): Phil says he’s not going to review until May (perhaps being petulant, for some reason) but he finally relents and knocks out the review. The timing still works for me, circumstantially. I didn’t re-read older comments.
One more thought on point (2)
This maybe a hair-brained (OT?) hypothesis but could Nature/Heike be acting funny (more stringent/more?) because of the recent rejection of a Santer paper?
“> >Dear Phil,
> >
> >I just don’t have much luck with the Heikes of this world. Heike L.
> >rejected our
> >Nature paper on the analysis of changes in tropopause height and
> >equivalent MSU
> >temperatures in ERA-40. She took six weeks to make this decision, and didn’t
> >even send the paper out for review! Very disappointing. I doubt whether
> >I’ll be
> >submitting any papers to Nature in the next few years.”
Maybe not but standard policies can help with ‘tough’ decisions. Sometimes people just get fed-up and rationalize themselves out of controversies.
“I am not defending nature…I was never here”
4 Trackbacks
[…] Climategatekeeping: the Nature Intervention Jones (of CRU fame) possibly involved as an additional 3rd reviewer for Nature on tree rings critical comment. […]
[…] More ClimateGatekeeping at Nature magazine […]
[…] Climategatekeeping: the Nature Intervention Today I’ll review one interesting sentence in Climategate Letter 1080257056 on March 22, 2004, in which Jones […] […]
[…] and insinuation, speculation and overreach. But that’s just my opinion. For example, the latest blog post focuses on one line of text taken from a Jones email sent around the time Nature rejected his […]