Comments on: Peer Reviewing Ammann and Wahl #1 – The Correspondence

By: BradH

BradH — Fri, 10 Mar 2006 13:51:17 +0000

One point of correction, in relation to my third sin – A&W did actually identify themselves as the authors of the mss.#3321 piece, so perhaps their sins are only 2.5…although what’s 0.5 between friends?

By: BradH

BradH — Fri, 10 Mar 2006 13:25:18 +0000

Re: 29

Given A&W are claiming that it was space pressure rather than content which saw their paper rejected, why haven’t they published on-line?

I don’t quite see that as being the main issue, John. It seems to me that they (and others) have used their unpublished work as proof that M&M should be discounted. If they had only just submitted the first paper, then were commenting on the review of the second short thereafter, you could understand them counting on the first [after all, no-one submits a paper in the expectation it will be rejected].

However, this entire saga has played out over an extended period of time. During that time, A&W have relied on it, Mann has relied on in, Houghton has relied on it, et al.

This isn’t even the most egregious problem. What A&W said on 10 June was that they had shown [read: proven] via their submitted, but unpublished paper, that M&M was “deeply flawed”.

However, at the time they wrote this statement, they knew that their submitted paper had been rejected [this occurred on 6 June].

Bear in mind that they did this in defense of a paper submitted to Climate Change. However, the rejected paper they were referring to as negating M&M was one submitted to GRL – two totally different papers.

So, let’s succintly summarise what’s happened:-

* A paper was submitted to GRL
* A later paper was submitted to CC
* A critical review was received in relation to the CC paper
* The GRL paper was rejected
* In response to the CC paper critical review, the authors cited the GRL paper as negating the criticism
* The authors’ assertion was not just that the GRL paper (which they already knew was rejected) supported their position, but that it effectively rebutted the criticisms of their new CC paper

Now, I would argue that this is a triple sin. The first sin, is referring to a rejected paper at all.

The second, is knowing it had been rejected, but still referring to it as supporting your point of view.

The third, is not specifically (other than by an obscure alpha-numeric reference) disclosing the fact to the journal and the reviewer that your “proof” of the criticism being unfounded was not only in a paper that had already been rejected, but in a paper which YOU had authored.

There is no possible excuse for this sequence of events. I hesitate to call things frauds and I suspect that A&W, having a deep emotional and professional commitment to their work, found it difficult to accept the criticism and rejection of it. Nonetheless, if Steve’s account is accurate, there is no other possible explanation, except that that they were disingenuous…and they lied.

It is also a serious and (I would suspect) a reviewable matter in relation to a paper staying on the published record, when an incorrect fact is used to support the case for its publication.

By: James Lane

James Lane — Fri, 10 Mar 2006 06:35:36 +0000

Having read Steve’s withering review of the original WA submission, I’m absolutely amazed that it has gone to press largely unchanged. It really makes you wonder about the quality of other peer-reviewd papers in the field (actually any field).

And I don’t agree with per – I think there is plenty to find fault with in the editor’s behaviour. It’s not about arcane statistical discussion. You only need to read MBH and MM05(a&b) to realise that WA completely misrepresent MM’s papers. I doubt that a bright 12 year old would have a problem seeing that.

Steve, I hope you have an opportunity to respond to this in Climatic Change.

By: Armand MacMurray

Armand MacMurray — Fri, 10 Mar 2006 03:46:06 +0000

Re #38
Leaving aside the whole issue of whether tree rings can really measure temperature reliably in a useful fashion, the whole high frequency vs. low frequency aspect of the reconstructions has only been brought up as a poor last-ditch attempt to justify ignoring r2 (not because of anything inherent in the tree rings). I don’t think tree ring proxies have any worse characteristics in the short term than in the long term.

By: Jason Lewis

Jason Lewis — Fri, 10 Mar 2006 02:54:22 +0000

The High-Frequency vs Low-Frequency discussion reminded me of two observations from other climate research. The top graph at the following link illustrates these points:
http://en.wikipedia.org/wiki/Image:Satellite_Temperatures.png (I couldn’t find a link to the original article)

1) The high frequency component the actual climate is real. 2) The high frequency component is of the same magnitude as the low frequency component

Even though the various hockey stick graphs span a much longer range (1000-2000 years), they are also reporting a temperature variation of almost the same range (0.6 – 1 C). The satellite vs. surface measurement debate also illustrates a parallel debate in the climate research community concerning the difficulties of quantifying low frequency drift. If the high frequency components of the satellite and surface measurements didn’t match, then there would be a completely different debate. At least one of the two would have been considered to be seriously wrong.

Am I correctly in that the tree ring proxies have trouble with short term correlations? If the short term bumps and dips in the graphs can be ignored, it’s more believable that spurious correlations might occur. Is there something wrong with my comparison?

By: Paul Penrose

Paul Penrose — Fri, 10 Mar 2006 02:41:49 +0000

Steve M,
I don’t seriously think that you were removed as a reviewer because you don’t have a PHD. More likely is that you were asking questions that they felt were too tough and/or didn’t want to answer. I just thought that Steve B’s idea had some merit and should not be dismissed out of hand.

By: jae

jae — Fri, 10 Mar 2006 00:23:05 +0000

Steve: You sure can’t rule out some type of conspiracy here. You are presenting some threat to an enormous bureaucracy and political force, here. There’s a tremendous amount of inertia to overcome.

By: Steve McIntyre

Steve McIntyre — Fri, 10 Mar 2006 00:14:53 +0000

#31, 34. This wasn’t in the version that we commented on. If you look at our actual review posted up today, I don’t think that we left much out. Here I merely posted up the introductory correspondence.

By: Armand MacMurray

Armand MacMurray — Fri, 10 Mar 2006 00:04:44 +0000

I also strongly agree with per’s comments in #31, especially “Even if this is palpable rubbish, you still have to explain to the editor why this is rubbish,…” Note the simplistic examples that AW give to show how r2 can go wrong.

By: Armand MacMurray

Armand MacMurray — Fri, 10 Mar 2006 00:00:09 +0000

Re:#19, #27.
I’ve got to agree with Pat Frank (#30), that “In the second place, common practice is that reponses to reviewers should be self-contained. ”
However, it seems that the problem began earlier, when AW did not supply a copy of all non-published references (i.e. their GRL submission) along with their manuscript for review. This is an absolute requirement when submitting a manuscript for peer review in all the journal processes I’ve seen. This makes it doubly “unusual” that they did not submit a copy along with their response-to-reviewer.
Their attempt to justify that RE was not at issue might be ascribable to their “lack of skill,” except that, besides misrepresenting the contents of their unpublished GRL paper, they didn’t even explicitly reference it in their June 10 response. I can’t imagine a serious scientist writing in a point-by-point detailed response that a point is justified by “…other material referenced in mss. #3321″ unless they are deliberately trying to hide the identity of that “other material.”