An Exchange at Scientific American

David Appell has posted up an exchange in Scientific American in response to his hagiography of Michael Mann. Here are some comments on Appell’s points.

The exchange as posted up by Appell here is as follows:

CHECKING MICHAEL MANN
In “Behind the Hockey Stick,” by David Appell [Insights], nary a mention was made of Michael Mann’s publication of a minor correction to his graph.
A detailed article in the February 14 Wall Street Journal pointed out how nonscientist Stephen McIntyre uncovered some flaw’s in Mann’s math. Although Mann is adamant that his theory is correct, McIntyre forces us to take a closer look. When Mann’s calculations are corrected, the global warming of the medieval times and the subsequent Little Ice Age return. Bye-bye, hockey stick.”
Clare Goldsberry
Phoenix, Ariz.

APPELL REPLIES: Michael Mann has not published any corrections to his graph. As stated in the story, his corrigendum, published in the July 1, 2004, Nature, relates only the existence and data on his Web site. He states that “none of these errors affect our previously published results.” Moreover, several grounds using different methods have replicated the results of Mann and his colleagues. This claim cannot be made for the work of Stephen McIntyre and Ross McKitrick, the two nonclimatologists cited in the Wall Street Journal article, which appeared after the March issue went to press.

As to Clare Goldsberry’s point about temperature reconstructions, as I’ve pointed out many times, we have not presented any temperature reconstruction as other than a reductio ad absurdum of MBH98. We do not endorse MBH98 methods or any reconstructions made with slight variations of MBH98 methods. However, reconstructions with slight variations show quite clearly the reliance of MBH98 on bristlecones and Gaspé cedars and the relationship of their erroneous PC method to over-emphasis of the most flawed proxies. It’s one thing for a civilian reader to misunderstand this point, but it’s a little tiresome seeing Mann and his associates either fail to understand this nuance or misrepresent it.

Secondly, there’s an interesting point about the Nature Corrigendum by Mann et al., which I’ve been meaning to write up for some time. The Nature Corrigendum was not independently peer-reviewed, but was cooked up between Mann and the Nature editors. I became very suspicious of this during the refereeing of our submission. In a reply, Mann went into a tirade about our emulation of MBH98 for supposedly not re-scaling at the reconstructed principal components level (in the emulation version used for those articles, we re-scaled at the NH temperature index stage. This was done merely as a guess and as an expedient as there was no description of a step like this in MBH98 itself.) One of the referees checked the original article to see the original description of the procedure; when he found that it hadn’t been described at all, he twitted Mann for the criticism and regretted that the issue had not been addressed in the recent Corrigendum. There is a downstream and rather ironic impact of this Corrigendum failure. As I noted before, von Storch et al [2004] attribute the attenuated variance of MBH98 to inverse regression and suggest that they should have re-scaled. In fact, MBH98 did carry out the re-scaling proposed by von Storch; they just never reported it. It should have been dealt with in the Corrigendum and the failure to do so has resulted in von Storch et al. boxing with shadows, although no one seems to realize this.

Back to the refereeing of the Corrigendum: it seemed highly doubtful to me that a different set of referees would have been involved in the Corrigendum than our submission. So if our referees were not involved, was it possible that the Corrigendum had not been peer reviewed at all? I wrote to Nature a couple of times asking this question, but they wouldn’t answer.

When Marcel Crok wrote his article for Natuurwetenschap & Techniek, he pursued the matter with Nature and they admitted that they had not had the Corrigendum independently peer reviewed. They argued that their policies did not require them to do have a Corrigendum independently peer reviewed. Such policies sure seem strange to me – I would have thought that independent peer review of a Corrigendum would be particularly important. Aside from their policies, they had promised us that they would do so at the time of our original complaint, but they failed to live up to this promise.

The Corrigendum acknowledged that datasets said to have been used in MBH98 were not actually used. The explanation provided in the Corrigendum is false. I had notified Nature of this, but they ignored it.

The Corrigendum assertion that the “errors did not matter”? was not in the preprint version of the Corrigendum which I saw and was inserted at a very late stage. It’s not clear what review, if any, this claim had even from Nature editors. Yet this claim, which amounts to little more than the self-interested puff of Mann et al., has been widely disseminated.

It’s pretty obvious that the erroneous description of principal components methodology did matter. At a minimum, none of the important North American PC series can be obtained using the reported PC methodology. The Corrigendum was an opportunity to correct the record on the PC methodology. But Mann et al. failed to do so.

Appell argues that “several grounds using different methods have replicated the results of Mann and his colleagues.”? I presume that he is referring to Rutherford et al [2005] and Wahl and Ammann [rejected by GRL; and in submission to Climatic Change]. It’s mystifying to see these guys continue to ignore the GRL rejection of Wahl and Ammann and the continued UCAR silence.

Rutherford et al. [2005] is co-authored by Mann, Bradley and Hughes and is hardly an independent study. This study actually uses the erroneous MBH98 tree ring PC series without any explanation or apology. So any support from Rutherford et al. [2005] is pretty meaningless.

Wahl and Ammann have already been rejected by one journal. I’ve posted up comments on Wahl and Ammann, noting that their results fail verification tests (R2 ~0.0) – perhaps this is what Appell means by replicating Mann’s results. Wahl and Ammann’s results up to the reconstructed PCs yielded identical results to mine, which they did not acknowledge. There was a difference in scaling, where Wahl and Ammann were fiddling with their code up to April 2005. Wahl and Ammann are hardly independent of Mann. Wahl and Ammann do not replicate Mann to audit exactness; it’s merely an approximation. Their emulation is virtually identical to our prior emulation, even as to the language used, although they don’t seem anxious to admit it.

Cubasch, who is not a close associate of Mann’s, reported that they could not replicate his results – see a prior post on this.

On the other hand, a number of independent people have been reported as replicating our finding of the horrendous bias of the MBH98 PC method (Zorita, Zweiers, Hubert). Our code has been closely examined by people not mentioned in the press and has so far stood up.


11 Comments

  1. John A
    Posted Jun 22, 2005 at 9:58 PM | Permalink

    Of course if the “errors did not matter” then Mann should have been able to demonstrate that statement in a proper impact assessment, which is the normal procedure for a Corrigendum. That Mann did not do this and that Nature did not insist this be done is one of those “Great Mysteries of Climate Science”.

    Of my response to David Appell’s profile, I can only say that I’m proud to have brought to a lot of people’s attention one of the most preposterous, vacuous and one-sided articles ever published as serious scientific journalism.

  2. Richard Lewis
    Posted Jun 23, 2005 at 4:30 PM | Permalink

    The hockey stick not merely broken, but shredded? An interesting hypothesis is offered here:

    http://service.spiegel.de/cache/international/spiegel/0,1518,357366,00.html

  3. Michael Jankowski
    Posted Jun 23, 2005 at 9:03 PM | Permalink

    The “it does not matter” cry has spread beyond the potential errors of MBH98 and/or the proxies and reached the hockey-stick itself. Now, “it does not matter” to many people if the MWP was real and warmer than today…you just have to believe that GHG emissions warmed the planet in the 20th century towards devastation and that comparing today to the MWP is apples vs oranges. It certainly makes for a convenient out should the hockey stick meet a fateful end.

  4. Peter Bickle
    Posted Jun 24, 2005 at 5:35 AM | Permalink

    David Appell is [such a strong Mann supporter - SM: I've toned this language down from post; I don't want to parse posts] it does not matter how much is said, he will alwats say the HS is gospel. Sure the earth is warming, but it has been a lot hotter over the eons.

  5. Michael Neibel
    Posted Jun 24, 2005 at 1:26 PM | Permalink

    I feel like a fish out of water becaue I am not a scientist of any kind. But I’ve been studying philosophy for quite awhile and have been getting better at spotting an invalid concept when I see one. When I saw Mann’s HS, in my mind, I knew I was looking at flawed science on the premise that there can’t be a linear explanation for a non-linear system and that is how the HS appeared to me. I had no clue why the HS was wrong. I only knew that it could not be right and would advise others (if they asked) to disregard it as folly.

    Now I know that non-linear data points can be plotted on a chart to reveal a linear line if the number of such points is sufficiently small, and that adding more data points makes the line disappear revealing the true non-linear nature of the data. My question is: is there anything in the nature of statistics whereby a nonlinear system has a linear explanation and still be valid/

  6. John Simon
    Posted Jun 24, 2005 at 4:46 PM | Permalink

    Making linear assumptions is fairly common in statistics. The trick is making sure that the assumption is not ‘too bad’.

    Allow me to explain somewhat. Most of statistics does linear regression (the workhorse is OLS – Ordinary Linear Regression). Some tricks can be done to ensure that this regression is correct even for non-linear functional forms. For example, taking logs of the data is one such trick Y=XY can be transformed into the linear regression log(Y)=log(X)+log(Y) (you can have exponents on the X and Y but I didn’t want to unnecessarily complicate this example).

    Another trick/assumption is that you can linearise any non-linear function by taking a first order Taylor expansion. The point with this is that the approximation is ‘locally’ correct. However, depending on how important the other terms in the Taylor expansion are ‘local’ may be a very small area. Thus, in the present context, the question is whether the true temperature/proxy function is near enough to linear over the relevant range. Relevant is a rather vague term but at one level if you are talking about observations within the bounds of the data set then you are on safer ground than if you are extrapolating to values outside the bounds of your data set.

    Hope this helps in some small way…

  7. John Simon
    Posted Jun 24, 2005 at 4:49 PM | Permalink

    Addendum – I had a brain fade in the previous post. OLS is Ordinary Least Squares but it is also a linear estimator, sorry. (It’s BLUE (Best Linear Unbiased Estimator) if you really want to know)

  8. Steve McIntyre
    Posted Jun 24, 2005 at 5:39 PM | Permalink

    One of my main points is to provide a very lowbrow explanation of the hockey stick. I don’t think that linearity/nonlinearity are the issues: the issue is the bristlecones. These have a hockeystick pattern, which is imparted to the temperature reconstruction because of non-robust statistical procedures. So the question is then – what causes the hockey stick in bristlecone pine growth? The answer is almost certainly 20th century fertilization by CO2 primarily, perhaps by nitrates as well. The issue IMHO is not nonlinearity, but simply leaving out a factor. Steve.

  9. Michael Neibel
    Posted Jun 24, 2005 at 7:28 PM | Permalink

    Thanks John. It did help. I’m going to look up the concepts you mentioned and study them.

  10. Posted Jul 31, 2005 at 10:51 AM | Permalink

    Thanks for mentally connecting CO2 fertilized growth rates as a significant factor in the fabrication of the hockey stick with the bristle cone data. I had just considered increased growth rate data as a factor to eventually balance CO2 emissions and offset the previous cries of gloom on plant growth with increasing CO2 levels. May I suggest that you “remind” the audience periodically, with 1-2 choice links that show the dramatic effect on plant growth and if possible, bristle cones. It might help newcomers and surfers passing by, perhaps a few ardent GWers. I found this link useful, but think a more “GW neutral” reference would be nice.

  11. TCO
    Posted Sep 19, 2005 at 11:00 PM | Permalink

    Do the bristlecones still produce a hockeystick curve with a “simple average” or a normal centered reconstruction vice the PCA cherrypicking method?

    I agree that Nature should have peer-reviewed this Corrigendum given all the controversy and Mann’s defensiveness. And they should have published MM’s comment in some form. But MM still had a big effect and should not get discouraged. And they did the right thing by going to another peer-reviewed journal (GRL).

Follow

Get every new post delivered to your Inbox.

Join 3,251 other followers

%d bloggers like this: