Those who forget the past are condemned to repeat it. The SI for MBH (and Mann et al 2007) had incorrect geographic locations for numerous proxies.
The same error is repeated in Mann et al 2008, a defect encountered by Jeff Id and myself in trying to replicate reported correlations to gridcell temperatures. Nearly 100 “Schweingruber” MXD series have incorrect geographic locations in the SI; strangely occasional “schweingruber” MXD have correct geographic locations. It’s the usual dog’s breakfast.
Mann et al 2008
In the graphic comparing calculated correlations to reported correlations, there was a very high degree of inconsistency between my calculations and reported correlations for the “Schweingruber” gridded MXD series (relative to other proxy classes). To try to understand this, I attempted to examine this data set in more detail.
The Mann et al 2008 SI (SD1.xls) lists 105 Osborn MXD series (rows 910-1014 on the spreadsheet) together with latitudes and longitudes. The SI contains ID references such as “schweingruber_mxdabd_grid91”; unfortunately, although I’m as familiar as anyone with proxy IDs, these IDs do not correspond to any data version that I’ve examined. The SI contains latitudes and longitudes and I used these for calculating grid cell correlations. Because of the inconsistency, I went back and plotted these gridcell locations (as shown below).
Figure 1. Location of Gridded MXD series (Mann et al 2008 SI information.)
These locations do not match any conceivable locations from the Briffa MXD network, or, for that matter, the location map in Mann et al 2008 Figure1 (shown below), where Briffa MXD locations are shown as diamonds. I guess the listing of site latitudes and longitudes in the PNAS Supplementary Information is the “wrong” data set. It would be nice if they archived the “right” SI.
Figure 2. Mann et al 2008 Figure 1 Location map.
Update: As suggested by Jean S below, I examined the lat-longs in rtable1209 and it turned out that these lat-longs, rather than those in the PNAS SI, are the “right” coordinates.
Briffa MXD Locations per rtable1209.
Provenance of the 105 ‘Schweingruber’ Series
The Mann et al 2008 SI contains no information on the provenance of these 105 series (provenance is sometimes given for other series.) The main article says of this data:
The gridded maximum density dataset was developed differently by Osborn et al. (http://www.cru.uea.ac.uk/~timo/datapages/mxdtrw.htm) [as used by Rutherford et al. (2)] with the specific goal of capturing century to multicentury variability. [2. Rutherford S, et al. (2005) Proxy-based Northern Hemisphere surface temperature reconstructions: Sensitivity to methodology, predictor network, target season and target domain. J Climate 18:2308–2329.]
The 105 gridded series are not actually referred to in the above webpage. The above webpage lists 300+ MXD series constructed by Schweingruber, but does not contain any reference or identification to the gridded data referred to here. (In passing, I had some role in the creation of this webpage, which is quite recent. Briffa did not list the sites or provide an SI listing the sites in any of his many papers on this network. I asked for this information, which they refused to provide. Eventually, I resorted to the FOI process, finally resulting in the identification of the sites. Most – but not all – of the sites have been archived by Schweingruber at ITRDB, with the incompleteness hampering replication efforts.
To my knowledge, the first mention of gridded MXD series occurs in Rutherford (Mann) et al 2005, which describes the construction of 115 (not 105) gridded MXD series as follows:
OSB therefore worked first with the traditionally standardized data at the individual chronology scale and gridded them to provide values in 115 5° by 5° grid boxes (26 available back to A.D. 1400) in the extratropical NH (Fig. 1b). They then developed temperature reconstructions by the local calibration of the MXD grid-box data against the corresponding instrumental grid-box temperatures.
Their Figure 1b, referred to above, is shown below and compares to the sites in Mann et al 2008 (though the match is not exact as 10 sites seem to have disappeared in the translation.)
Figure 3. Location map from Rutherford, Mann et al 2005. Original caption: FIG. 1. Distribution of proxies for the two networks used in this study. …(b) The age-banded MXD network of Briffa et al. (2001), where each dot corresponds to the center of one 5° x 5° grid box.
Although this caption certainly implies that the 115 grid boxes derive from Briffa et al 2001, that publication contains no mention whatever of 115 gridded series. Instead they report on 9 regional composites. As far as I can tell, the 115 gridded series are calculated in Rutherford et al 2005 and not by Briffa. Looking at the definition of “OSB” in Rutherford et al 2005, it is defined as follows:
The version of the MXD dataset used here was compiled using a combination of grid-box estimates based on traditionally standardized MXD records (with limited low-frequency information) and regional estimates developed to retain low-frequency information (OSB; the data in the MXD network are available online at http://fox.rwu.edu/~rutherfo/supplements/jclim2003a).
Unfortunately, the sentence on availability of data is untrue. I was blocked from this website for a long time, but the block seems to have been lifted and I re-visted this website today, which says today (as it has since it was first put up):
MXD Network Contact Tim Osborn
So the representation in the Journal of Climate was totally bogus. The data in the MXD network was never online at the referenced website. I brought this up with Journal of Climate, but, like subprime accounting offices, they simply blew me off. At this time, the 115 (or 105 or whatever) Rutherford, Mann et al gridded MXD versions simply do not exist anywhere online, other than in the versions at Mann et al 2008.
Here we see another inconsistency between the representation in the SI and the “original” data actually provided. For example, the SI states that the end data of schweingruber_mxdabd_grid10 series is 1980. However the “original” data provided at WDCP has truncated values after 1960. This is not mentioned in the main text, but is sort of mentioned in the SI. (In my opinion, this mention hardly justifies the truncation, let alone the removal of the data from the “original” data, particularly when there is no external provenance for the “original” data.
Because of the evidence for loss of temperature sensitivity after ~1960 (1), MXD data were eliminated for the post-1960 interval. The RegEM algorithm of Schneider (9) was used to estimate missing values for proxy series terminating before the 1995 calibration interval endpoint, based on their mutual covariance with the other available proxy data over the full 1850–1995 calibration interval.
The 1960 truncation of the Briffa data has been a sore point here at Climate Audit, being discussed in many posts, going back to the truncation of the inconvenient bits in IPCC 2001. It’s disappointing that it’s still being done. What makes this incident particularly bizarre is that, after Mann deletes the post-1960 values of the Briffa data, he then substitutes “infilled” values and then calculates correlations using the imaginary data. Out of the 484 series that “pass” Mann’s correlation test, no fewer than 95 are these spliced Briffa MXD series which exemplify the divergence problem without splicing; another 71 are the Luterbacher instrumental series.
I’ve just located an old post in which I calculated the average of all Schweingruber MXD series. I’ll do a comparison of that the Mannian RegEM versions in my next post.
Climate Audit 
32 Comments
You know, of course, that these guys read your site. Probably at home, at night, in the dark, in the basement, under a dim bulb, after their families have gone to bed. It’s probably not easy going to sleep afterwards.
I gather MXD stands for Maximum Density, so that these Briffa series are different from the usual treering width series. What is the rationale for expecting a correlation between this and temperature? Should the correlation be positive or negative, in the view of Mann et al? I see from the yellow Briffa symbols in Fig. 1 of your 9/20 post that there is a preponderance of positive correlations, particularly in the “best out of two” version on the vertical axis. Are these all MXD, or did Briffa give ring widths as well?
Should one infer from your 9/20 graph that Mann et al 2008 primarily used the erroneous SI coordinates? There does appear to be at least one yellow symbol that lies to the right of the X, but almost all are on, above, or below it as expected.
Steve: There have been articles connecting MXD and temperature, but Briffa’s MXD recon is also the poster child for “divergence”. Anyhow a positive correlation was expected. All of the Briffa network are from Rutherford Mann applying RegEM to the Briffa MXD network. So there are layers upon layers of Mannomatic RegEM here and finding firm ground will take some time.
Are the locations the same in ‘rtable1209’ as in ‘SD1.xls’?
You’d think that after the pasting Mann got for incorrect locations of proxies causing them to be calibrated against the wrong gridcells, Mann might have been a little more careful this time, but noooooo such luck.
Hmmm. the locations in rtable1209 look like they might be reallocations.
Yes, these look like the correct lat-longs for the Briffa series. They’ve screwed up the SI somehow. You’d think that they would have duplicate versions running around.
I’ve looked at other differences between locations in the SI and rtable. The M08 figure seems to be drawn from the rtable version, which contains an amusing and characteristic Mannian error, almost identical to one of the most amusing MBH errors.
Maybe it’s risky to speculate on psychology, but surely Mann is too smart not to be aware of the problems by now. It must come down to the fact that it was McIntyre who pointed the problems out, and to correct this would be to publicly concede the point.
It seems that the new article simply repeats most of the mistakes of the old. My take on this is that Mann is just thumbing his nose at the critics, while reassuring his disciples. The message of this publication is, “I can get anything I like past peer review and into major journals, while the critics can’t publish anything. Ignore the critics, the Hockey Stick is still mainstream.”
#6: I get the sense that Mann was trying to surprise everyone with a study where he archived the data (albeit incorrectly), provided a SI that was more descriptive than previous studies, and where the notorious bristlecones could be removed with the results staying intact. In other words, I think he might have been trying to legitimately address criticisms against him and produce a good study. He obviously failed, but at least some things improved from the last iteration. If Steve or Mann himself can publish a correction/retraction of this study more quickly than 4-5 years after the fact, there will have been some progress.
It is ironic that giving Mann the benefit of the doubt (i.e. that this was a legitimate attempt to produce a good study) actually makes the errors more hilarious.
Hi Steve —
> I was blocked from this website for a long time, but the block seems to have been lifted
I think it would be best if you did not refer to being blocked from websites. If it is true, fume about it silently and use an ‘anonymous proxy’ to casually sidestep any such blocking. If it’s not true, if for example it is due to unrelated network issues, it’s probably not worth other’s attention. But whether true or false, to those eager to find reasons to dismiss you, it make you look paranoid and thus makes your scientific criticisms more easily ignored.
And I’m sorry I haven’t followed up on your response in the thread regarding Gavin Schmidt — I still haven’t figured out a constructive way to phrase things.
The short answer, though, I trust your honesty and admire your scientific impulses, but worry that some of your rhetoric runs counter to your purposes, which I presume is to get to the bottom of the science. I strongly disagree with Gavin’s assertion that it is your ‘first instinct to impugn [his] motives and honesty’, but I also don’t think you are blameless. While I understand your frustration, and probably couldn’t deal with it nearly as well as you have, there are times when you have treated him with less than full respect.
For example, despite your caveats, I think your implication that Gavin might have might have lied about silently adding a link to a previous post is an example of you unnecessarily impugning his motives in a case where there are alternative explanations. And thread titles from several years ago like Is Gavin Schmidt Honest? would lend credence to this being a long standing trend. While your complaints may be valid, titling a thread in a manner that directly insults Gavin seems non-productive.
Anyway, here’s hoping that the any flaws in MBH08 can be corrected in something faster than the 10 years it took the first time around!
Steve: Nathan, the blocking at various websites is a matter of fact. In cases where I’ve been blocked, I can get the information by going to the university library so it’s pointless.
In the Gavin Schmidt thread, the incident that provoked it was Realclimate censorship, where they refused to post my comments (as well as other readers.) We mentioned it in threads at the time, but I didn’t make an issue of it. When they ran a thread in 2005 criticizing our papers (at the time of the Von Storch comment and our Reply), I tried to post a response and again they censored. If you’re going to criticize someone on your blog, you have to let them reply. They said that their policies were to permit scientific comments. My comments were as scientific as you could ask for, but Gavin censored them. So the question was a fair one. The thread in question did not “impugn” his motives in a case where I couldn’t “work” something out. Whatever my defects, being impatient in working things out is not one of them. Nor do I automatically assume the worst of people.
Re: Nathan Kurz (#8),
The Team chose to make an enemy of a guy who had a solid point and has been proven right many, many times over. If you are so hungry for peace, then you should convince the Team to start peacemaking. They are the ones who chose to make an enemy of Steve. Don’t forget that. They are the ones who started the mudslinging using a progaganda device to mass target their message. Don’t forget that. It’s almost like they were thinking they could bankrupt him, bringing NASA resources to bear in a propaganda war of attrition.
Re: team bender (#9),
Dear Bender,
I always enjoy your comments. But when you state that: “The Team chose to make an enemy of [Steve McIntyre]”, I believe you inadvertently misstate the circumstances. The Team may have designated McIntyre an enemy and consider him as such, but I do not think that McIntyre is an enemy of the Team. If the Team chose to consult and collaborate with McIntyre, I do not believe he would refuse.
Regards,
WJR
Re: Nathan Kurz (#8),
Re: Nathan Kurz (#13),
Oh, the irony.
Dog’s Breakfast indeed.
They will eat anything.
I can’t get through a paragraph of this without wanting to rant. I give up.
Bender #9 (presuming ‘Team’ was not intended there)
> If you are so hungry for peace, then you should convince the Team to start
> peacemaking.
I tried but had limited success. My final response at RealClimate did not make it through moderation, but is posted at ClimateAudit here. I’ve been reading this site since Steve’s initial postings on MBH98, and am aware of the reception that Steve has been given. I encourage him to try to make amends not because I disagree with his message, but because I think that doing so is the best way forward from the current situation.
Re: Nathan Kurz (#13),
Kurz, that’s all nicely said, but for the Team, the world isn’t like Lennon’s “Imagine” song. As much as Steve McIntyre would want to be in peace, it’s not as if they wouldn’t simply shut him off, as they have always.
This is a grave red herring. This attitude is what drove me past the GW alarmist preachings to become more of a “skeptic”, though not a “denier”, per se. Only more of an interested reader of scientific debates and struggles. Unfortunately, the sheer lack of rigorous attitude and methodology, quality, etc., makes me very concerned at the legitimacy of the alarmists claims.
It could end up being a massive-sized pseudo-science. But I don’t jump into conclusions. It may end up being a good estimate with lousy methods and data collections. A lucky counter-example of GIGO?
Like the brits say, “wait and see…”
quote Maybe it’s risky to speculate on psychology, but surely Mann is too smart not to be aware of the problems by now. unquote
You know that magician’s technique of making large and obvious movements with the right hand while the left hand does the switch? Is there anything else going on at the moment, something not connected with the proxy stick?
It’s what I’d do if persecuted by a workaholic gadfly.
JF
In my skepticism I even include Gavin’s remark:
It’s probably the paranoid in me, but it almost reads as in “resistance is futile, conform and we’ll all be happy”.
If Gavin simply censors Steve’s scientific comments, then he’s being a[auto-snipped].
It is interesting to note that Gavin did a detailed attack on the proxies I used in my reconstruction in 2007, picking on small details, but at RC has given a pass to Mann on all 1000+ proxies, with no critique at all. It is also interesting that Gavin only counts 19 proxies that go all the way back to 2000 yrs ago, where I had 18 (different ones in some cases) which was of so inadequate at the time…but I am sure I’m just being too sensitive.
#17. Luis, you quote Gavin as saying:
It’s an odd accusation. I’m not sure that I’ve ever “claimed” to be a “gentleman”. This sort of petty class distinction is something that Canadians typically don’t set much store by – with Conrad Black being a notable exception. I did a search on the word “gentleman” in my posts and did not find any uses by me of the word “gentleman” in my head posts. It’s not the sort of thing that I worry about or the sort of “claim” that I tend to make on my behalf.
In a comment about my visit to Georgia Tech, where I was criticized for snark spilling over from the Team to the broader climate science community, I adopted Oscar Wilde’s famous definition (and surely Gavin would accept Oscar Wilde as authority on such matters):
I think that I’ve stuck to this resolution pretty well. I’ve tried to eliminate the “hey, it’s climate science..” type of comment and, if I haven’t adhered to this resolution totally, I’ve done a pretty good job of it. But my goodness, I don’t think that I can give up irony altogether. I write the blog for fun.
Anyway, while I don’t believe that I’ve ever made “claims” of the type asserted by Gavin, nonetheless, if Gavin is aware of any situations where I’ve insulted anyone unintentionally, I will concede that this was ungentlemanly and will willingly add appropriate commentary withdrawing any unintended slight.
I thought the definition of a gentleman was someone who used elegant and clever insults. Eh? It would be the saint who never uses insults.
Then there is “a scholar and a gentlman”. Keep up the good work and the quotes. I only had Dilbert before this.
#20. You say:
Gavin says:
Using your definition, maybe Gavin’s comment could be interpreted as follows: he is saying that the Team would be twice as happy if I had only 50% of whatever cleverness in repartee that I presently possess.
Note that it’s an interesting example of a non-linear response. I might add that this is also a non-symmetric situation. If their cleverness in repartee was reduced to 50% of its present level, it would be de minimis change as far as I’m concerned.
OK back to work. Here’s a plot comparing Mann’s rtable correlations and my calculations for the Briffa MXD network after inserting rtable lat-longs for the incorrect SI lat-longs.
These calculations use both gridcell and nearest neighbor. Some match exactly, but others don’t. It’s a mystery. There are 49 more series with correlations gt 0.14 in the SI than in my emulation, so the effect is not negligible.
There are a few stations where the next nearest cell has no value and my algorithm doesn’t search, but this isn’t what’s causing the differences.
UPDATE Sep 24: The above graphic didn’t completely fix Mann’s incorrect SI lat-longs. I re-examined this and spotted a flaw in my attempt to patch Mann’s SI error and this explains much of the discrepancy. The main issue with these series will be the 1960 truncation.
Re: Steve McIntyre (#23),
I think (a part of) the difference might be coming from the fact that not all instrumental cells which have values are used (those with less than 10% data are disregarded). See here.
I’ve uploaded a script from which the above graphic was calculated.
http://data.climateaudit.org/scripts/mann.2008/temperature.correlation.txt
It might still contain some local references – I have tested it fully turnkey. However, it will deliver you a collated version of Mann’s temperature matrix (inst) and an R list statlist with 6 items each being a 1209×3 table of correlations as described in the script.
A few plots are shown but I’ve done others.
I tried to post this earlier but something went wrong.
How hard is it for your software to sort through all gridcells to find an r match to the reported value? It might clear up some of the lat long issues also.
#25. I’ve already controlled for that. In the Briffa MXD area, the issue doesn’t really arise.
The following graphic replaces the one posted in #23. I had incompletely patched Mann’s erroneous SI lat-longs in the graphic in #23. The big issue with this data set is the 1960 truncation. The proportion of high correlations in this dataset appears to be higher than for tree ring data as a whole, which also needs to be examined.
Re: Steve McIntyre (#28),
could you post the id’s along withs locations of (let’s say) 10 series with greatest (remaining) difference? It would help to figure out from where the remaining difference is coming.
discussion moved to Replication thread
I see. Another data pile. How can it be so mixed up?
I be those guys stop by just to watch the confusion.
quote a “gentleman” unquote.
Tut tut. With a Scottish highland name and you’ve forgotten the bard.
The rank is but the guinea’s stamp.
JF
One Trackback
[…] “data quality control” when Mann himself has issues with his own papers such as incorrect lat/lon values of proxy samples, upside down Tiljander sediment proxies, and truncated/switched data, is mind boggling. It’s […]