In a draft Corrigendum dated Oct 10, 2009 (most recently modified Oct 21, 2009), Kaufman gives what is obviously a warm and heartfelt shout-out in which they:
thank those who have pointed out errors and have offered suggestions.
More later today. [Note: see Sep 15 post here here for a preliminary assessment of Kaufman using upside-up Tiljander and using non-Briffa tree rings: Grudd’s Tornetrask, Esper’s Polar Urals and Moberg’s Indirka River. This needs to be updated since only 2 of 3 Finnish series need to be inverted.]
10.30 am Eastern: Kaufman’s Draft Corrigendum reported that (so far) “Four of the 23 proxy temperature records included in the synthesis contained errors.” A little under 20%. Not too bad for the Team. The Corrigendum itself doesn’t actually say which series contained errors, referring interested readers to the Draft Revised Supplementary Information (dated Oct 7, 2009). This states:
Record 20 was corrected to reflect the original interpretation of Tijander et al. (S32) that X-ray density is related inversely to temperature.
Record 21 was corrected to reflect the interpretation of Haltia-Hovi et al. (S33) that varve
thickness is related inversely to temperature.
CA readers may recall that the issue of upside-down use of the Tiljander series was originally raised at CA in the wake of Mann et al 2008 in Sept 2008 here, that it was further pointed out in a published comment on Mann et al 2008 (McIntyre and McKitrick PNAS, 2009), where the upside down use was denied by Mann et al (PNAS 2009). On the day that Kaufman 2009 was released, its upside down use was again noted here and a note on the matter sent to Kaufman by email. I invited Kaufman to post a thread at CA and requested source data not publicly available. Kaufman told me not to write to him again.
In my first post, I observed that, because Kaufman truncated this series in 1800 thereby not including the huge HS portion, the upside-down use of this series wouldn’t “matter” as follows:
I’m sure we’ll soon hear that this error doesn’t “matter”. Team errors never seem to. And y’know, it’s probably correct that it doesn’t “matter” whether the truncated Tiljander (and probably a number of other series) are used upside-down or not. The fact that such errors don’t “matter” surely says something not only about the quality of workmanship but of the methodology itself… What does “matter” in these sorts of studies are a few HS-shaped series.
On other occasions, we’ve discussed why it doesn’t seem to “matter” whether some (usually the majority) of the series are used upside down or not. Grace Slick explained the situation as well as anybody: “one pill makes you larger… and the pills that mother gave you don’t do anything at all.” Yamal makes you larger; truncated Korttajarvi doesn’t do anything at all.
Atte Korhola, a Finnish paleo, learning of the problem with the Finnish series from Climate Audit, took a dim view of the upside-down use of the proxy and the continued belligerence of realclimate, Kaufman and associates in a Finnish language blog post covered at CA on Oct 2, 2009 by Jean S, who translated his Finnish language comments as follows:
Another example is a study recently published in the prestigious journal Science. It is concluded in the article that the average temperatures in the Arctic region are much higher now than at any time in the past two thousand years. The result may well be true, but the way the researchers ended up with this conclusion raises questions. Proxies have been included selectively, they have been digested, manipulated, filtered, and combined, for example, data collected from Finland in the past by my own colleagues has even been turned upside down such that the warm periods become cold and vice versa. Normally, this would be considered as a scientific forgery, which has serious consequences.
A few days later (Oct 7), Kaufman grudgingly drafted the Corrigendum presently at their website. I might add that Korhola made some favorable comments about CA in the Finnish language comments to his blog entry. One of his readers (Jokimäki) asked about giving credence to claims made on blogs:
If we are talking about scientific community, then why should any criticism there be taken seriously, if the criticism is not given through scientific channels? Otherwise we end up to a situation such that the scientific community needs to respond to every single humbug claim (“criticism”) that someone puts to Internet. Or how would you plan to differenciate between whose claims are to be taken seriously and whose not?”
Korhola responded:
“Jokimäki is absolute right: it is not worth reacting to every criticism on Internet. Researchers could be doing nothing else, if we started to do that. The criticism by McIntyre and CA is an exception maybe in the sense that it relates strongly to the previous discussion, and the criticism in CA previously directed to the same issue (statistical analysis of proxy material) has been shown to be scientifically valid (Wegman committee). McIntyre & co also try to publish their results and criticism in scientific forums.
and later:
The criticism by McIntyre and Climate Audit has to be taken seriously. RealClimate by Mann & co is mainly ridiculing [Climate Audit] in the latest blog post. In the long run, they may well turn out to be shooting themselves in the foot.
In this particular case, it’s interesting to contrast the handling of using the Tiljander data upside down in the “Peer Reviewed Literature” and in the blogs. We pointed Mann’s upside down use of the data (with a worse impact than on Kaufman) in the correct channels. Mann denied it. Once the matter is pointed out, it’s not rocket science to determine who was right, but PNAS took no steps to resolve the contradiction. realclimate readers took Mann’s denial as being proof that he didn’t use it upside down e.g. on the recent Yamal thread (#651):
651. Over at Dot Earth, McIntyre is taking another shot at Mann et al. 2008. link.
He seems to still be worried about inverted data despite Mann et al. publishing a formal reply to this. At this point bizarre is not the word any more.
and later on Oct 13:
673. Could someone point me to where this “inverted data” issue is addressed by Mann or someone else who knows? I’ve so far been unable to debunk McIntyre’s claims that there was an error there.
Thanks!
[Response: The original commenter appears to be referring to: Mann, M.E., Bradley, R.S., Hughes, M.K., Reply to McIntyre and McKitrick: Proxy-based temperature reconstructions are robust, Proc. Natl. Acad. Sci., 106, E11, 2009. – mike]
While Kaufman has admitted using the data upside down, Mann hasn’t. Here is a plot of the two Kaufman versions – decadally averaged XRay density (truncated in 1800):
Figure 1. Lake Korttajarvi Versions – Old and New.
Mann used not one by four Korttajarvi series upside down. Here is a plot of the varve thickness series (which gets to over 9 sigma in recent years!) A Finnish varve thickness series was used in Tingley and Huybers 2010? and one wonders whether it might be this one.
Figure 2. Korttajarvi Thickness Versions.
4.30 pm Oct 26: Dye 3
Another change in the Corrigendum is to Record 12 (Dye 3) as follows.
Record 12 was revised to omit the high-pass filter used by Andersen et al. (S25)
Dye 3 was one of the series for which I requested annual data (refused by Kaufman). I noticed today that annual data has now been placed online – in a pdf format rather than a digital format, a stupid paleclimate pet trick that prevents the use of scripts directly linking to their site. (Yeah, year, it can be turned into an ASCII file in 10 minutes or so, but it’s a waste of time).
No reason is provided in the Corrigendum as to why it is now believed to be appropriate to “omit the high-pass filter”, as compared to the procedures of the original article (see below figure). In this case, the correction “helped” Kaufman.
Figure x. Dye 3 (SD Units) Before and After Correction
Andersen et al (JGR 2006) online here reports the following in connection with filters at Dye-3:
In order to derive annual accumulation rates from the observed annual layer thicknesses, the data had to be corrected for densification and thinning of the ice layers due to ice flow. This was done by using a flow model [Johnsen and Dansgaard, 1992; Johnsen et al., 1999] also accounting for firnification at the top of the ice. In this way we obtained cross-dated chronological time series of annual accumulation rates over the latest two millennia, with relative dating errors being at most a few years. The ice flow in the DYE-3 region is complicated by upstream surface undulations, and the obtained accumulation rate profile thus contains longer-term variations of nonclimatic origin [Reeh, 1989]. In order to remove these variations we have filtered the DYE-3 accumulation record with a Butterworth filter of order 3 with a cutoff frequency of 0.001 year^{-1}, eliminating the lowest-frequency variations.
As I read this paragraph, the purpose of the high-pass filter in Andersen et al 2006 (the Butterworth filter of order 3 with a cutoff frequency of 0.001 year^{-1}) was to remove a “longer-term variation of nonclimatic origin”. In the case of the Tiljander series, Kaufman’s corrigendum is restoring the series to the interpretation of the peer reviewed article; in this case, the corrigendum appears to be doing the opposite: the original version seems to have implemented the interpretation of the original peer reviewed article, while the corrigendum seems to be making changes to the interpretation without submitting the changes to fresh peer review. At this point, I’m just asking the question in the way that I hope a peer reviewer would ask the question (and will probably include this question in a letter to Science on the topic.)
Climate Audit 
102 Comments
“The original conclusions
of the paper have been strengthened as a result.” Of course they have, dear.
Re: dearieme (#1),
pea & thimble:
The “results” are indeed strengthened. But their interpretation is weakened. Isn’t that clever?
So the dark Ages become a bit colder, MWP unchanged, LIA a bit colder and the hockey stick remains the same. Funny thing is, that whole hockey increase occurred before 1950. Not clear what happened to upside-down Tiljander, but probably not that much.
Of course, “those” cannot be named, and of course the four errors don’t matter.
What did cause that big jump in about 1920???
There remains a mountain of issues Kaufman has neither corrected nor clarified. Personally, I would like to see Kaufman’s response to the issue of Invalid Calibration raised by Hu McCulloch who wrote:
Perhaps Kaufman will clarify his reasons for not doing this in his final draft.
Re: Ron Cram (#5),
Actually, we haven’t even finished parsing all the errors in Kaufman et al. (2009) … we got interrupted because of the Briffa distraction. I wouldn’t be surprised if a second corrigendum were necessary. They should be aiming for correctness over speed of correction.
Re: bender (#6),
Oh I agree. I am just pointing out one I wish he would discuss. One of the biggest problems in science’s self-correcting mechanism is the fact a Corrigendum is not peer-reviewed. No one looks to see if it is complete in dealing with all of the problems or deals with the problems in the proper way.
Re: Ron Cram (#5),
In fact there is a lengthy response to
Hu’san anonymous person’s critique on calibration issues, see the updated Note D in the new SOM text.I think the word “robust” (*) is quickly becoming the most misused statistical term in the Team world. Hopefully someone calls paramedics if this torturing continues.
(*) e.g., “takes advantage of the most robust calibration”
Re: Jean S (#10),
I look forward to reading the link to the SOM text you provided. For some reason, it will not display for me. Perhaps their server is simply overloaded with requests right now. I will try again.
Re: Jean S (#10),
Okay, I read through Note D and see Kaufman provided some detail regarding how calibration was done, but I do not see that he addressed the issue raised by Hu that as the composition changes it has to be calibrated separately. I don’t know if Hu is right or not, but I cannot see where the issue was discussed.
OK, I can’t tell if that is sarcasm or not
They say their original from Fig. 3C is included, but it is not. The biggest problem is the upside down Tiljander and other varve proxies, but no mention of those.
The reason why it doesn’t “matter” whether Tiljander is used upside down or not, is explained by Grace Slick here. These proxies fall into the category of the “ones that mother gave you”. She also explains why Yamal does “matter” in the Kaufman reconstruction.
I wonder if the Canadian Press and the reporter who broke the story will publish a corrigendum…
What’s Behind the Green Door?
===================
I did not ever take pills or smoke caterpillar heads. That would be a foolish and incorrect thing to do.
No researchers in this field have ever, to our knowledge, “taken some kind of mushroom” to cause “logic and proportion to fall sloppy dead” in any reconstruction. It is somewhat disappointing to find this specious claim (which we usually find originating from industry-funded climate disinformation websites) appearing in this forum.
#14 bender – I agree. – mike
Everyone associated with CA, especially Steve, should be proud when such Corrigenda are published. The team may pretend to ignore you but they are listening!
Re: EddieO (#17),
Of course they’re listening. We do their work for free. But their responses are unresponsive, so there is actually little to be enouraged about.
Perhaps “listening” is to strong a word!
Steve,
Your passage that, “it’s not rocket science to determine who was right” seems archaic to me. Rocket science is simple compared with climate science so shouldn’t we be substituting “it’s not climate science” in the future?
I disagree Bender. As long as CA continues to produce valid and robust criticisms the pressure on the team to acknowledge the inadequacies of their data and analysis will grow. Of course it might take a another decade or two to make any difference to the popular science and urban myths they have generated over the last 20 years.
snip – forbidden word
All they can do is implement the message in a way that allows plausible denial that they got it from the messenger.
#21 Ron Cram
Just clicked on SOM link. Works fine for me (18:25 in UK).
Pielke, Jr. has commented on this post, saying, in effect, that Mann and RC are wrong and Kayfman was right to make the correction.
snip – let’s leave things where they are.
Typo under Fig 1: “Mann used not one by four Korttajarvi series…” you surely mean “but”
Hey, its progress not perfection.
I’ve added the following discussion to the post.
Another change in the Corrigendum is to Record 12 (Dye 3) as follows.
Dye 3 was one of the series for which I requested annual data (refused by Kaufman). I noticed today that annual data has now been placed online – in a pdf format rather than a digital format, a stupid paleclimate pet trick that prevents the use of scripts directly linking to their site. (Yeah, year, it can be turned into an ASCII file in 10 minutes or so, but it’s a waste of time).
No reason is provided in the Corrigendum as to why it is now believed to be appropriate to “omit the high-pass filter”, as compared to the procedures of the original article (see below figure). In this case, the correction “helped” Kaufman.
Figure x. Dye 3 (SD Units) Before and After Correction
Andersen et al (JGR 2006) online here reports the following in connection with filters at Dye-3:
As I read this paragraph, the purpose of the high-pass filter in Andersen et al 2006 (the Butterworth filter of order 3 with a cutoff frequency of 0.001 year^{-1}) was to remove a “longer-term variation of nonclimatic origin”. In the case of the Tiljander series, Kaufman’s corrigendum is restoring the series to the interpretation of the peer reviewed article; in this case, the corrigendum appears to be doing the opposite: the original version seems to have implemented the interpretation of the original peer reviewed article, while the corrigendum seems to be making changes to the interpretation without submitting the changes to fresh peer review. At this point, I’m just asking the question in the way that I hope a peer reviewer would ask the question (and will probably include this question in a letter to Science on the topic.)
Re: Steve McIntyre (#29),
Steve, was this proxy used with the filter in the paper and then the filter was removed in the Corrigendum?
Re: Peter (#41),
That’s my surmise from what they say. However, Kaufman refused to provide the annual data used in the original paper. A new version is at the Danish website (dated Sep 29) that looks consistent at first glance with the Corrigendum version, but I haven’t yet seen anything that looks consistent with the original article.
If anyone can provide a rational explanation for why Kaufman is replacing the version in the original article with a previously unpublished version in the Corrigendum, I’m all ears.
Re: Steve McIntyre (#42),
If this is the case, it seems Kaufman may have had to make a quick trip to the methadone clinic when his drug of choice became unavailable. Needed a hit to remain robust.
Re: Steve McIntyre (#42),
If the Corrigendum was peer-reviewed, then Kaufman would probably have to explain in the Corrigendum. I am still amazed journals don’t require peer-review for the corrections. Of course, it should probably be a different reviewer than the original.
This would be Ari Jokimäki, a Finnish astronomer with whom I have been having a long and mostly very tedious argument under my other netname, stroller, over the last six months or so here:
http://www.bautforum.com/science-technology/70431-general-agw-discussion-thread-74.html
The thread is now 74 pages long and I’m weary of the constant nitpicking of the assembled warmista. If anyone feels like helping out with the occasional drive-by, be my guest.
Re: tallbloke (#31), Methinks you’re doing an excellent job, but can do with a little help of some friends…
Re: Hoi Polloi (#43),
I make tactical errors and get stuff wrong too, but seem to have managed to keep the pot boiling. Please do sign up, drop in and sound off. 😉
9??!?! Holy Schmoley!
yeah, I have no idea what that means 😉
Re: Jeff Alberts (#32), Just a quick recap: When you have some data that looks like a normally distributed bell-curve when plotted around the median value, sigma is the standard deviation. That is: It is the distance from the median that would hold 68.2% of the data. Two sigma holds 95%, Three sigma hold ~99% of the data. Four standard deviations is 99.994%.
.
Somewhere around here people often make a decision that the remaining data is quite likely contaminated – it is quite far from the median – and they’re lumped together as “outliers.” Outliers are often examined in excruciating detail – checking the logbooks for the slightest irregularity, etc. And they’re often dumped after some consideration. (While paying attention to how dumping those points affects things.)
.
Seven sigma is 99.999 999 999 7440%, which would mean that a “true” datapoint that far from the fit should happen only once in 390,600,000,000 datapoints. Nine sigma is just that more outrageous.
.
But it doesn’t inherently mean the datapoint was poorly measured or inaccurate. In this case, it probably highlights “Hey, something unusual is happening that isn’t accounted for in your fitting model.” These are varve thicknesses. The local conditions could be dramatically changed by a non-temperature effect (Oil slick? Sunken vessel? Who knows.) Or – the local conditions could have a large temperature swing. Or – the true response of varves under the not-so-large temperature shift could have a strong non-linearity – meaning that the model needs serious help. It isn’t clear here (to me anyway) what, exactly, is diverting things. But a sustained nine-sigma deviation is a clear sign -something- is diverting from the basic understanding and relations on the earlier portions of the chart.
Re: Alan S. Blue (#34), thanks for that Alan. While I can follow the straightforward definitions offered elsewhere, this bit of context made it “real” enough to understand the issue. I opted for “Mechanics” in mathematics at school rather the “Statistics” which seemed too dry and theoretical. Never thought I’d regret that choice until recently.
Re: Jeff Alberts (#32), in a normal distribution, the chances of a 9-sigma event occurring are less than 2 x 10^-15. I’m not completely sure, but I’m fairly confident that some of the statistical techniques Mann employs assume normal, or at least nearly normal distributions. In any case, a 9-sigma event is a red flag that something has changed, as it would be virtually impossible for this to be part of any stationary distribution. If anyone working for me reported a 9-sigma event in any monitored value, I would tell them immediately to find out what changed in the process. I would consider it gross negligence to consider it part of an unchanging process.
Re Ron Cram #30, Jean S #10,
I wrote some remarks replying to Section D of the new SOM in Comment #30 of the Invalid Calibration in Kaufman 2009 thread.
Re: Hu McCulloch (#33),
Thank you for the note. I think I understand now.
Hey, I give Kaufman some credit for at least giving criticism some consideration. He could have gone to the bunker slappin’ high fives from all the com padres. He now knows that if you are going to be jousting with the jesters you should bring your competition lance.
Steve: As I noted before, I suspect that, unless Korhola had spoken out, he would have toughed it out. We’ll never know.
I agree with EddieO, but as it said it will take some years to elaborate and pass that to science.
This correction is being reviewed. That’s why for now it is only on an outside website.
Re: MikeN (#46),
It might not be. Nature didn’t peer review the MBH Corrigendum.
Re: Steve McIntyre (#48), OK, maybe not peer-reviewed, but Kaufman says the corrections are being reviewed by the editors at Science.
Re: MikeN (#50),
I don’t know where Kaufman made that statement. Was it public or in a personal communication to you? Do you think having the editors of Science review the Corrigendum is adequate? It does not leave me brimming with confidence.
Re: Ron Cram (#51), Personal communication. That could also mean peer-reviewed.
Once again leave it to a Real Climate alum to explain why McIntyre gets it all wrong, again:
[The proxy is correlated against the instrumental record before being amalgamated into the global record. So T_glob = avg(Proxy_i * corr(Proxy_i, T_ins)) (it isn’t, but its like that. Avg() is over i, corr is over the period of the instrumental record, the proxy series is assumed to run over a longer period). So if you reverse the sign of Procy_i, it makes no difference at all to T_glob -W]
Re: Thad (#47),
Connolley’s statement is completely and totally beside the point. Even a climate scientist should be able to understand when something is used upside down.
Hint to realclimate: bridges, ditches, farming…
Re: Steve McIntyre (#49),
Connoley is saying that the issue of the contamination of the Tijlander varve “proxy” is a collateral issue. He is saying that whatever interpretation is given to the varve record (thin-warm or thick-warm) is immaterial as the multivariate technique used only cares about the magnitude of a correlation and not its sign. The contribution (neglecting the contamination issue) will be the same regardless of the interpretation used.
Is there any validity in this?
Re: TAG (#59),
If there is, then why do they feel the need to ‘flip’ the proxy?
Re: TAG (#59),
He is correct because he is arguing the wrong point.
.
Whether the physical interpretation of the proxy is (thin=warm, thick=cold) or (thick=warm, thin=cold) doesn’t matter because they both mean the same thing. What does matter is that the physical interpretation of the proxy is not inverted so that thick=warm becomes thick=cold because now the correlation is broken. You’re positively correlating a decrease in apparent proxy temperature with an increase in actual temperature and asserting that the two agree! Mann’s method seized on the contaminated part of the proxy which showed *unprecidented cooling* and reinterpreted it to mean *unprecidented warming*! Complete and utter nonsense!
I’ve been attempting to discuss this with a lesser-known climate blogger on another forum, and he, too has been having difficulty in understanding the problem with the flipped proxy even in the face of Kaufman’s corrections, so I’ve resorted to what I hope is a slightly more concrete analogy. Does this seem aropos? Does it reflect an accurate understanding of the issue here (not a climate scientist or statistician here…)?
“…Say an agrarian population has kept detailed and accurate records of agricultural production for the last 1000 years. Historians have detailed and convincing reasons to believe that their per capita agricultural production was directly related to temperature for the first 800 years of the records, and let’s say that while a bit noisy, they do in truth have a significant direct relationship.
However, the historians who collected the records caution that relationship breaks down in the last 100-200 years, due to an invading nation that increasingly pillaged towns, salted fields, etc. Over time, this influence of the invaders increased, because of non-climate related reasons, and/or because of some temperture-dependent reasons (perhaps as the temperature starts to rise sharply in the last 150 years, the mountain passes between the two countries melt sooner and snow in later in the year), so that as the temperature increased during this period, country A’s agricultural production developed a strong inverse relationship with the temperature.
When you come across the data looking to use it as a climate proxy, as it’s been represented (with qualifications) by the historians, you (or your algorithm) sees the strong inverse relationship during the recent calibration period, and so decide the entire proxy is inversely related. Will that give you accurate an accurate record? Is that proper science?…”
Re: kmye (#53),
your analogy seems apt to me. This shouldn’t be hard to understand even for the Team. It’s hard to believe that they don’t understand the point.
Korhola’s input is a very promising development. Its up to colleagues to point out that choosing whatever samples you want, in whatever orientation, to reinforce your POV is unacceptable scientific practice.
Has anyone simply recalculated Mann’s/Kaufman’s work by using the proxy data the right way up?
Briffa’s response should have a direct effect on Kaufman et al: they should use the re-worked Yamal chronology (without the post 1990 part). Kaufman et al is missing from Briffa’s Table A.
Re: Jean S (#58),
Without Yamal the statement
might need some adjustment, but now we have many choices:
So, how to keep the last half-century as the warmest of the past two millennia?
Re: UC (#71),
If they choose to skip Yamal, what will be the result?
Re: Jean S (#77),
Something like this:
I’m trying hard not to comment – trying to leave lots of room for “giano and the dendro flood” – but this “upside-down” Tiljander issue – it has been explained before. The sign going in to the analysis doesn’t matter. But the interpretation coming out does. It’s the interpretation coming out that is counter to Tiljander. Connolley is perhaps so quick to assert his mastery of multivariate regression that he hasn’t bothered to look at the specific analysis? Do send him here.
Re: minimalist bender (#61),
I parsed through the code (once again) to see exactly what actually happens with regards to the sign issue in Mann et al (2008). Maybe I do a post on this (is there a need?) if time permits. Anyhow, here are results:
1) reconstructions with screened network: ALL reconstructions need to be recalculated. If the sign is flipped, pick-two correlations are negative and Tiljander series do not get selected.
2) CPS (without screening):
input sign is immaterial. There is a step that flips all series to the orientation such that pick-two correlations are positive. Especially, Tiljander series are always flipped to the orientation that the blade is towards the sky (they are upside down).3) EIV (without screening): input sign is immaterial. The output sign is determined in RegEM and it may (and does) vary from step to step.
[Edit: wrong statement striked out (see below #74)]
Re: Jean S (#63), one thing I have never understood is that Mann 2008 says
Clearly he was aware of the sign issue for lake sediments. So how could he then say
as if there was no sign issue for lake sediments?
Re: Jonathan (#64),
Yes, Tiljander proxies are in the category with the code 3000 (lake sediments), for which, according to original Mann statement, correlation should a priori be positive and one-sided test should be used (it is used, one can check that from the code). However, accoring to the original authors, Tiljander series should have a priori negative correlation (keeping the orientation as both in the original paper and in Mann’s data). So, indeed, one can only wonder the statement:
One thing I have never understood is that how it is possible after all these demonstratably incorrect statements, whose validity everyone can check from the code/papers, we still find relatively many intelligent people (like Connolley) desperately trying to defend Mann. That’s truly “bizarre”.
Re: Jean S (#65), thanks, I know it’s bizarre, I’ve just been trying to work out exactly how bizarre, and in exactly what way.
It looks like the most generous interpretation of this that can be made is that (1) Mann genuinely believed (incorrectly) that the sign of Tiljander should be positive, and (2) given that belief he then used it “properly” – so Mann 2008 is just a genuine error. The comment reply is still odd, but the statement “Multivariate regression methods are insensitive to the sign of predictors” could just be seen as blather.
As you say the continuing denial (recent comments on Pielke’s blog have provided some fine examples) are truly bizarre and staggeringly counter-productive.
Re: Jonathan (#66),
Yes, but it is also obvious that after the error had been pointed out to him, he had to be aware of, at least, the effect to the screening. On the other hand, it would be interesting to know, if he is even aware (as someone else seems to have written most of the code) of the fact that
CPS is always using proxies in the two-pick correlation orientation regardless of the intended sign. This can be seen from the CPS proxy selection file gridproxy.m, e.g.:for i=1:m1-1 % This is for searching annually-resolved proxies if (z(3,i)==9000 | z(3,i)==8000 | z(3,i)==7500 | z(3,i)==4000 | z(3,i)==3000 | z(3,i)==2000) &... x(kk,i+1)>-99999 & x(kkk,i+1)>-99999 &... z(1,i)>=ilon1 & z(1,i)=ilat1 & z(2,i)=corra n=n+1; %%%% low pass filter to 0.1 temp=x(kk:kkk,i+1)*sign(z(ia,i)); [smoot,icb,ice,mse0]=lowpassmin(temp,0.1); yc(1:kkk-kk+1,n)=smoot; locc(1:2,n)=z(1:2,i); end endNotice the line (z(ia,i) is the correlation value):
[Edit: wrong statement striked out (see below #74)]
Re: Jean S (#67), Some scientists don’t like to get their hands dirty, and have techies and students write their code, measure their tree rings, etc. So maybe Mann has no idea what the code does. who knows?
Re: Craig Loehle (#73),
Yes, and as you can see from my struggling (I’ve been going through that code a quite many hours and I’m fully fluent with Matlab) with the code, unless you’ve written it, it is very hard to see what’s exactly is going on. I think I said when the code was released that the code in Mann et al (2008) is one of the worst I’ve ever seen. If any of our students returned me code like that for any assignment, they would get F without further consideration.
Re: Jonathan (#66),
Thanks guys. Now I see that even I was thrown off by Mann’s blather. I could identify it as blather and refute it easily. But now I see that what I was refuting was a straw man. I’m glad you guys are on the case and communicating back and forth.
Re: Jean S (#65),
This is my reply to Connolley at Cruel Mistress:
The topic was “peer review”. Hence the closing line.
Re: bender (#85) —
Bender,
Connolley has remarked on your lucid Cruel Mistress exposition at Stoat. I submitted a response to Stoat‘s moderation queue. Here it is, edited for the altered context —
In a Stoat comment snipped yesterday as off-topic (reposted at Pielke’s blog), I was alluding to the issue raised by Bender at Cruel Mistress, albeit less eloquently. This later comment in the same Stoat thread focuses on the conflict between the physical meaning of the Lake Korttajarvi varve properties and their treatment by Mann et al’s CPS method. This is a central point that Bender raised, too.
This question of the flipped sign of the varve series was also being discussed in the comments of the Pielke post archly referenced by oh dear oh dear oh dear, as well as in this ‘Kaufman’ thread (comments #s 61-85) at Climate Audit.
In this instance–speaking as an outsider and coming late to the party–McIntyre is right and Mann is wrong. This is the case on multiple levels.
In basic, lay terms, the fundamental problem with the use of the varve series proxy in Mann et al (2008) is Garbage In, Garbage Out.
In the comment at Stoat to which I am replying, Connolley writes,
No. The problematic nature of the Lake Korttajarvi varve series is discussed in the Methods of Mann et al’s SI, page 2. The Tiljander personal communication which McIntyre sought but which Mann and Kaufman did not seek simply provides added specifics. The general point was raised and then wrongly (per Bender) “solved” in Mann et al (see Mann’s Fig. SF8a and certain figures following SF9, I can’t remember which).
Mann’s handling of this issue (e.g. in the PNAS Reply to McIntyre’s Comment) appears to be far out of line with what is understood to be proper conduct in the areas of science with which I am familiar. Specific remarks in another comment if you want them.
Re: bender (#85),
And there you have it folks. The answer to the issue of ‘upside-down Tiljander.’ Found only on the blogosphere. Thanks bender, et.al.
Re: minimalist bender (#61), For the Tiljander varve proxy, there are two problems. Flipping the sign of a physical proxy is silly enough because now we are going to assert that when the author of the study says it gets warm it really gets cold, but the Tiljander proxy actually has a change of behavior due to agriculture after 1800 or so. This means that after this point, whatever the proxy does is NOT related to climate. Picking it as a proxy and correlating the post-1800 part with instrumental temperature data is pure nonsense. It is like putting a plastic cow into the sausage grinder with the real cows and claiming that your patties are all beef.
Re: Craig Loehle (#72),
Yes, this, as expressed equally in AMac’s #68, I was aware of. I keep forgetting that the argument is not, in fact, about the signs going in on a “multivariate regression”. The real issue is the splitting of the calibration phase and the reconstruction phase, which is not “multivariate regression”. The code incorrectly assumes that a proxy-climate divergence never happens at the calibration-reconstruction split-point. But it happens. Clearly. This is pure and simple a coding error – caused by confirmation bias. But can you blame them? Good proxies are not supposed to diverge 180° as you move from calibration phase back into the reconstruction phase! GIGO! They simply assumed that they weren’t working with G. Wrongo!
Re: bender (#83),
bender, you say in respect to Mann’s use of Tiljander:
there are multiple problems in Mann’s use of the Tiljander series and reasonable people can disagree as to what the “real” issue is. My own first characterization of the matter – and one’s first comments are sometimes (not always) the clearest – was:
In retrospect, I would rather have said “data mining paleoclimatologists”, but otherwise the point is valid. Among other things, this is a classic spurious correlation, every bit as spurious as the correlation between temperature and incidence of the name Gavin.
Re: Steve McIntyre (#89),
First, I hope you are not insulted that I attempt to rephrase your discoveries in my own terms. I think it helps to have the same result described two ways by two independent perspectives. Although I will not always get the nuances exactly right, I think it does far more good than harm.
.
Second – and case in point – when I say “this is pure and simple a coding error” I mean at the source of the problem is a coding error, based on a flawed assumption that a proxy will never switch sense. But, it goes beyond that. You are, in my opinion, correct: there is a confirmation bias at play here. This is important to understand because people like Connolley accuse you of “wrapping yourself in conspiracy theories”, when in fact you (and I) have stated the exact opposite. There is no conspiracy. There is just confirmation bias: “yep, that looks about right, let’s publish”. This is VERY hard to prove, of course. But over the years you have rooted up enough errors in Team work that I am certain there is a confirmation bias operating. The errors are always small, but significant, and in the alarmist direction that favors career advancement.
.
If it was just once I would say you’re apranoid. But there is obviously a pattern here. For example, your “pea and thimble” analogy is dead on the money. I watched it in real time with Martin Juckes. I watch in time-delayed slow motion with Mann, and now with Briffa.
.
For teh record, if I do make errors in my characterizations I am most happy to be corrected. Real scientists *love* to be corrected – even if (or especially when?) it hurts.
Re: bender (#90), Hey, I appreciated your comment. And yes, you and I are of one mind on confirmation bias. The Team are very good at detecting errors that go the “wrong way”, but tend not to cross-examine results that go the “right way” equally diligently. Richard Muller expressed this concern way back when we were being put off in our original Nature comment.
When these things go on and on as this has, sometimes one loses track of where one started. When I looked back at the original post, it had a nice clear focus on spurious regression which is, of course, an ongoing concern of mine. However, as with many of these issues, there are multiple interrelated problems and reasonable people can disagree on which is the most important. However, ultimately I don’t accept that reasonable people can disagree on whether this proxy was used correctly in Mann et al 2008. It obviously wasn’t.
In this comment chez Pielke and Connolley, I try to walk through the issue of the polarity of the Lake Korttajarvi varve proxies used in Mann et al. (2009), and discussed above in comments 59 – 67.
My conclusion aligns with that of DaveJR #62. Local human activities during the calibration period (1850-1995) appear to have caused the CPS algorithm to assign thicker, more mineral-rich varves to warmer temperatures. This is the opposite of Tiljander’s contention. The CPS calibration mistake is then propagated through the reconstruction, 400 – 1850.
Mann’s Figure S8A shows two northern hemisphere CPS reconstructions. The black line is plotted from eight screened proxies. The green line enlarges the proxies to include the four (inverted) Lake Korttajarvi proxies and three additional proxies.
The green and black lines have almost identical traces. If this analysis is correct, 4 of the 15 green proxies must yield bad temperature information for the entire duration of the reconstruction. Thus, the superposition of the two lines seems hard to explain.
Re: AMac (#68),
Yes, in CPS, the orientation of proxies is decided only once. After that CPS is nothing but scaling (by positive numbers) and averaging the proxies. Now, as I showed in #67, Mann’s version of CPS actually opportunisticly flips the orientation according to the correlation in the calibration period
(regardless of the physical interpretation and leading to the wrong orientation for Tiljander series).That procedure is easy to demonstrate to generate “hockey sticks” from noise only, similarly to what Lucia recently demonstarted for “cherry picking”. I do not remember if this effect of Mann’s CPS algorithm has been noticed here before, but I do recall that the same thing was used in the original version of Juckes et al. (which he corrected after it was pointed out here).[Edit: wrong statement striked out (see below #74)]
Re: Jean S (#69), I don’t quite follow your argument about opportunistic sign flipping. The code is messy, but think about the different if statements. The first one selects for the case where the sign is meant to be positive, and it requires positive correlations, so in this case sign(z(ia,i))=1 as it should be. In the second if it selects for the case where the sign is meant to be negative, and requires negative correlations, so in this case sign(z(ia,i))=-1 as it should be. Only in the third case, where the sign is unknown, does it permit either positive or negative correlations, and in this case it assigns opportunistically. Then in repeats the whole process for decadal proxies.
As far as I can see case (3) is pure data mining, but I can’t yet see the sign problem in cases (1) and (2).
Re: Jonathan (#70),
Ah! You are absolutely right! I was confusing with the EIV selection code INPUTall.m. So the effect is even “funnier”! In CPS and in “all proxy”/”no screening” situation (corra=0, corra2=0), there is still some screening going on: all proxies with opposite two-pick correlation sign to the “a priori known sign” gets thrown out. This mean especially for Tiljander series, that if their signs are corrected, they never enter to CPS reconstructions. Correct?
Re: Jean S (#74), that’s my reading: if Tiljander is used with the correct sign then it fails the correlation test (even with case ‘FULL_’) and does not enter the reconstruction. But this is the first time I have looked at the code.
Re: Jean S (#69),
Mann et al.’s Fig. S8a (PDF).
This is a plot of Temperature Anomaly vs. Time for 200 AD – 2000 AD. The figure legend for S8a alone would read Comparison of long-term CPS Northern Hemisphere land reconstructions (full global proxy network) both with and without the seven potentially problematic series discussed.
The 15 Northern Hemisphere proxies on which the reconstructions are based are plotted in Fig. S9…
… stops…
Ahh. “Full global proxy network” appears to mean that the green trace in Fig. S8a is not constructed from the 15 Northern Hemisphere screened long-duration proxy series plotted in Figure S9. Rather, it is generated from the hundreds (?) of proxies listed in Dataset S1.
That would explain why the removal of four flipped proxies and three other problematic proxies doesn’t affect the shape of the reconstruction.
There’s a possible minor bookkeeping problem here, though. Besides the four Tiljander proxy series, the three others identified as problematic by Mann (pg. S2) are —
These three series are in the Dataset S1. But it appears that two of them are Australian, thus Southern Hemisphere proxies.
Re: AMac (#78),
On further reflection, there seems to be something very wrong with Mann (2008)’s Figure S8a. If the green line is plotted from the “full global proxy network” of the hundreds of series listed in Dataset S1, then the effect of removing 3 “problematic” proxies and the 4 inverted Lake Korttajarvi varve proxies should be mimimal where there are lots of remaining proxies (recent times).
But the effect should become progressively more evident as one goes back in time, say to the 200 AD to 800 AD era, where there are very few proxies.
By this reasoning, the black line in Figure S8a should diverge progressively more from the green line as one moves back in time. But it does not seem to do so to any great extent.
How can a an algorithm that performs temperature-anomaly reconstruction be so rugged that it produces equally good renderings of paleoclimate, irrespective of the presence or absence of bad input data?
So are all of the Kaufman proxies now online with annual values? For some of them I could only locate them in 2-3 year values, or 5 year values.
It’s good to have the full-strength bender back in these perilous times.
Re: Tom C (#82),
Just 3 days. This is an important phase.
I hate to sound gee-whiz about it, but this is really great stuff. What must really annoy the other side is the amount of fun you are having with this. While their professional reputations and occupational relevance are on the line, you guys are over here treating it like a baseball game with SM, the big 20-game-winning right-hander on the mound. (I could fill out the whole line-up card, but I think I’ll stop now.)
Re: theduke (#91), exactly 🙂 Integrity is more fun, plus the Universe itself conspires to help integrity. And it’s a holistic integrity, that allows paradoxes and dissent, it’s not just the obvious scientific issue. But it’s a privilege and a sense of participating in significant history, to see the real science being distilled in the process, drip by drip … er, like moonshine…
…heck, I didn’t mean to sound patronising, I can well imagine what committment has cost Steve. And others.
Hi Steve
following your & benders advice, i’ve been ‘reading the the/early blog’, still not finished!!.
whats funny is the number of early posts by you, with no/or next to no comments, how times have changed!!
the number of years you’ve been plugging away at the multiproxy problems is astonishing & you deserve to be ‘pissed’ at times to say the least.
how anybody can look at a HS graph shaft & think ‘yes, that makes sense in the real world i know & the past history i read about’ makes no sense to me (why no comment from histos/arceos on this?).
your early posts are exactly how best a newbie to this debate/ blog can start to get to grips with above.
cheers
dougie
Re: dougie (#95),
Good for you. Are you going through sequentially? What chapter are you on? Now you see why I say: READ THE BLOG. You must read it to see the Team patterns. NONE of this behavior is new. Do like dougie; read it.
Re: bender (#96),
hi bender
i’m at http://www.climateaudit.org/?p=95 post
a feb 2005 SA article by David Appell about M.Mann funnily enough.
i like comment 41 since Patrick asks the same questions as me,
only better & more concisly.
Re: Patrick Trombly (#41),
be at it till xmas i reckon, see ya!!
Re: dougie (#98), post cliff notes for me when u r done. pssst. dont tell bender.
Re: steven mosher (#99),
had to google – CliffsNotes (formerly Cliffs Notes, often, erroneously, CliffNotes, but originally Cliff’s Notes) – got it now 🙂
if i had a big enough brain i would give it a go. but dont hold your breathe waiting.
psst. it’s ok, benders attending to your pool as we speak. it’s our secret.
ps. how do you get smilies in comments?
Re: dougie (#100), oops, just found out!!
>As I noted before, I suspect that, unless Korhola had spoken out, he would have toughed it out.
Steve, do you have any experience in dealing with Kaufmann beyond the e-mails last month?
Given your conservative nature in making accusations, as others have noted, I’m surprised the degree to which you lump him in with the rest of the Team.
The restructuring of these old threads is very nice.
.
Can’t wait to move well off “climate gate” and back on to auditing.
I found this comment at Pielke, Jr. (from Oct. 2009, in a discussion of “upside down” data from lake sediments) to be a very helpful take on how Mann & co. have managed to prolong these debates for years:
succinct summary of one Mannian method
2 Trackbacks
[…] See Climate Audit: ‘The Kaufman Corrigendum’ […]
[…] So what this means is that Under either method (CPS or EIV) it is not possible to get a validated reconstruction to before 1500 without the use of tree rings, or the Tijlander sediments. The tijlander sediments were used incorrectly and upside down from the original published version and a corrigendum by Kaufmann et al. (who also used it upside down) was issued pertaining to this. https://climateaudit.org/2009/10/26/the-kaufman-corrigendum/ […]