NASA blogger Gavin Schmidt as part of his ongoing attempt to rehabilitate Mannian paleoclimate reconstructions, characterized here as dendro-phrenology, has drawn attention to a graphic posted up at Mann’s website in November 2009. In this graphic, Mann responded to criticisms that his “no-dendro” stick had been contaminated by bridge-building sediments despite warnings from the author (warnings noted by Mann himself but the contaminated data was used anyway.) I’ll show this figure at the end of the post, but first I’m going to show the “raw materials” for this “reconstruction” and my results from the same data.
I’m going to show a lot of plots of “proxies” today. The intuitive idea of a proxy is that the thing being measured (tree ring width, sediment thickness, ice core O18, etc) has a linear relationship with a temperature “signal” plus low-order red noise. Therefore, if the temperature “signal” is a hockey stick, the various proxy plots should look like a hockey stick plus low-order red-noise. I encourage readers to look at the no-dendro no-Tilj data for Mann’s November 2009 example with that in mind. If the topics were being discussed by proper statisticians, the properties of the “noise” would be discussed, rather than ignored.
To illustrate the calculation, I’ve picked the AD1000 Mann 2008 data set as an example since it covers the MWP. I’ve used the late-miss version (calibration 1859-1949) to work through, since it will give a look at any potential “divergence problems” in non-dendro data.
There were 29 “proxies” in the data set- 11 sediments, 2 “documentary” (both Chinese), 9 speleo and 7 ice core. Eleven of these were annually resolved; the other 18 were “decadal” resolution. 22 were NH; 7 SH.
The first step in Mann’s algorithm is determining the orientation of speleo and documentary proxies through their after-the-fact correlation to instrumental data. (The orientation of other proxies is presumed to be known a priori). In this network, there were 11 speleo+documentary proxies and 5 of 11 were flipped. (Interestingly, it is possible in Mann’s algorithm for the same proxy to have opposite “significant” orientations depending on the calibration period.)
The next step is to screen out proxies that do not have a “significant” correlation to gridcell temperature. Although we’ve heard much invective against the meaningful of r^2 statistics from Mann, Schmidt and others in the context of MBH98, Mann then uses correlation (r) to screen series in Mann et al 2008. (Perhaps it is the squaring of the correlation statistic that Schmidt takes exception to.)
There were 16 proxies that “passed” Mannian significance: – 3 of 11 sediments, both “documentary (Chinese), 7 of 9 speleo and 4 of 7 ice cores. Seven of 11 annually resolved passed; nine of 18 decadally resolved passed. 12 of 22 NH passed; 4 of 7 SH passed.
In the figure below, I’ve plotted all 22 NH “proxies” (standardized), coloring the “rejected” proxies in green. I don’t think that anyone can reasonably look at these 22 series and say that the individual “proxies” can be reasonably interpreted as different linear transformations of a Hockey Stick plus low-order AR1 red noise or that the individual proxies look much like one another. They are a hodge-podge to say the least. This is the problem of proxy inconsistency that I’ve talked about frequently and that Ross and I reported in our comment at PNAS in Mann 2008. Mann either didn’t understand or pretended not to understand the problem, which is fundamental to the entire enterprise of proxy reconstructions and readily apparent merely by plotting the “proxies”.
While “ex post screening” by correlation is accepted as a given by realclimatescientists, ex-post screening by correlation is not a statistical procedure that is recommended or discussed in Draper and Smith or standard statistical texts. The tendency of this procedure to produce sticks from red noise is well known in the technical blogosphere (Jeff Id, David Stockwell, Lubos Motl and myself have all more or less independently noticed and reported the phenomenon, with David publishing a short note in an Australian mining newsletter that Ross and I cited in our PNAS comment. However professional climate scientists appear unaware of the effect and it remains unreported in the PeerReviewedLiterature.
The top left proxy (192) is an interesting one. It is Baker’s speleothem record from Scotland that was discussed at CA in early 2009 and here as an interesting example of Upside-Down Mann. In the orientation applied in Mann’s no-dendro no-Tiljander reconstruction endorsed by Gavin Schmidt, Scotland is shown as having experienced the unique phenomena of the Medieval Cold Period and Little Warm Age – bizarro Hubert Lamb, as it were.
The “proxies” show little evidence of an overall pattern, let alone a Stick.
Figure 1. 22 NH No-Dendro No-Tilj Proxies in M08 AD1000 network, rejected in green.
Next, here is a summary plot of the 12 NH “proxies” that “pass” Mannian screening, this time showing flipped proxies shown in red. The top left proxy is still the speleothem with the Scottish Medieval Cold Period and Little Warm Age. This is the same as the above graphic where proxies are accepted. The proxy with the hockey stick shape here is Fisher’s Agassiz, Ellesmere Island melt series, a proxy which has been around for a long time, used in Bradley and Jones 1993, for example.
Figure 2. 12 “Passing” NH No-Dendro No-Tilj Proxies in M08 AD1000 network, flipped in red.
In hte next step in Mann 2008 CPS, the series are Mann-smoothed (Butterworth filter plus Mann endpoints). The smoothed series are then re-standardized on the (short) calibration period. The smoothing of the ternary series in the third column ( a Chinese documentary series) has an interesting effect.
Figure 3. 12 “Passing” NH No-Dendro No-Tilj Proxies in M08 AD1000 network, smoothed and re-scaled on (short) calibration period.
The proxy series are then averaged within a gridcell. You’ll notice that some gridcells are identical. This results because the Mann algorithm contains what I called (in 2008) a “stupid pet trick” – if Mann transcribed the location of a proxy as being exactly on the border of a gridcell (e.g. 25E), the proxy is allocated to both gridcells, in effect doubling the weight of the proxy. In the case of the Socotra stalagmite, the stalagmite is not actually located at 25E and the doubling occurs only because of a transcription error – not that the doubling makes any sense in the first place.
The gridded data are then re-centered and re-scaled to match the mean and standard deviations of the corresponding gridcell instrumental data – thereby yielding an estimate of the gridcell temperature. The 12 NH gridcells are shown below.
Figure 4. 12 NH Gridcells from averaging Mannian proxies
Of the resulting 14 gridcells, 8 are north of 30N. Mann attempts to balance the weights through an odd Mannian mechanism of re-gridding the north of 30N cells into 10×10 cells, averaging the data within each gridcell. This reduces the number of gridcells from 14 (NH – 12) to 10 (NH – 8). The series with the Scottish Little Warm Age survives these various operations pretty much unscathed. These again are a sort of temperature estimate.
Figure 5. Eight re-gridded NH gridded series.
Mann then does a weighted average of the gridcells – weighting each by the cos (latitude) – to yield a NH (and SH) estimate.
The figure below shows the No-dendro No-Tilj for the AD1000 network, using Mannian methods endorsed by Gavin Schmidt.
Figure 6. Emulation of Mann 2008 No-Tilj No-Dendro Reconstruction.
Now here is the version at Mann’s website, which looks nothing like my emulation with the 29 proxies (16 screened) from the AD1000 no-Tilj no-dendro network.
Figure 7. Mann Notilj No-dendro reconstruction.
What accounts for the difference? I’m pretty sure that this calculation is pretty close to the M08 calculation for the corresponding step. I’ve groundtruthed my R-emulation against Matlab intermediates calculated by UC and Jean S in 2008. Because the CPS calculations are, at the end of the day, weighted averages, the composite is going to bear some relationship to the proxies – hence the methodical plotting of intermediates at each step to benchmark the calculation. So while there’s always the possibility of a misstep in emulating Mannian calculations, I don’t see how such a misstep would alter the general shape of the AD1000 CPS calculation (since the general shape can be discerned in the average at each stage.)
Here’s where I think the difference lies. Mann’s graphics all show the results of spliced reconstructions rather than what you get with proxies going back to AD1000. The provenance of the network used in Mann’s November 2009 revision of a figure in his SI isn’t described as clearly as it might be. My interpretation of the figure is that the network includes 71 Luterbacher gridded European series which use instrumental temperature data.
It is my surmise that in its latter portion, the stick-ness of the “new” no_tilj no-dendro reconstruction derives from splicing the Luterbacher gridcell data (using instrumental data) onto the horrible no-dendro reconstruction. I’m not 100% sure of this, but that’s my surmise. I’ll experiment with the splicing steps on another occasion.
Make a stick, make a stick, Michael Mann
Make us a stick as only you can
Flip it and smooth it and pick it to be
In the report for IPCC.
Update Aug 1, 2010: Script is http://www.climateaudit.info/scripts/mann.2008/benchmark_manniancps_blog_20100730.txt . I added a couple of operations at the end to calculate the CPS from the composite shown in the post and to calculate verification stats. The script here is used to step through; it is wrapped in a function manniancps that reconciles perfectly through the regrid and very closely to the composite.
Climate Audit 
106 Comments
You state re Fig 1 “rejected” proxies in green, but the caption says green is “passed.” Not quite sure of the meanings here, but a possible error?
Steve – fixed. Running text was right.
Wasn’t curling notably popular and widespread in early modern Scotland?
At Gavin Schmidt’s ‘Collide-a-scape’ thread “The Main Hindrance to Dialogue and Detente”, [archive] the third version of Mann08 Fig. S8a (the multicolor image at the tail of the body of this CA post) was the evidence that Gavin called upon to prove that Mann08’s use of the Tiljander proxies “doesn’t matter” as far as the merit of the paper’s paleotemperature reconstructions.
AGW Consensus advocates use it for the same purpose in the ongoing ‘Tiljander’ thread at ‘Not Spaghetti’, Arthur Smith’s blog.
If Steve McIntyre’s emulation is correct, the credibility of the only eyewitness called to the stand has been placed in doubt.
Gavin’s definition of “doesn’t matter” means, in laymans terms, “it was crap to start with”. He quickly follows with the “new” research shows” blah blah blah.
Regardless of AGW or not Gavin is a poor representative of AGW “science.”
The main hindrance to dialogue and detente is (insert team members name here) inability to admit that at least one flaw, however small it may turn out to be, in the teams work.
Cheers,
Martin
“Medieval Cold Period” and “Little Warm Age”.
Now that had me chuckling.
Simultaneously lucid and horrifying, thanks.
Maybe I can help…
Load your graph into a graphics program (flip it so down is up and up is down). Then time shift it so the peaks and valleys line up. Look up a fast rising temperature record from a busy airport. … then tack it on the end (after normalizing in a fashion that gives the results you need).
A bit tongue in cheek I admit — but I was able to match it up — with a few well chosen adjustments here ann there I admit — but all acceptable under Socio-paleo-pseudo-scientologistic rules.
Try it — it works! Forget that statistics stuff — this way is simpler.
“Here’s where I think the difference lies. Mann’s graphics all show the results of spliced* reconstructions** rather than what you get with proxies going back to AD1000. The provenance of the network used in Mann’s November 2009 revision of a figure in his SI isn’t described as clearly as it might be. My interpretation of the figure is that the network includes 71 Luterbacher gridded European series which use instrumental temperature data.”
* Splicing or mixing?
** the whole thing is a reconstruction, no? What do you mean by the term in plural? Do you mean proxies? Some sort of intermediate results (by grid square)?
Really hard for me to follow your explanation and I’ve followed this thing for a while. It sounds like you’re talking to yourself, not explaining a concept to the readers.
Steve: If you find my explanations unhelpful, you can always consult the original article. 🙂
Re: scientist (Jul 30 20:00),
Actually if you think about what you wrote you did get the point. At least I think you did. 🙂
A. Someone else’s writing does not excuse yours.
B. Editorial demands make papers easier to read than stream of consciousness, same-day-published blog posts.
C. Clear writing is clear thinking. In science, engineering, business, military, etc.
D. You don’t even have a citation for the “original article”.
Steve: there is a link to Mann’s website. Mann does stepwise reconstructions. “Splicing” of reconstructions means splicing of the stepwise reconstructions. In any given blog post, I’m afraid that I don’t necessarily re-define terms and, to that extent, some familiarity with the methods is often presumed on the part of readers. I try to write clearly, but unfortunately I don’t have time to recap things in each blog post. There are other excellent climate blogs if you want articles on a more popular level. Or other posts at this blog may interest you.
I’ve read your blog for years. I’m also very well read in other fields including science.
You get to make the headpost. I get to comment.
My comment is that your argument is so hard to follow, so far from any sort of normal technical or business report in any field, that you don’t even make clear assertions for the reader to judge, to react to. That’s my informed comment.
A link to a picture is NOT a link to the article and it’s not a citation. There’s a reason why people use them. This is a small detail, but a perfect example of why your work is not worth dealing with. Add in the mislabeled graphs, the snark, the meandering, the off-topic stuff. It’s a mess.
Don’t bother rebutting. It’s not debateable.
Most people with at least average intelligence are able to read and understand the blog, with some effort.
Some points from this anonymous writer are valid. But his underlying primary assertion is not.
Scientist.
If the blog is truly so difficult to decipher, then why have you been reading it for all these years.
Not that he needs an advocate, but if you do have a particular question, then why not ask it?
Sorry, but your comments seem to come across as a bit pedantic.
You can’t understand it. It must be wrong. Thanks for clearing that up for us.
Hmm, judging a blog post as if it were a scientific article?
At least in my field, the software industry, it is quite common for researchers to co-author applied-science papers with non-scientists. It is widely accepted that many good research ideas originate from the business side, but if the working professionals have the training and inclination to write an academic paper on the subject (and that’s not always so), they practically never have the time.
It seems to me as if Steve’s blog would be a great source of ideas for scientists in the field. They could follow up on his findings, write papers and have Steve as co-author or advisor.
I will refrain from speculating about why this is not happening.
I personally have gotten lots of ideas from this blog and have further developed and published them.
Craig:
Excellent, and apologies. I should have written “why this is not happening more often”. Perhaps I should also have checked Google Scholar first – there were more references to Steve than I had expected – I’m happy to concede my mistake.
The thing that struck me when reading through the details of the above post, was the simple (almost childish) nature of the process used by Mann. This really isn’t rocket science. It seems to me that the thing that makes it all look and sound rather complex is the inevitable use of vast amounts of jargon. That has the effect of making things seem very technical, and almost indecipherable. But once you start to translate all the confusing words, terms and abbreviations you are left with something that really isn’t very intellectually impressive.
Have another go at trying to understand it, Mr “Science”.
I couldn’t agree more.
I also think that the defences put up by ‘the team’ (as they are referred to here) are the only thing keeping this whole argument alive.
It’s just like a really slow motion game of chess…. we have been at ‘check’ for years but, as long as they don’t make a false move, the team can just keep moving bishops, queens and castles around, never getting to ‘check mate’
The climate gate emails show this.
I was reading the comments on Oppenheimer’s response thread over at Roger Pielke Jr’s blog…… (Blood Bath would be a good term to describe it).
It doesn’t take a rocket scientist to see that Mann’s stuff holds about as much water….. and I thank CA for showing me that…
The correct chess analogy is for the losing side to not move at all – if there are no time constraints, this tactic works very well indeed
In fact, civil wars have been started this way 🙂
The analogy of ClimateAudit blog posts to business or technical reports is unhelpful. To me, a better analogy would be to a contribution to an ongoing seminar or workshop. In the seminar, the professor expects the student participants to be prepared. The expectation is that they will have read and absorbed the background material and and will be prepared to contribute. Time and space are limited and sufficient preparation is expected and is necessary to make the presentation useful. Students who have not done sufficient preparation cannot expect that their lack of preparation will be made up during the seminar. I believe that this is the expectation of this blog. it is certainly the expctation that I have found in my business and professional life.
Steve: I agree with the analogy of an ongoing workshop. People also complain that I spend too much time on backstories and background. It amazes me that there is as much public interest as there is in technical discussions.
I had no trouble following. Unfortunately, there aren’t many secrets in Mann08 hockeystickization anymore.
Defending its false result is work for a sophist or politician.
“You get to make the headpost. I get to comment.”
Agreed. Now tell that to RC.
I understood the post no problem. Mind you, I am Scottish and know all about the Scottish Medieval Cold Period and the Little Warm Age. (Someone mentioned curling – we invented that during the Little Warm Age by floating blocks of granite on the water.)
I loved Steve’s poem at the end…
Re: Jimmy Haigh (Jul 31 05:37),
This wouldn’t be the Jimmy who used to post on AH and was a friend of PoD would it?
Hmm? I don’t think so. What is AH and who is PoD?
There are lots of Scotsmen called Jimmy – I was in a pub once in my home town – population 2000 – and there were 7 guys in at the time and we were all called Jimmy.
Hey, Jimmie – got the time?
How’d you know my name was Jimmie?
Just guessed…
Well, ye can guess the feckin time!
Re: Jimmy Haigh (Jul 31 13:04),
It wasn’t just the name. It was your writing style. AH and PoD are not things to be described here.
YUP! you are right. All The Scientist has spoken, go dosomething else.
ETA:
I was struggling to see the differences a bit, aside from the lack of stickiness at the end of SteveMc’s graph.
So I’ve done a very approximate overlay of the two with approximate scaling here:
(First version seemed to be zapped, please delete if duplicated)
That is great! I’d love to see a non team climate reconstruction.
So would I – it is sorely needed. And for reference to instrumental temps, we can include UAH satellite temps instead of rocketing surface temps.
_MichaelM
What’s fascinating to me is to look at all the original “proxies” above and NOT see any Hockey Sticks in nearly all of them, “passed” or not. It is my understanding that a Hockey-Stick-in-each-proxy is what SM was expecting way back when he first decided to look into this stuff. It seems to me that one does not need to look any further than these “proxy” graphs themselves: “The dog does nothing in the nighttime”, but Mann uses a kind of statistical sleight-of-hand to make it appear that the dog really did bark, and bite too! (Didn’t Mann write “Der Zauberberg”? It seems to me that this is this Mann’s Hockey Stick: merely a mountain of magic, nothing more.)
But this plot seems to be more Der Runenberg than Der Zauberberg.
I loved the magic mountain – my favourite novel. By the way, I found Steves post perfectly understandable – but then I have been reading Steve for many years.
‘Scientist’, I don’t mean to impugn your honesty but I would like a little more evidence for your alleged familiarity? Only asking.
Or maybe it’s a kind of Doktor Faustus? All ends badly, anyway.
For novice readers, could it be clarified if the flipped graphs are shown flipped in this post, or are they shown correctly here, but Mann flips them when he uses them? I had assumed that they would be shown flipped in figure 2 since they are shown in red, but this is not the case as they are identical to figure 1. This is why I’d be interested knowing if the red graphs are shown flipped or not.
Also, what is the definition of flipping? Is it a reversal of the sign? If so, who decides where the ordinate of zero is located? How are these zero ordinates correlated between all the proxies? Or are these stupid questions?
Steve: for some proxies, more negative is interpreted as warmer by the original scientists and flipping the proxies is fair enough for making a consistent data set where you can center, scale and carry out averages. There’s nothing per se wrong with this sort of flipping. So don’t get bent out of shape on this point.
However, opportunistically deciding after the fact which way is “warm” is unattractive. If a scientist doesn’t know a priori which way is up, then it isn’t a very plausible proxy. And if the orientation differs depending on calibration period, it seems like an unattractive proxy.
I colored the flipped proxies only to keep track of them.
I’m still left a bit uncertain on this particular point of presentation:
1) Some proxies are coloured red. This is to indicate that they are flipped *as far as their contribution to the Mann 2008 is concerned*. Yes?
2) Where a flipped proxy is plotted in your own graphs above, are you showing it in it’s original “raw” orientation state, or in it’s flipped state?
One problem is that – for my eyes at least – the proxy index numbers at the top left of each sub-plot is too small to be easily read.
Assuming you are using Internet Explorer go to:
View->Zoom->150%
Or whatever level is comfortable. Firefox is similar, as I suspect are other browsers.
Firefox = CTRL and roll your scroll wheel on the mouse if you have one. ;=)
Ah, I can make it out now for at least one example:
Series 203 appears in black in the top lattice graph in Figure 1, labelled “unflipped”, and in the red in Figure 3, labelled “flipped”.
So, the series are always presented in your lattice graphs in their unflipped orientation, and occasionally coloured red to indicate that their contribution to Mann 08 is in flipped orientation.
When choosing to accept or reject a proxy, is it possible that Mann’s code is written to check each proxy in both orientations and to accept the proxy if either orientation happens to correlate to the instrument record? This would be a sort of lazy programmers way of dealing with data of mixed orientation. Rather than figuring out the proper orientation as a first step (which would probably have to be done manually), just check it both ways (a couple lines of code) and accept anything that matches. They then don’t bother to check if the accepted orientation is the correct orientation after acceptance (because it would again require manual intervention).
Steve; he only does this for speleo and “documentary” proxies. Documentary “proxies” include things like precipitation estimates, such as the rain in Spain which fell mainly in the plains of Kenya.
Mann’s code is available, and it isn’t too difficult to determine how individual proxies are treated. There is a corr(z(i)) function that is the main actor, and just see if sign() or abs() are called.
Steve: to Mann’s credit, his code is archived and is a guide to methodology – particularly given the weirdness of many of methods and given that results are not always clearly reported.
For visuals, I did an overlay and found it had already been done by mrsean2k. I’ve used different vertical scaling as it seem arbitrary.
There is a step I am missing here. The introductory graphs eyeball about an average of zero Y axis units in each case. When you come to the punultimate graph (Steve’s) there’s a minus 0.1 units offset on average,then a minus 0.4 deg C offset on the ultimate Mann composite graph. Given the normalising and scaling that has gone into the raw graphs, how does the final graph show such displacement? Is it by application of an eaquation where x proxy units = Y deg C +/- a constant?
Having followed this topic for some years now, I cannot understand the recalcitrance of Mann, for surely the weight of arithmetic is against him. There are some learned societies that would reject this continued presentation of prima facie flawed data and invite the inventor to resign, or else.
Would you really expect proxies from across the globe to exhibit much in the way of common trends in a period with weak external forcing?
This would be easy to test with pseudoproxies derived from a long model integration.
Could someone explain, in idiot terms, if you please, the difference between ‘pseudo-proxies’ and ‘actual’ proxies? My understanding is that the former are some kind of neat replication derived from the GCMs – how are they used, what is their status?
Steve: pseudoproxies are indeed derived from models. Mostly they seem to be a signal plus red noise i.e. they don’t model highly relevant behavior of actual “proxies” – which, if they had such properties, would leave no doubt as to the actual history.
While “ex post screening” by correlation is accepted as a given by realclimatescientists
Words fail me. If that is the procedure, then what is the test it is supposed to be performing? This 101 stuff. I will never believe anybody on any side of any argument that fails such a basic principle of testing, which is that your findings are not merely the product of your assumptions. Who the hell does Mann think he is kidding with this nonsense, after a decade of his work being knocked over by unpaid non-climate scientists?
Dear Steve,
I’m a great admirer of your blog and your work in auditing climate science.
I always found the almost super-human calm, cool and objectivity shown in this blog in the face of ignoble critique to be one of your greatest strengths. In this article, technically fine – as always, the strain is clearly showing. It is only human and understandable, but I still think it undermines your overall, long-term credibility. You are moving in the direction of your foes.
The final little poem, excuse my openness, comes over as just childish.
This is meant more as a personal note. Be free not to let this through screening. Your choice, which I’ll be happy with either way.
Again, thanks for your great work,
Marcus
Steve: you’re right about the little poem.
Seconded completely, with enormous respect and awe for Steve nonetheless.
And great thanks to you Steve, you are my main reason for hope that the truth will come to light!
Please feel supported by all of us in your determination to keep the moral high ground and kick Team *ss with facts.
All the best,
Wijnand
I guess you know you’re getting on Steves nerves when he resorts to poetry.
Personally, I find Steve’s humour essential in what is often a “dry” subject. It certainly makes me grin;)
Aye, but in fairness you might feel very differently if your were its target.
Re: Marcus Kesseler (Jul 31 03:51),
Actually, while the poem is fun (if childish) as is, it really doesn’t work quite well in the last line. The original line in the nursery rhyme is “and stick it in the oven for baby and me.” (at least that’s how I always heard it.) So why not something like “stuck in the report for (or “of”) the IPPC”? In any case I think it should be “the IPPC” rather than just “IPCC”
Dave, I respectfully disagree with your labelling of the poem as “childish” .. as one who is as graphically and iconically challenged as she is statistically challenged (notwithstanding Steve’s best efforts – and even after reading Montford’s eminently readable opus), I often find myself lost in translation, so to speak. To me, the poem reinforced my (very limited) understanding of Steve’s arguments.
But I do agree that the last line, “In the report for IPCC.” would scan better (and more closely approximate the original on which it was based) if it were to read:
“In a report for the IPCC.” 🙂
P.S. Were I to have written the parody, I might also have made a minor change to the first line so that it reads:
“Make a stick, make a stick, Michael E. Mann”
Can the full sample be tested for a measure of signal to noise? Eyeballing the charts looks to me like that the signal is probably very weak.
Steve: the Brown and Sundberg Inconsistency test is way of testing calibrations. The Mann network fails miserably – a point that Ross and I made in our PNAS comment on M08. Mann ignored the point in his response.
Considering the history of the intended targets, I believe Steve’s poem is exactly perfect in tone and efficacy. I picture an amused adult tweaking the nose of a recalcitrant 12 year old.
The problem with the Mannian trick is the use of the algorerithm!
Sorry–I couldn’t pass that up.
Steve
Now look here, you are obviously laughing at a person with disabilities so entrenched they can’t be helped, and ‘coram publico‘ at that. He just can’t help it. He absolutely loves hockey sticks. Some say it is the cause of all his difficulties! He see them here, he sees them there, he can even see them in well stirred mud, poor chap! Why, you can bodge them up out of almost anything if you can just hold the shape in your mind’s eye.
While others around with less challenging deficits find the poor fellow’s contortions quite amusing, many will join you and start giggling, first to themselves, then gradually louder and louder in gales of derision.The slippery-logic fellow will be forced to slither away and find a different source of grant in aid to maintain his disability, or find e new way of exhibiting it, maybe even a new audience.
Another brilliant calm amusing dissection – thank you.
I rather like your poem!
Nice characterisation – though I don’t think it extorts fellow feeling. What did Horace say – I forget my latin – but ‘In laughter I…’ etc etc.
dendro-phrenology?
more like dendro-alchemy.
I was just over at RC, and I think someone brought up a valid criticism of Steve’s graph presented here. Correct me if I’m wrong, this graph uses only proxies that stretch back to 1000AD with no interruption. I see the logic in using only those proxies. But how can you compare it to Mann’s graph, that has more proxy data, proxies that are shorter? Isn’t this comparing data from two unequal sets?
Here’s the comment, snark and all.
Sonicfrog:
It is Gavin’s response to #525 at RC that fleshes out the issue.
If you are making estimates for the MWP, then the only relevant proxies are the proxies for the MWP. It’s a bit rich that Gavin on the one hand complains that I don’t analyse sensitivities to various assumptions (an untrue claim since I do this frequently and we did so in our MBH articles) and then criticizes an analysis using the AD1000 network as not being a “proper” reconstruction. There is no law of nature saying that the effect of splicing should not be analysed. Gavin is completely out of line to object to me analysing the AD100 network.
I haven’t commented on Mann et al (2009 Science) to date. It was published only in October 2009 – in the midst of Yamal discussions and just before CLimategate. I’m surprised that anyone would use this as a scientific reference, but Gavin has done so. Deep in the SI (Figure S8) is a figure which doesn’t mention Tiljander specifically, but responds to the CA criticisms of not doing a no-Tilj no-dendro reconstruction. Far from Tilj having no impact on the reconstruction – even granting Mann his methodology, the removal of the contaminated Tiljander “proxies” means that the EIV no-dendro reconstruction is not “valid” prior to AD1500 (whether it is “valid” after 1500 is another story.) Disproving the disinformation that Tilj have no impact on the no-dendro.
Most remarkably, Mann et al continued to illustrate reconstructions with the known-to-be-contaminated (and upside-down) Tiljander “proxies” in the 2009 Science article – which was not submitted until June 2009 – long after the CA posts on this topic and long after the issue was noted in Ross and my PNAS comment. None of us, even AMac, have commented on this latest entry into bizarro reconstruction-world.
Steve, your response is a good one.
In his inline response to comment 525 , Gavin says
It may be true that Mann does his reconstructions in this somewhat clumsy step-wise manner, but it does not make it correct nor the only way to do it. In fact, if including the later-starting proxies creates a radically different result for any sizable portion of the reconstruction, then this would call into serious question whether the entire set of proxies are consistent enough with each other to be proxying the same effects. Removing the late-strarters may increase the error bars, but should not cause the substantial differences observed in the graph that you constructed.
I coulda been a contender. I downloaded the code July 27, but didn’t start looking at it til today. The code appears to be blind to the sign of the predictor, unlike Mann 08.
All four Tiljanders passed validation.
It would be nice to see a reconstruction using all the shorter data sets as well – or rather, two more reconstructions, one with all the non-thermometer-compromised proxies and one with the lot. And where, in all this, is Tiljander?
I see that gavin refers to the 2009 Mann dendro-non-dendro recon here – it would be helpful to have this contextual URL right at the top of Steve’s thread here. And I’d like some light shone on Mann’s graph – (1) what is “minus 7” (2) does it actually show a non-Tiljander recon at all? it appears doubtful… and if Tiljander is there, maybe it wouldn’t need Luterbacher at all (3) why is the turquoise line buried so much? (does it have items they don’t want to draw attention to?) And finally, at what URL can one get the data anyway eg to check the presence of Tiljander?
I hope I’m not asking stupid questions.
Steve, thanks for the reply. There is so much going on in that thread (even excluding the trashing of Judith Curry) that I missed some of the context.
Supernatural Michael Mann reads everything these days, even Canadian newspapers…
http://telegraphjournal.canadaeast.com/opinion/article/1157413
Or at least someone reads it for him and the whitewashing references come handy.
There’s more to it than meets the eye. It appears to be a part of a PR campaign against people who send letters to the editor of newspapers.
This is the third incarnation of the letter that I have seen, although it can possibly be explained here by the fact that the Telegraph Journal is a sister-publication of the Daily Gleaner and they may have been trying to get mileage from having a “personal letter” from such a personality.
Thank you, the fellow who noticed these also was surprised by “Mann’s” supernatural reaction… 😉
Thank you, Gavin Schmidt.
I consider this tantamount to an admission of what I wrote a few days ago, namely that Mann apparently cannot “get” a HS if he removes BCPs and puts uncontaminated Tiljander data in right-side-up.
RTF
Sorry, the above was me. I have used that name at times. I put it in unintentionally.
RTF
Re: Steve McIntyre (Jul 31 11:56),
Gavin’s in-line response to Judith Curry at Comment #525 includes,
So the only part of Mann08’s Fig. S8a that speaks to whether the use of the Tiljander proxies “matters” runs from 1500 to 1849. With CPS, reconstructions prior to 1500 are unvalidated–with or without Tiljander. Readers should recall that this figure is the only evidence that Gavin and his fellow AGW Consensus advocates have offered to support their “Tiljander doesn’t matter” thesis. (Note that some work by Jeff Id might also favor that idea.)
With EIV, Gavin says that Mann09 shows that no-dendro validates to 1000, but no-dendro/no-Tilj fails to validate before 1500.
Restated, these seem to be Gavin’s two arguments:
“For Mann08, only a 350-year stretch of the twice-corrected November 2009 version of Suppl. Fig. 8a is validated anyway; most of the figure doesn’t speak to the issue one way or another.”
“For Mann09, adding in the Tiljander proxies strenghtens the reconstruction so that it passes validation tests for 850 years instead of 350 years.”
These are offered to support the idea that use of the Tiljander proxies in these reconstructions “doesn’t matter.”
Now it is getting even more peculiar: (You may want to reformat for clarity)
This is a very odd statement from Gavin. If the early temperature record is not relevant or is “not a very interesting scientific question” why go beyond the instrumental record? Why a chapter in the TAR? Am I over-intrepreting what Gavin is trying to say?
I don’t think that when looking into his shaving mirror Gavin S. thinks the paleo-climatology oeuvre is particularly impressive or important. I do think he understands that politically the whole of climate science’s reputation is tied to the Mannian enterprise because of how prominently it has figured in IPCC/Gore/etc. Hence the inspired and hyper-active defense, as an immunization against further attacks and loss of scientific credibility and reputation into his modeling area, and other types of collateral political damage.
Re: Bernie (Jul 31 12:54),
So Gavin has now conceded that the no-Tilj no-dendro reconstruction is a farce. But he says it doesn’t matter because
Puh-leeze. This is neither true nor relevant.
It’s not true. There are dozens of posts and hundreds of comments about the medieval-modern differential. In fact, relatively little discussion at CA has been about the 1400-1500 step in MBH98.
It’s not relevant, because they were the ones that claimed that use of contaminated Tiljander didn’t matter. Even granting them Mann 08 methodology (which has all sorts of other issues besides the contaminated Tiljander proxies), Gavin now concedes that the no-dendro reconstruction has no use prior to 1500.
After all Gavin’s huffing and puffing. Pathetic.
Besides the snarks, it is almost as though Gavin has given up on paleoclimatology altogether. Is he simply worn down trying to defend that which is hard to defend?
Maybe I am missing something, but even Mann’s graphs do not show unprecedented warming (or slope of warming) in the reconstructions ex the grafted instrumental record. But I guess the game is still to tack on the instrumental record to as well as possible averaged noise.
Hi Steve,
English is not my mother tongue, but it seems to me that “emulation” was not
what you were even attempting to do? Care to comment?
The “Little Warm Age” must have included the period known in Scots history as “King William’s Ill Years”. How terribly farsighted of them to hold that warmth was a bad thing.
I’ve been doing some research into the origins of the reconstruction method that’s used by Mann and I came across this. It gives context into the proxy selection criteria and why some proxies are added while others are removed:
I have uncovered the provenance of that little ditty, the girls used to sing it jumpin’ rope:
Paddycake, Paddycake, baker’s Mann,
Bake me a cake as fast as you can;
Roll it, pat it, mark it with a B,
Put it in the oven for baby and me.
Is it possible for Steve to give a summary of all the various papers (Mann, WahlAmman etc) showing which ones include the various unsuitable proxies? A simple table would be very helpful for reference and future discussion both here and elsewhere.
TS,
There is a summary table in Montford’s Hockey Stick Illusion (page 303) which references the Wegman Report (http://www.uoguelph.ca/~rmckitri/research/WegmanReport.pdf, probably the table on page 45)
Although these don’t include the the latest papers.
TS, I followed your conversation at RC with Gavin with great interest. Thank you for that, I hope you’ll find the time to continue it where you left off =)
TS:
Let me second the comments of others. Your patient, persistent and perceptive discussion pushed Gavin into new levels of clarity – that may now be having additional effects on him.
NASA blogger Gavin Schmidt, defender of paleo-phrenology, in reponse to a post in which I carried out careful analysis of the AD1000 no-Tilj no-dendro network accused me of not “performing due diligence” and of “simply making stuff up”:
Uh, Gavin and The Team that Can’t Shoot Straight, the idea of testing a “frozen” AD network is discussed in Mann et al 2008:
Maybe Gavin can explain why the frozen AD1000 network was “proper” in Mann et al 2008.
Only Mann is authorized to perform such a delicate operation. In any other hands it would be “improper”.
The whole idea of this splicing just leaves me cold. Different sets of proxies in each time slice? Please… I just don’t have the stomach for such “creative” techniques I guess.
Craig:
Can you say more about what you see as the problems with splicing? I can see the possible value and legitimacy of linking series from the same source from the same geographical location, e.g., fossil trees, dead trees and live trees of the same species in the same physical environment where there is sufficent temporal overlap, but you seem to be suggesting something else? Is it the inherent increase in uncertainty/error that must accompany such a procedure or is there something more fundamentally wrong?
Steve: in a simple case, one could have a network of white noise before AD1850 and then use instrumental temperatures. The spliced series will verify like a champ but the reconstruction is worthless.
I thought each proxy was tested on its own? Your comment above makes it sound like what they are doing is actually grafting instrumental onto bad proxies and then testing that. Stepwise reconstruction would be different than splicing within a proxy.
Perhaps if you wrote out your ideas clearly and set up actual falsifiable hypotheses in the headpost and clearly made your points, this discusion would go better. You could eschew the poems and the “look at proxies and tell what their noise structure is” comments and the like.
If the key idea is “never do stepwise reconstruction”, then assert that and back it up. Find some literature on the concept (even outside of climate). REally the concpets here have to do with sampling theory relative to hypothesis tests (as I’m pretty certain that just drawing the best line for the period would involved using all the data with steps). So fine, make a mathematical and logical examination of which data should be looked at for a judgement about a test of WMP versus modern. I honestly don’t know the answer…but I know you are not even arguing your case!
Scientist,
Your writing (and probably thinking) is unclear. Are you of the opinion it is possibly acceptable to splice or graft instrumental data onto a good proxy? Are you thinking Mannian tree-ring proxies are good proxies?
If it is your opinion that data splicing is a possibly acceptable practice, please find a statistical textbook that would allow splicing or grafting data taken from instruments measuring two different things. Thermometers measure temperature. Tree-rings measure tree growth which can have a variety of growth limiting factors at tree line – temperature, moisture, available light and fertilization. What textbook explains the proper procedure to splice data from such divergent tools?
If you are of the opinion Mannian tree-ring proxies are good proxies, please demonstrate how we can know tree-rings present a constantly reliable measure of temperature over centuries measured in tenths of a degree globally. How can we be convinced of the accuracy of these proxies in ancient days when the Divergence Problem proves tree-rings are not good proxies during the second half of the 20th century?
Re: Scientist (Jul 31 23:38),
Since you bring it up, it fair to ask for your opinion about “testing proxies on their own”?
Suppose a reeseach group proposes a proxy-based method for building paleotemperature reconstructions that requires calibration of those proxies to the instrumental temperature record.
(Mann08 is a paper with two such methods, one CPS-based and the other EIV-based.)
Suppose certain proxies extend back 3000 years, but they cannot be calibrated to the instrumental temperature record: say, because any 19th and 20th century climate-related signal is overwhelmed by other things.
(The four Tiljander lakebed varve series are such proxies.)
Suppose the research group employs these proxies anyway, and publishes the resulting reconstructions in the 5th-highest-impact peer-reviewed scientific journal.
(Mann08 is such an article.)
Would you say that it is not knowable whether such uncalibratable proxies should have been used?
(This is Gavin Schmidt’s position.)
Would you say that it doesn’t matter whether such uncalibratable proxies are used, or not?
(This is Gavin Schmidt’s position.)
If calibration and use of the uncalibratable proxies caused changes in the pattern of one reconstruction, would their use still not matter?
(This is Gavin Schmidt’s position.)
If calibration and use of the uncalibratable proxies caused another reconstruction to pass validation from 1000-on, while validation is passed only from 1500-on if the uncalibratable proxies aren’t used, would their use still not matter?
(This is Gavin Schmidt’s position.)
You note that Steve McIntyre could write more concisely and clearly about issues with Mann08.
Gavin Schmidt has written a rather different narrative about Mann08.
Very few of the critics of McIntyre’s work have a cross word for Gavin’s methods and conclusions. And vice versa.
So I’m curious: as a critic of McIntyre’s opining on Mann08, what do you think about Schmidt’s opining on the same subject?
You are asking off topic questions. Tjilander was not a part of the headpost. Let’s define the terms under discussion and sharpen the analysis of the issues from the headpost. Rushing in to challenge me with some other possible sin of the enemy so as to bring them down, or show my fairness, is not helpful for actually digging in and defining and deciding on this issue of Luterbacher usage and stepped reconstructions. You should be able to figure out things independently and fairly.
Regardless of Gavin being right/wrong on Tjilander, regardless of me admitting it or discussing it or researching it (and I’m not up to speed on it), you should be able to engage on the topic of methodology under consideration NOW. And Gavin being right or wrong on B, won’t make Steve right or wrong on A. And we’re discussing A, now.
If we could cut all the crap with people dragging in non-related methodology issues as some sort of pugnacious, quivering lip challenge (e.g. Bender), the blog would be half the length…AND we would understand more things, more clearly. All of us. Including Steve. Including the participants.
@Scientist:
“Tiljander was not a part of the headpost.”
This is an odd claim to make. Tiljander is alluded to in the first three lattice graph titles showing the constituent accepted / rejected series, in the final emulation graph, and a few times in passing in the text.
It also rates consideration by Gavin S. in his RC responses.
So by at least two measures – “that word isn’t written on the card in front of me” or “it isn’t relevant as far as the theme of the post is concerned” – I don’t find your assertion convincing.
None of which means you’re obliged to respond of course.
And at least part of the methodology “under consideration” is the manner in which the graph linked to from here: http://www.meteo.psu.edu/~mann/supplements/MultiproxyMeans07/ is constructed, in the absence of a sufficiently clear and unambiguous exposition from the source.
Re: Scientist (Aug 1 01:47),
No. I am asking questions that (a) are as on-topic to the post as yours, (b) are relevant to some of the points you have raised, and (c) you don’t like.
You can see the relevance of Tiljander for yourself by reviewing your prior contributions to this thread, July 30 at 8:00 PM, 10:03 PM, July 31 at 12:13 AM, 11:38 PM. From the latter:
Discussion of the use of the Tiljander proxies in Mann08 is entirely on-topic for this question. You may read Richard Feynman’s Caltech address “Cargo Cult Science” for a primer on the meaning of falsifible — a word you were happy to use at 11:38 PM — in the context of hypothesis tests.
I clarified your writing.
Your “A” is “make a mathematical and logical examination of which data should be looked at for a judgement about a test of WMP [sic] versus modern.”
“A'” is “the decision of Mann08’s authors to include the Tiljander proxies in their reconstructions of the MWP.”
The marked preference of AGW Consensus paleoclimate scientists for complicated, novel, esoteric, and poorly-explained mathematical argumentation over logic and common sense: this is one of the signal characteristics of this field.
If experience is a guide, you will prefer to move on to “C” or “D” rather than explore the implications of “A'” for “A”.
Carry on.
It could be just your snarky writing style, but this paragraph suggests that you don’t understand the Wahl and Ammann 2007 objection to using r^2 as a validation statistic.
Note that for any constants a and b:
cor(x, y) = cor(ax+b, y).
If we have two series x and y that haven’t been calibrated against each other (i.e we don’t know a & b) then measuring correlation makes sense, since correlation is independent of a and b.
Once we’ve calibrated the series though, we want to know the combined effect of x, a, and b. At this point correlation is useless because it’s unaffected by the algorithm’s choice of a and b.
So post calibration it makes more sense to use something like mean squared error (or RE, which is essentially the same thing except for being slightly easier to interpret).
Steve: Puh-leeze, I dare say that I understand the math of this stuff a lot better than Wahl and Ammann. I studied pure math in a hard program; Wahl studied divinity.
The issue is whether or not the relationship between world temperature and strip bark bulges is spurious or not. The idea of RE as a unique arbiter of statistical significance is a Team novelty that was advocated only after the MBH failure of other common verification statistics (including the verification r2 statistic illustrated in MBH98 Figure 3 was exposed). Against this Team novelty is the fact that high RE statistics are readily achieved by totally spurious correlations e.g. the classic Yule examples.
The failed verification r2 should have been reported in MBH and defended as best they could at the time. Instead MBH showed a case where there was a verification r2 pass (their Figure 3) but didn’t report the failures. The WA attack on verification r2 came only after the fact. ONly after the failed verification r2 had been exposed – an exposure that wasn’t easy. Wahl and Ammann issued a press release saying that our results were “unfounded”; they admitted that our results on this point were right only after an academic misconduct complaint against Ammann.
Wahl and Ammann provided no third party statistical references for their argument. In the context of a contentious empirical question, it smacks of special pleading and, since it was after the fact, Texas sharpshooting. An impression in which Wegman concurred – Wegman said that the Wahl and AMmann 2007 had “no statistical integrity”.
If you understand the mathematics behind the WA07 objection to r^2, then you must understand why that objection wouldn’t apply to the screening step.
In that case why write the quoted paragraph?
Not that credentials matter when you’ve clearly made an error, but don’t you think this statement is somewhat misleading?
Steve: I agree with you that credentials don’t matter when you’ve clearly made an error. In this case, there’s no reason for me to think that I’ve made an “error”. In the case of math, it’s pretty hard to learn it well if you didn’t learn it when you’re young. It’s like most athletic things. My snippy point about Wahl’s background arises only because I get a little tired of the idea that Wahl and Ammann are some sort of statistical authorities. They’re not. I’ve read their work very carefully and I find it very dull and uninteresting – which is the main reason that we haven’t bothered submitting a response. It doesn’t seem to me that they have much a feel for the issues. Wahl and Ammann mostly plagiarizes Mann’s 2004 submission to Nature and early realclimate posts.
I’m not sure why you’re so worried about “statistical authority” when the RE vs r^2 issue is easily resolvable from first principles.
The objections to using r^2 for validation don’t apply to using r for screening. The paragraph I quoted suggests that either:
a) you think that they do apply; or
b) you don’t understand why they apply to validation but not screening.
It’s also possible c) you know that the arguments against r^2 don’t apply in to screening, but chose snark over clear exposition.
Yes, you need statistically significant slope to obtain satisfactory confidence intervals.
Why that makes correlation useless?
UC, it does not necessarily make correlation useless, but there can be a need to evaluate the effectiveness of a reconstructed temperature series in a different manner for some cases. For example, one could visualize a reconstruction wiggle matching the temperatures exactly, but with amplitudes which differ substantially from the actual temperatures. This explains why other methods such as RE and CE exist.
Although there are problems with these latter two methods in regard to choosing levels for deciding whether the “fit” is a good one, in the case of CPS, it is not to difficult to show (from “first principles”) that the RE for the calibration period satisfies the identity: RE = 2*Correlation – 1. For other types of reconstructions, I don’t think any simple relationship exists.
And if the reconstruction wiggle-matches very well in the calibration period (r2=0.688), but looses this capability during verification (r2=0.00003) it is better to hide the r2 🙂
SteveMc,
They may be relatively trivial but, in the interests of full disclosure, at some point can you post links to the R scripts that generate the scene-setting lattice graphs and the final emulation, and links for the data sources?
Steve: I’ll post them up. After the fact, I’m always glad whenever I do so, as the process of tidying is always a good idea and placing scripts online removes them from getting edited.
See http://www.climateaudit.info/scripts/mann.2008/benchmark_manniancps_blog_20100730.txt. SHould be turnkey. DOwnloads a large package of information to use in calculations.
Almost turnkey;
It looks as if the penultimate graph – Figure 7 generated at line 438 or so – causes problems for the plotf() function – it’s expecting to plot 12 series at a time, every time, and only has 8 to work with in that call, leading to a “subscript out of bounds” error.
Steve – worked for me. Easy enough to patch if you need to.
Agreed, just feeding back. I wonder if there’s some environmental difference (Ubuntu 10.04 this end) ? Thanks anyway.
5 Trackbacks
[…] Make a stick, make a stick NASA blogger Gavin Schmidt as part of his ongoing attempt to rehabilitate Mannian paleoclimate reconstructions, […] […]
[…] https://climateaudit.org/2010/07/30/make-a-stick-make-a-stick/ […]
[…] done nothing to correct them but then speaks of data quality control issues in rebuttal. And yet Schmidt defends these sort of things on RC. It’s like the Team never read the McShane Wyner paper, because they clearly said this […]
[…] computer models had been “adjusted” always to yield the desired results (see, for instance, the hockey stick controversy), I gradually found the whole rigmarole a lot more than suspicious. Smacking, in fact, of a […]
[…] to Gavin Schmidt's reference to the CPS reconstruction at Mann's website in a lengthy CA post here, presenting my own estimate of the no-Tilj no-bristle CPS reconstruction using Mann 2008 methods on […]