After a year of stonewalling, Mann has published an update at his “grey” Supplementary Information (not yet reported at PNAS) in which he acknowledges an “error” in his figure S8a as follows:
UPDATE 4 November 2009: Another error was found in the corrected Supplementary figure S8a from December 2008: The previously posted version of the figure had an error due to incorrect application of the procedure described in the paper for updating the network in each century increment. In the newly corrected figure, we have added the result for NH CPS without both tree-rings *and* the 7 potential “problem series.” Each of the various alternative versions where these sub-networks of proxy data have been excluded fall almost entirely within the uncertainties of the full reconstruction for at least the past 1100 years, while larger discrepancies are observed further back for the reconstruction without either tree-ring data or the 7 series in question, owing to the extreme sparseness of the resulting sub-network. The new figure can be downloaded here (PDF)
Continues discussion from here. See technical discussion of emulation of CPS at, for example, http://www.climateaudit.org/?p=4244 http://www.climateaudit.org/?p=4274 http://www.climateaudit.org/?p=4494 http://www.climateaudit.org/?p=4501.
The latest correction deals only partly with the most egregious issue of upside-down Tiljander and even this isn’t dealt with clearly. I’ve tidied my Mannian CPS script a little (in particular, applying some of Ryan O’s documentation style which is well worth paying attention to) and will show the net impact of no-Tiljander on the AD800 network illustrated in Mann 2008. (Pretty much the same network extends to their AD1100 step.) I’ll also illustrate the effect of Mannian ex post screening on the same network – previous illustrations of the cherrypicking impact of ex post picking have been done in a red noise context e.g. Jeff Id, David Stockwell, Lubos, and it’s sort of interesting to see it in the context of Mannian “proxies” which are surely hard to distinguish from red noise.
No Tiljander No Dendro
The modern-medieval differential in the Mann CPS AD800/AD1000 networks (AD800 illustrated below), even with ex post correlation screening (CPS = “Cherry Pick and Scale”), has a swing of up to 0.7 deg C from the Mann base case.
Figure 1. Mann NH CPS AD800 Network. Left: black – base case. cyan- no Tiljander no dendro. Right – Difference of Mann version and no-Tilj no-dendro version.
No Tiljander No Bristle
While Mann and the Team petulantly show variations with “no dendro”, the active ingredient in this particular sensitivity are the Graybill bristlecones, which the NAS Panel said should be “avoided”. In our PNAS Comment, we noted that, although Mann 2008 purported to follow NAS panel recommendations, they flouted the NAS panel recommendations to “avoid” bristlecones. Mann’s response on this was no better than his ignoring of upside-down Tiljander. Mann said that:
They [MM 2009] ignore subsequent findings (4) concerning ‘‘strip bark’ records
Ref (4) here is of course the absurd Wahl and Ammann 2007 (see Bishop Hill’s Caspar and the Jesus Paper), which Wegman famously summarized as having “no statistical integrity”. It was available as a preprint, considered and cited by the NAS Panel and the version finally published in 2007 had no “subsequent findings” that were not considered by the NAS Panel. That NAS Panel chairman North was a reviewer of this article shows the limited due diligence of typical peer review processes. For completeness, the graphics below show the same results as above, but for “no-bristle” instead of “no-dendro”. Again, the modern-medieval differential for the relevant AD800 to AD1000 networks (AD800 shown below) – even with Mannian ex post correlation picking – is about 0.6-0.7 deg C.
Figure 2. Mann NH CPS AD800 Network. Left: black – base case. cyan- no Tiljander no bristle. Right – Difference of Mann version and no-Tilj no-bristle version.
No Ex Post Screening
The perniciousness of ex post screening is simple and well understood in the blogosphere – the same idea has been more or less independently reported by me, David Stockwell, Jeff Id, Lubos Motl and Lucia – with Jeff Id being particularly insistent recently on the issue. Prior discussions have mostly been on red noise networks. The Mann 2008 network is interesting as a sample for this sort of thing simply because the proxies are so horribly inconsistent that, like the MBH network, it is a little shop of horrors that produces all sorts of interesting statistical results that one doesn’t usually get to see in actual scientific literature.
Here is the AD800 network (notilj & nobristle) using Mannian CPS but without ex post screening. This CPS version uses 34 “proxies” as compared to the 18 “proxies” used in Mannian ex post picking. The statistical issue – that is totally ignored by Mann and the Team – is whether their results have any significance relative to cherry picking from red noise series.
Figure 3. MBH Notilj Nobristle Network without ex post screening.
This issue was raised in our short comment. Mann’s “response”:
McIntyre and McKitrick’s claim that the common procedure (6) of screening proxy data (used in some of our reconstructions) generates ‘‘hockey sticks’ is unsupported in peerreviewed literature and reflects an unfamiliarity with the concept of screening regression/validation. As clearly explained in ref. 2, proxies incorporating instrumental information were eliminated for validation and thus did not enter into skill assessment.
The observation in our PNAS Comment obviously didn’t reflect “unfamiliarity with the concept of screening regression/validation”. It is something that we understand thoroughly. The point is simple and easy to demonstrate. We cited David Stockwell’s note on the matter from AIG News – smiling a little – as a reference. I wonder whether Mann even bothered reading it. I’m sure that Jeff Id will be at no loss for words for this particular Mannian prevarication.
And as to Mann’s statement that “proxies incorporating instrumental information were eliminated for validation and thus did not enter into skill assessment”, it can readily be seen that the Luterbacher proxies were included in his “skill assessment” and this retort of Mann’s is also untrue.
The idea that Mannian-style operations should have to out-perform corresponding red noise networks is something that is “supported” in the peerreviewedlitchurchur – this is a theme of MM2005a, MM2005b and our Reply to Huybers, which contained useful additional analysis (a point conceded even by Ammann and Wahl.)
In the first comment below, I’ve attached a script generating the above results. The script http://data.climateaudit.org/scripts/mann.2008/utilities.txt contains an emulation function and retrieves relevant collations.
PS. Oh yes, here’s the new “corrected” figure. How does it tie together to the figure that I showed above? It always takes a long time to figure out what is in any of these muddy graphics. No digital results were provided. Nor was the code for the new results provided. It’s not that hard to provide working code. I’ve done it for my analysis here and this blog isn’t even in the PeerReviewedLitchurchur – shouldn’t PeerReviewedLitchurchur outdo mere blogs for transparency and documentation? I presume that Mann’s diagram isn’t a diagram of the relevant medieval proxies, but splices everything together – including the instrumental Luterbacher “proxies”. For sensitivity analysis, the relevant comparison is the actual medieval network.
Climate Audit 
37 Comments
The following script is located at http://data.climateaudit.org/scripts/mann.2008/notilj_impact.txt.
“Upside Down Mann” could be abbreviated:-
uu@W.
Re: dearieme (#2),
It took me a minute but, I like it!
Hey, this coffee is cold. Bring me some freshly brewed Devi08!
======================================
Apologies for not being able to parse this, but is he saying that “we flipped the sign of the series”, or “we flipped the sign for part of the series”, or something else? Thanks!
Steve: “Apologies for not being able to parse this” – join the club. Surely you should ask the authors 🙂
No prob. I just asked the following question on the cover-up thread at RC (newest article published by Mann).
Weird thing is – my comment didn’t show up (I used to get an “awaiting moderation” note) so I tried posting again, but then got a “duplicate comment” error.
I seem to recall JeanS mentioning in the prior thread that of the 7 possibly problematic proxies identified by Mann et al 2008 in the SI, only the four Lake Korttajarvi varve proxies went extremely far back in time.
For the period 700 – 900, how many proxy series are removed from the “NH Land (CPS) Temperature Anomaly” trace by the “…minus 7” operation?
Re: AMac (#7),
Four (all Korttajärvi series). The three other series do not enter the network before AD1700 step.
Perhaps this is a minor point, or an obvious one that I haven’t grasped.
I don’t understand enough statistics to evaluate the competing arguments made here and at Real Climate, so this question involves taking a giant step backwards–all the way back to the raw Tiljander data and the “Mark I Eyeball.”
Tiljander Proxies in CPS Traces of NH Temperature Anomaly, 810-850
A very prominent feature of revised Fig. S8a is the large decline in Temperature Anomaly that begins around 810. The Anomaly then heads back up at about 850.
This is the only point where the Anomaly stays below -0.8 C for an extended time (there are also two brief downward spikes at ~1640 and at ~1700).
Raw Data
Data from the Lake Korttajarvi varves is in figures in Tiljander et al 2003, and (for 3 of the series) at BitBucket as .csv or .xls files (primary source NOAA).
The definitions of the four proxies —
Comparing raw Tiljander data to revised Fig. S8a, 600-1000
Tiljander Fig. 5 for thicknessmm, 20-year averages of NOAA series for the other three —
Interpretation
With respect to the orientations apparently used by Mann, for the 810-850 period, the Lake Korttajarvi proxies seem to offer this information —
Discussion
It seems that there is little support for a major 40-year cool period from 810 to 850 in the raw data for the Lake Korttajarvi proxies.
This sharp cooling period is shown by the “NH CPS minus 7” trace (black line). From this, the “Original NH CPS” trace (green line) is constructed by adding the four Lake Korttajarvi series, and possibly three others. The contribution from the varves should be:
In general terms, the addition of this signal to the Green Line should cause it to run higher than the Black Line, 810-850.
Instead, the Green Trace and the Black Trace are virtually superimposable.
This result seems counterintuitive.
The “minus 7” Black Line does diverge from the Original Green line at other times, suggesting that the Lake Korttajarvi proxies carry sufficient weight in the CPS algorithm to affect results. Note the divergence between Black and Green lines in the 1330-1360, about the time of heaviest varve deposition, prior to the 19th Century.
Again, I offer these remarks cautiously, as they naively involve no transformation of the data, no consideration of “white noise” or “red noise,” and no statistical treatment at all.
Re: AMac (11/9/09 at 2:16 pm currently #9),
My comment upthread has the (dubious) honor of being one of the only nonstatistical considerations of paleoclimate reconstruction. To review, I was puzzled that exclustion of the Tiljander proxies from the CPS reconstruction leaves the Temperature Anomaly unchanged (i.e. Green Trace = Black Trace) during the cold spell of 810-850. This is despite the absence of an obvious Cold signal in the Tiljander proxies during those decades.
There may be a trivial explanation. If the RegEM algorithm calculated that the regional grid covering Finland did not have markedly lower temperatures 810-850 (notwithstanding the general NH cooling), then the presence or absence of the Tiljander proxies from the data set would be expected to make little difference for those decades.
Is the gridded temperature data archived? Is it possible to specify a grid location (e.g. Finland), and download a the calculated Temperature Anomaly over time?
If this is the type of question that has been repeatedly addressed, then apologies for the use of bandwidth. In that case, thread readability might be improved by moving the referred-to comment to Unthreaded.
Re: AMac (#26),
Take a look at the commentary to the script CA/scripts/mann.2008/utilities.txt which describes various Mannian objects used in CPS including instrumental.
Re: Steve McIntyre (#28),
How exactly do we get to that script? I tried just copying “CA/scripts/mann.2008/utilities.txt” to the address bar, but it got Google to search for it and lead to the home page of a company called “CA” which produces IT management software.
John: I’ve edited the reference in the article to the correct URL
Re: Dave Dardinger (#29),
Try: http://data.climateaudit.org/scripts/mann.2008/utilities.txt
It worked for me.
It is rather interesting that figure S8a has been updated now twice due to errors in code, but no code, which sopposingly is all on-line as gavin told us already 9/3/2008, has been updated due to these errors!
Of course, in non-Mannian reality there is still some code missing. I suppose the first correction was due to some mistake in code calculating some verification stats, and now this second correction is due to a mistace in the actual splicing code. Of course, both scripts are missing, so this is only a pure guess.
During the interval 800-899, the black and cyan curves should match exactly to those in Figure 1 (I think they do).
Now the interesting part: due to the “overfitting avoidance” the black curves should actually match (I think they do) over the whole period 800-1399 (verify cps-valdiation-1.xls/NH full CRU) as AD800 step is carried forward until AD1400 steps in. But I think cyan curves do not match after AD900! This means that in the case of no-Tiljander-no dendro, adding extra AD900 proxies “does not overfit” but in the original case we are just adding “redundant proxies” all the way up to AD1400. In normal English: besides ex post screening Mann has (pretty much unreported) ex post validation, which should be very similar in effect to ex post screening.
[Addition: looking again those figures, it may be that cyan curves are not exactly matching over period 800-899. In that case cyan curve in Mann’s figure during that interval may actually be from a previous step.]
I wonder, why he didn’t update the global plot and the southern hemisphere plots. I think we will see the reason soon.
Maybe Rattus can tell us if the RealClimate hockey stick graphs on the Hey YA!(mal) thread have been fixed yet. Hotel RC. Room service.
Steve: The R Code produced an error very quickly
Re: John A (#13),
You are obviously not an academic… 🙂
length(bristle.id) !!!!
Re: RomanM (#14),
Roman
I’m not going to debug the code, I’m going to run it “turnkey” and see what comes out.
Steve: I fixed the line. This was an item inserted so that readers could see that they were picking up the same thing as me and was not a bug that interfered with the code.
Re: John A (#13),
By the way, you might wish to erase the line:
rm(details)
before you run the rest… 😉 (Otherwise, you will have to run the part before that again.)
Steve: No. This is right. The first details is for North American tree rings and the index of bristle ids is all that’s needed. “details” is also used for info on the Mann proxy network and is extracted in the wrapper utilities.txt program.
Ya know I’m holding my tongue.
For sure it does.
I’m curious, can anyone see the cyan line behind that glaring lurid red for the temperature record? If so, does it follow that huge line, or does it taper off at a lower value suggesting that the temperature record curve should be lowered on the scale in order to match the “historic record” from the chronology?
Re: Keith W. (#18), Re: Craig Loehle (#19),
It is quite amazing that he is still drawing the ‘instrumental’ line in a thick red line on top, in an attempt to hide what is (or is not) happening underneath. Keith, if you click on Mann’s pdf version at the top link (not Steves graph), you can zoom in a lot and see that when you compare apples with apples, the cyan line only goes up to about 0.2, so that present temperatures are about the same as the MWP.
I think the dataset used for the red line is CRUTEM (ie land only) Northern Hemisphere.
I just love how the red line is so thick it hides all recent behavior of the proxies. Also, his red line shows 1.4 deg C warming over the duration of redness–kind of high, eh?
Re: Craig Loehle (#19),
The psychology of red.
Steve\
Which package do you have to load before…
..works?
Steve: signal . It’s just the butterworth function that’s used. The wrapped data set includes butterworth-smoothed versions of all proxies. I think that it’s only needed for MAnnian opportunistic sign inversion, – I’ll check some time.
It’s interesting to see the non-sorted version of CPS. If there is a true temperature signal in this data and temperature is truly unprecedented, the average of the whole shouldn’t be that different. The proxies look kinda terrible as unprecedentomometers.
Why didn’t Mann list which 7 ‘potential problem series’ were. hehe. Lessee if we can guess some of the letters in the sight name – I get T. The irony is that his full method throws out 725 other potentially problematic proxies in standard operation.
>Maybe Rattus can tell us if the RealClimate hockey stick graphs on the Hey YA!(mal) thread have been fixed yet. Hotel RC. Room service.
Where’s Richard Sycamore. He told us he’s never been censored by RC, and that he would make me eat my cynicism.
OK. The result of my problem with the line:
was caused by the fact that the signal library is available for version 2.7.x of R but not for version 2.9.x (which is what I downloaded)
To get around this, you need to download http://probability.ca/cran/bin/windows/contrib/2.7/signal_0.5.zip to your PC and then install the package from there into R via Packages > Install package(s) from local zip files… from the menu.
Then I get this as a result:
I’d like to know what instrumental record is claiming a 0.8 C warming. And whats with the cooling from 800-1000 AD?
Re: Mike Lorrey (#25),
“I’d like to know what instrumental record is claiming a 0.8 C warming.”
GISS claims an anomaly of about -0.3 from 1880 to 1920, and a little over 0.5 recently, so that is 0.8 C.
Steve: GISS has nothing to do with this. please do not distract this thread into instrumental issues. There are dozens of other relevant threads.
Sorry, posted the same time John corrected it. John, you can delete mine.
Interesting choice of words.
Rather than say Corrigendum 1, 2 etc, so you can track how many, we just get a date and “Another”.
Not a good look in any event, “Another!”
Amac – 26 – Are the proxy weights relevant to your query?
Re: pete m (#33),
> Amac – 26 – Are the proxy weights relevant to your query?
My understanding of these issues isn’t adequate to answer.
The concept is that the 810-850 valley is a prominent feature of the anomaly trace. According to Jean S (#8 supra), the addition of the four Lake Korttajarvi series is the only pre-1700 difference between the Fig. S8a “NH CPS minus 7” trace (Black) and the “Original CPS NH” trace (Green). Given that there is no obvious “cooler” or “warmer” signal in those series (with respect to, say, 750-800 and 850-900), the coincidence of Black and Green is notable.
That superpositioning is likely the consequence of the absence of a 810-850 dip for temperatures calculated by RegEM for the grid box that covers Finland.
So my question becomes, is there a way to identify that grid box and the RegEM-calculated temperature anomalies for that box? If so, that series could be graphed against the Lake Korttajarvi proxies, 750-900.
I have been looking again at the UK House of Lords Economic Affairs Committee 2005 report on Climate Change. As readers may know this is very balanced and thorough – and therefore largely ignored.
They looked at the hockey stick debate and concluded that they were not qualified to give a view, but did comment that “One curious feature of the debate over Professor Mann’s time series is that the critics appear to ignore other studies which secure similar hockey stick pictures”. The report’s footnote to this is to: K. Briffa et al. Low frequency temperature variations from a northern tree ring density network. Journal of Geophysical Research, 106, (D3), 2001, 2929-41. Hilarious.
Re: Chris Savage (#36),
Not sure if you are refering to the report on this page…
http://www.publications.parliament.uk/pa/ld200506/ldselect/ldeconaf/ldeconaf.htm
On June 21, 2011, I wrote a comment at Collide-a-scape on the use of a paleotemperature reconstruction in the new Kemp et al. paper on sea level variations. I cross-post most of it here, as it seems relevant to the (continuing) Tiljander saga.
– – – – – – – – –
Yesterday, Kemp et al. 2011 was published in PNAS, relating sea-level variation to climate over the past 1,600 years (UPenn press release). Among the authors is Prof. Mann. (Kemp11 is downloadable from WUWT.) Figs. 2A and 4A are “Composite EIV global land plus ocean global temperature reconstruction, smoothed with a 30-year LOESS low-pass filter”. This is one of the multiproxy reconstructions in Mann et al. (2008, PNAS). The unsmoothed tracing appears as the black line labelled “Composite (with uncertainties)” in panel F of Fig. S6 of the “Supporting Information” supplement to Mann08 (dowonloadable from pnas.org).
This is one of the Mann08 reconstructions that made use of the four (actually three) uncalibratable Tiljander data series.
As scientist/blogger Gavin Schmidt has indicated, the early years of the EIV Global reconstruction rely heavily on Tiljander to pass its “validation” test: “…it’s worth pointing out that validation for the no-dendro/no-Tilj is quite sensitive to the required significance, for EIV NH Land+Ocean it goes back to 1500 for 95%, but 1300 for 94% and 1100 AD for 90%” (link). Also see RealClimate here (Gavin’s responses to comments 525, 529, and 531).
The dependence of the first two-thirds of the EIV recon on the inclusion of Tiljander’s data series isn’t mentioned in the text of Kemp11. Nor is it discussed in the SI, although it is an obvious and trivial explanation for the pre-1100 divergence noted in the SI’s Figures S3, S4, and S5.
Peer review appears to have been missing in action on this glaring shortcoming in Kemp11’s methodology.
More than anything, I am surprised by this zombie-like re-appearance of the Tiljander data series — nearly three years after the eruption of the controversy over their misuse as temperature proxies!
2 Trackbacks
[…] For example, the use of the Tiljander proxy data UPSIDE DOWN! Here is some reading material: Another Correction from Upside Down Mann Climate Audit [Welcome to our new home!] Upside-Side Down Mann and the "peerreviewedliterature" Climate Audit [Welcome to our new […]
[…] Another Correction from Upside Down Mann « Climate Audit […]