Bristlecone Addiction in Shi et al 2013

Recently, Robert Way drew attention to Shi et al 2013 (online here), a multiproxy study cited in AR5, but not yet discussed at CA.

The paper by Shi et al (2013) is fairly convincing as to at least the last 1,000 years in the Northern Hemisphere. I am actually surprised that paper has not been discussed here since it aims at dealing with many of the criticisms of paleoclimate research. They use 45 annual proxies which are all greater than 1,000 years in length and all have a “demonstrated” temperature relationship based on the initial authors interpretations.

Robert correctly observed that Shi et al was well within the multiproxy specialization of Climate Audit and warranted coverage here. However, now that I’ve examined it, I can report that it is reliant on the same Graybill bristlecone chronologies that were used in Mann et al 1998-99. While critics of Climate Audit have taken exception to my labeling the dependence of paleoclimatologists on bristlecone chronologies as an “addiction”, until paleoclimatologists cease the repeated use of this problematic data in supposedly “independent” reconstructions, I think that the term remains justified.

While Robert reported that all these series had a “demonstrated” temperature relationship according to the initial authors’ interpretation, this is categorically untrue for Graybill’s bristlecone chronologies, where the original authors said that the bristlecone growth pulse was not due to temperature and sought an explanation in CO2 fertilization. (The preferred CA view is that the pulse is due to mechanical deformation arising from high incidence of strip barking in the 19th century, but that is a separate story.) As a matter of fact, by and large, the bristlecone chronologies failed even Mann’s pick-two test.

Shi et al also show a nodendro reconstruction. This has a much lesser semi-stick. This mainly uses a subset of Kaufman et al 2009 data. In a forthcoming post, I’ll show that even this weak result is questionable due to their use of contaminated data and upside-down data (not Tiljander, something different.)

Shi et al stated that they “started” with the 79 (Northern Hemisphere) proxies from the Mann et al 2008 proxy network, from which they discarded 60 series as “poor-quality data with coarse resolution”:

We started our proxy data collection using the large data set of climate records compiled by Mann et al. (2008), who collected 79 proxy records spanning 1000 yr to reconstruct NH temperatures. Of these series, 60 were not retained for this study, as they did not meet our very restrictive standards: e. g. some poor-quality data with a coarse resolution.

One very obvious quality control standard would be to implement the 2006 NAS panel recommendation that stripbark (bristlecone) chronologies be “avoided” in temperature reconstructions – a measure that surely ought to have recommended itself to the others, if only because so many prior studies have depended on Graybill bristlecones.

However, 10 of the 19 series carried forward from Mann et al 2008 were Graybill stripbark bristlecone chronologies (highlighted in yellow below):

shi-2013_proxies
Figure 1. Excerpt from Shi et al 2013 Table showing Graybill bristlecone chronologies.

Shi et al 2013 also appear to incorrectly believe that Graybill bristlecone chronologies have a “significant statistical relationship with the local instrumental temperature”:

Consequently, only 19 of these 79 proxy series were in common with both the present and the Mann et al. (2008) reconstructions. We also collected 26 other records, resulting in a total of 45 series used for our temperature reconstruction (Table 2). All references for the 45 proxies come from 21 studies (see Table 2). Here, every series was required to exceed a 90% confidence level with either one of the 2 closest instrumental temperature grid points over the calibration interval to ensure that it had a significant statistical relationship with the local instrumental temperature signal.

However, only a couple of these 19 series pass Mann’s pick-two test. It appears that Shi et al did not themselves verify this local correlation (since their Table 1 shows “ND” under the explained variance column) and relied on a presumed prior calculation in Mann et al 2008 establishing this supposed “significant statistical relationship”. However, Mann et al did not require such a relationship for inclusion in their overall network (which is a grab-bag), but only for inclusion in their CPS reconstruction. Mannian EIV methodology does not use this criterion – or even require that proxies be in the applicable hemisphere. Thus, Shi et al cannot assume that the inclusion of a proxy in the Mann et al 2008 network means that it has a “significant statistical relationship with the local instrumental temperature”: they seem to have misunderstood Mann et al 2008 procedures, something that is easy enough to do even for those for whom English is their first language.

But regardless of the reasoning of Shi et al for inclusion of these proxies in their network, the net result is that 10 of the 45 proxies in their network are Graybill bristlecone chronologies.

Many of their other proxies are familiar to the point of being sterotyped.

They use an older supersticked Yamal chronology (Briffa 2008). Plus Briffa chronologies from Taimyr and Tornetrask, a troika that is used in numerous multiproxy reconstructions and hardly new material.

Their running text is curiously silent (and even contradictory)on their use of Briffa chronologies (though they are listed in their Table 1). For example, they say of Tornetrask:

Briffa et al.’s (1992) and Grudd’s (2008) Fennoscandian treering sampling areas are the same (Torneträsk, Northern Sweden), but the Grudd (2008) record was updated from AD 501 to 2004 using new samples from 35 relatively young trees. This new Torneträsk treering maximum latewood density (MXD) record includes samples from a total of 100 trees and covers the period AD 441-2004. Therefore, we used the most up-to-date data from Grudd (2008).

Despite this, they used the Briffa’s Tornetrask chronology (under the alias “Fennoscandia”). This series was also used in Mann et al 2008.

They also use two Taimyr versions – one from Naurzbaev and one from Briffa. In their running text, they say:

Naurzbaev and Vaganov (2000) and Naurzbaev et al. (2002) both studied tree-rings in eastern Taimyr; the newer data by Naurzbaev et al. (2002) were used here

But, once again, they additionally used the Briffa et al 2008 Taimyr series (“Avam-Taimyr”). The Naurzbaev version that they used was also used in Mann et al 2008, where it was incorrectly attributed to Naurzbaev and Vaganov 2000. The Briffa 2000 Taymir version was also used in Mann et al 2008, though Shi et al can be forgiven for missing this: the “Tornetrask” series used in Mann et al 2008 was actually a composite of Briffa’s Tornetrask, Yamal and Taymir chronologies.

In fact, 27 of 45 (not 19 of 45) series are used in Mann et al 2008: 20 (not 19) ITRDB series, the three Briffa RCS chronologies, the Naurzbaev Taimyr version, the D’Arrigo Mongolia series, the Tan et al 2003 speleothem and Donard Lake varves.

Many of the series were also used in Kaufman et al 2009: all four ice core series; the five lake sediment series; the three Briffa tree ring chronologies and the D’Arrigo Gulf of Alaska tree ring chronology. The other series all appear to have been derived from Ljungqvist collections.

In tomorrow’s post, I’ll look at the lake sediment data, which we’ve been considering from time to time. Of particular interest are two series from Baffin Island. In connection with the recent interest in Miller et al on Baffin Island, I’ve re-examined the full Holocene information on Baffin Island lake sediments: there is convincing evidence that Shi et al used these series upside down (as have the many other users of these series) – an error that we previously observed in the PAGES2K use of Hvitavatn, Iceland sediments.

Shi et al 2013 applied several different methods to their dataset, all resulting in HS-shapes. At the end of the day, any reconstruction is a linear combination of the underlying proxies i.e. after much huffing and puffing, a vector of weights is calculated. One of the huge disadvantages of more “sophisticated” methods is that you can end up with series having negative weights i.e. being used upside down. On the other hand, you do want to downweight nearby series: e.g. multiple bristlecone series. However, this precaution doesn’t appear to have been observed in Shi et al.

But the main problem is the longstanding one: if you take a small subpopulation of hockeystick shaped bristlecones and mix them with a population of “proxies” that are indistinguishable from white noise/red noise and apply typical multiproxy recipes, you will get back a HS-shaped reconstruction. When you remove the bristlecones and the Briffa chronologies, is there still a HS reconstruction? I’ll start by examining the no-dendro chronology tomorrow.


140 Comments

  1. Willis Eschenbach
    Posted Nov 29, 2013 at 12:46 AM | Permalink

    Well, this shows what we all knew—bristlecone junkies will do anything for a fix …

    w.

  2. Robert Way
    Posted Nov 29, 2013 at 1:17 AM | Permalink

    I should clarify that I simply repeated what was listed as criterion in the paper – I did not check to ensure the authors initial interpretations regarding temperature signal were retained (rightly or wrongly). During my read I was more interested in examining the differences between the reconstruction methodologies (EIV, CPS, PCA).

    Steve: Robert, specialists in this field have placed far too much importance on complicated multivariate methods, in part, I suspect, because they tend to be outdoorsy, rather than mathematical, and do not have intuitive understanding of the underlying linear algebra, a topic that I’ve written on from time to time. If there is a consistent “signal” in the data, it emerges with simple weighted-averaging methods. More complicated methods run the risk of heavily weighting some series, flipping others. People who use “complicated” methods also tend to pay less attention to defects in the data e.g. Tingley and Huybers’ ludicrous inclusion of the contaminated portion of the Tiljander data, long after it was known to be contaminated. The larger issue here is more the continued passing-off of Graybill bristlecone data used in Mann et al 1998-99 as well as Mann et al 2008 in a supposedly “independent” study, as much as the (almost certain) incorrectness of their assertions of having carried out significance tests on local temperature.

    • Don Monfort
      Posted Nov 29, 2013 at 11:25 AM | Permalink

      Way:”I should clarify that I simply repeated what was listed as criterion in the paper – I did not check to ensure the authors initial interpretations regarding temperature signal were retained (rightly or wrongly).”

      Don’t feel bad, Robert. The reviewers didn’t check either.

  3. Pat Frank
    Posted Nov 29, 2013 at 1:44 AM | Permalink

    I was more interested in examining the differences between the reconstruction methodologies (EIV, CPS, PCA)” and all of them are variants on correlation = causation. Deep science, that.

    Steve, “they seem to have misunderstood Mann et al 2008 procedures, something that is easy enough to do even for those for whom English is their first language.” — too funny! :-) I’d forgotten how much I enjoyed Steve’s dissection of proxy temperature reconstructions. Glad you’re back at ‘em, Steve.

    • Posted Nov 29, 2013 at 7:19 AM | Permalink

      I’d forgotten how much I enjoyed Steve’s dissection of proxy temperature reconstructions.

      Enjoyable and educational. We partly have Robert Way to thank.

      In a forthcoming post, I’ll show that even this weak result is questionable due to their use of contaminated data and upside-down data (not Tiljander, something different.)

      At the end of the day, any reconstruction is a linear combination of the underlying proxies i.e. after much huffing and puffing, a vector of weights is calculated. One of the huge disadvantages of more “sophisticated” methods is that you can end up with series having negative weights i.e. being used upside down. On the other hand, you do want to downweight nearby series: e.g. multiple bristlecone series. However, this precaution doesn’t appear to have been observed in Shi et al.

      Wrong in both directions. How do they manage that so often?

      • Fred
        Posted Nov 29, 2013 at 8:28 AM | Permalink

        “Wrong in both directions. How do they manage that so often?”

        It is a specialized area of paleo science called “Concurrent Suck & Blow”

        • tomdesabla
          Posted Dec 2, 2013 at 2:58 PM | Permalink

          Wind instrument players do have a technique called circular breathing which allows them to replenish their air supply while still putting out a stream of air sufficient to keep a note going.

          This seems to be a positive, cool thing, as opposed to the consistent misuse of “proxies” by allowing them to be used upside down.

  4. John Francis
    Posted Nov 29, 2013 at 2:07 AM | Permalink

    snip – any remarks to Way with even a hint of moralizing will be deleted

    • CG
      Posted Nov 29, 2013 at 8:04 AM | Permalink

      That isn’t really fair to say. Mr. Way is merely pointing out that he never claimed to have audited the paper’s data, rather, he assumed that things were done correctly and found the resulting analysis interesting. If Steve can show (as most here will expect given this post starting us off) that this paper is fatally flawed in data or methods, I’m sure that will change Way’s opinion of the paper and it’s results (or lack thereof).

  5. andy
    Posted Nov 29, 2013 at 3:00 AM | Permalink

    Is the Shi & al paper and/or SI material available, except behind the paywall?

    Steve: online here.

  6. Brian H
    Posted Nov 29, 2013 at 3:59 AM | Permalink

    Baffin Island upside down! That should make a big splash.

    • looch
      Posted Nov 29, 2013 at 9:26 AM | Permalink

      A dog by any other name is still a dog!

    • MikeN
      Posted Nov 30, 2013 at 3:14 AM | Permalink

      I think Congressman Hank Johnson will be interested in this study.

    • Jeff Norman
      Posted Dec 1, 2013 at 11:24 AM | Permalink

      Baffin Island upside down would also play havoc with melting ice and prehistoric moss results.

  7. Posted Nov 29, 2013 at 4:29 AM | Permalink

    In a saner world Robert would have at least tried to discuss Steve’s commentary of Shi et al. In a mad world Robert would have learned by now to keep quiet as most the Great Climate Scientists do, having understood their funny HS business is built upon nothing at all.

    I guess we’re somewhere in the middle here.

    Steve: please – no more moralizing to or about Way.

    • David Smith
      Posted Nov 29, 2013 at 12:37 PM | Permalink

      Hi, Steve. I enjoy, as do thousands of others, reading your analyses of reconstruction studiea. It’s invigorating to readers.
      And I enjoy seeing your snipping of personal comments aimed towards Way or anyone else. It’s good to see people focus on the issues raised and not on the personalities involved. Personal comments which correct the record are good. Negative comments about others, no matter how well deserved, leave a taint on the sender.

  8. Bernd Palmer
    Posted Nov 29, 2013 at 5:33 AM | Permalink

    @Robert Way: “During my read I was more interested in examining the differences between the reconstruction methodologies (EIV, CPS, PCA)” Will you let us know the outcome, please, when you are through.

    • CG
      Posted Nov 29, 2013 at 8:08 AM | Permalink

      Might want to wait until Steve clears away any issues, there are already enough in this post to make the results in question until replication and correction at least of the data (to match the paper’s stated methods) can be done.

  9. oneuniverse
    Posted Nov 29, 2013 at 6:17 AM | Permalink

    re: 2006 NAS panel recommendation on avoiding strip-bark chronologies

    Salzer et al 2009 challenged this recommendation. How have Salzer’s own findings stood up since publication ?
    There was some specific criticism at CA at the time (“Salzer et al 2009 – A First Look”) , but has there been anything further in the PRL ?

    Salzer and co-authors continue to disagree with the NAS panel’s recommendation. From a more recent paper :

    Multiple lines of evidence, including comparisons with instrumental climate data over the last century, show that patterns in the ring widths of upper treeline bristlecone pine are strongly influenced by temperature variability, particularly at decadal to centennial scales (LaMarche 1974; Hughes and Funkhouser 2003; Salzer and Kipfmueller
    2005; Salzer and Hughes 2007; Salzer et al. 2009; Kipfmueller and Salzer 2010; Towlinski-Ward et al. 2010; Bunn et al. 2011).

    “Five millennia of paleotemperature from tree-rings in the Great Basin, USA”, Salzer et al 2013

    • Posted Nov 29, 2013 at 6:40 AM | Permalink

      Salzer et al 2013 is a waste of time if anybody wants to know if bristelcones are good for temp reconstructions, given that _that_ is the main assumption.

  10. Geoff
    Posted Nov 29, 2013 at 8:45 AM | Permalink

    Salzer 2013 is open access at Springer http://link.springer.com/article/10.1007/s00382-013-1911-9

    Steve: the new article discusses treeline in bristlecone areas – a topic discussed in early CA posts.

    • phodges
      Posted Dec 6, 2013 at 2:28 PM | Permalink

      One of Salzer’s claims in just the abstract is disproved by simply visiting any Bristlecone stand…dead trees and debris above the current treeline contradict the claim that the current treeline is higher than in 4000 years.

      Steve: they don’t say this and their SI shows otherwise. they say that the “rate” of increase in treeline is the fastest, not that it’s the highest.

  11. Posted Nov 29, 2013 at 1:26 PM | Permalink

    “Robert Way Posted Nov 29, 2013 at 1:17 AM | …

    …During my read I was more interested in examining the differences between the reconstruction methodologies (EIV, CPS, PCA).

    Any particular insights that you enjoyed or found interesting in those differences? Particularly any methods (or pitfalls) relevant for discussion as Steve delves through Shi et al 2013?

    • Robert Way
      Posted Nov 29, 2013 at 6:59 PM | Permalink

      I’m finding the paper has lead me to do more reading on the EIV approach which in turn has left me somewhat confused. Theoretically an approach like EIV if used on idealized meteorological station data makes some sense, I find the particularities of how one would use a data imputation method like this for proxy reconstruction interesting but needing some better description in the literature.

      Steve: in linear algebra, the weights are the “dual” of the reconstruction. In my implementation of EIV (and MBH), I managed the linear algebra so that the weights were extracted concurrent with the reconstruction. One of many things that would be worth publishing. EIV is also a linear operation and results in weights. So you have to ask whether the weights “make sense”. If you know the orientation of the proxies (and you ought to if you know that they are proxies), then you should restrict weights so that they are all positive. EIV and PCA methods do not guarantee this as more factors are retained: in my opinion, this is a deterioration from simpler methods. IMO, 99% of the multivariate effort in this field is either pointless or bogus. Mostly because practitioners don’t understand the linear algebra in a thorough way and are easily led astray.

      • TAG
        Posted Dec 1, 2013 at 1:34 PM | Permalink

        Mostly because practitioners don’t understand the linear algebra in a thorough way and are easily led astray.

        Wow!

  12. Coldish
    Posted Nov 29, 2013 at 3:20 PM | Permalink

    It’s good to see Dr Way contributing here at CA. Beneficial all round!

    • Kneel
      Posted Nov 29, 2013 at 4:29 PM | Permalink

      “It’s good to see Dr Way contributing here at CA. Beneficial all round!”

      Indeed – hopefully, he will stay around long enough to consider Steve’s criticisms of the paper in question and come to his own conclusions.

      I think it only fair to warn him that should he chose to do so, and should it be the case that he finds the results, err, erroneous shall we say, and further that he mentions this in climate science circles, he will likely end up the same as Judith Curry has – labelled a “denier”, “mis-informer” etc.

      • JEM
        Posted Nov 29, 2013 at 5:24 PM | Permalink

        Indeed, the nonbelievers are bad enough, but it’s the apostates that get hammered.

        Seriously, though, continued use of stripbark chronologies bring to mind Marlborough’s quote ‘they came on in the same old way…’

        • pottereaton
          Posted Nov 29, 2013 at 6:56 PM | Permalink

          Correction: that was Wellington.

        • bernie1815
          Posted Nov 30, 2013 at 10:48 AM | Permalink

          At Waterloo and in reference to Napoleon’s use of closely packed columns of infantry advancing almost as a battering ram against infantry in line. OK against all but British infantry trained to fire at twice the rate of their European counterparts. (See Bonderchuk’s amazing movie, Waterloo (1971) ).

        • bernie1815
          Posted Nov 30, 2013 at 10:53 AM | Permalink

          Substitute “McIntyre” for the British Infantry – largely Scots, anyway – and you complete your analogy.

        • JEM
          Posted Nov 30, 2013 at 10:27 PM | Permalink

          Indeed, a sorry bit of brain fade on my part.

  13. morebrocato
    Posted Nov 29, 2013 at 4:23 PM | Permalink

    Y’all see the study now retracted by the Food and Chemical Toxicology Journal — The Seralini study published in September, 2012? …It purported to show genetically modified food as causing an increase in tumors.

    http://news.yahoo.com/journal-withdraws-controversial-french-monsanto-gm-study-144512338–sector.html

    Some relevant quotes…

    “This retraction comes after a thorough and time-consuming analysis of the published article and the data it reports, along with an investigation into the peer-review behind the article,”

    (an audit, perhaps?)

    “Within weeks of its appearance in the peer-reviewed journal, more than 700 scientists had signed an online petition calling on Seralini to release all the data from his research.” — apparently no FOI request necessary.

    One of the reasons for the retraction upon further examination was because of its small sample size not allowing its conclusions to be acceptably powerful/meaningful.

  14. kim
    Posted Nov 29, 2013 at 5:15 PM | Permalink

    I’m constantly running into math outdoors. Is there shelter anywhere?
    ===========

    • kuhnkat
      Posted Nov 30, 2013 at 8:26 PM | Permalink

      I hear Bristlecone Pines have been sheltering certain people for quite a time…

  15. David Jay
    Posted Nov 29, 2013 at 5:58 PM | Permalink

    Those trees sure do multiply.

    • Ed
      Posted Nov 29, 2013 at 6:26 PM | Permalink

      I believe the trees multiply by adding the logs. Ugh, sorry, bad math joke …

      • kim
        Posted Dec 1, 2013 at 12:38 PM | Permalink

        McIntyre’s Razor has a Stihl edge.
        ==========

  16. Svend Ferdinandsen
    Posted Nov 29, 2013 at 6:08 PM | Permalink

    I will only say, that whenever some matrix operations are involved, you have to be very cautious in the interpretions.
    It really makes me wonder why common averages apparently are altmodich.

  17. scf
    Posted Nov 29, 2013 at 6:23 PM | Permalink

    Looking forward to the non-dendro analysis. Is there any way to rank the strengths of the various proxies that exist?

  18. Craig Loehle
    Posted Nov 29, 2013 at 7:09 PM | Permalink

    I have been nagged about why I haven’t done any further paleo reconstructions. The reason is that when I look at this data now I feel like Indiana Jones looking down into the egyptian tomb full of snakes and saying “I hate snakes”. These paleo records seem just like that tomb–full of treachery and liable to bite you on the a**.

    • DocMartyn
      Posted Dec 2, 2013 at 8:45 PM | Permalink

      Do you know why isotope analysis, to estimate temperature signal, isn’t performed on cores after they have been digitized?
      You wouldn’t have to do the whole core only a few peaks and troughs.

  19. UC
    Posted Nov 29, 2013 at 7:29 PM | Permalink

    Fig. 4:
    “To facilitate comparison, every series was
    firstly expressed as anomalies from AD 1850- 1960,
    and was then variance matched using the instrumental data from AD 1850- 1960″

    Hey, I invented this before: http://www.youtube.com/watch?v=TTVRCXlqoT8 . I just used shorter anomaly/scale periods, shouldn’t matter.

    • James Smyth
      Posted Nov 29, 2013 at 8:09 PM | Permalink

      Can you post up some code, or meta code, for that? Assuming that your box is a (moving) baseline, what are you matching it to? (obviously not instrumental data from the same interval as the box)

      • UC
        Posted Nov 29, 2013 at 8:59 PM | Permalink

        “what are you matching it to? ”
        to mean 0 and std 0.1:

        %recons: load juckesrecons
        %recons: smooth juckesrecons (lowpass(in,1/40,2,2);)
        N=length(recons)
        tL=[1 30];

        for ii=1:N-30

        reconsm=nanmean(recons(tL(1):tL(2),:));
        reconss=nanstd(recons(tL(1):tL(2),:));

        reconsa=(recons-repmat(reconsm,N,1))./repmat(reconss,N,1)*0.1;
        plot(time(:,1),reconsa)

        aa=patch([time(tL(2),1) time(tL(2),1) time(tL(1),1) time(tL(1),1)],[-2 2 2 -2],’c’);
        set(aa,’FaceAlpha’,0.9)

        tL=tL+1

        % fig to file
        end

    • Craig Loehle
      Posted Nov 30, 2013 at 7:44 AM | Permalink

      They love to do this variance matching, but if polar regions are more volatile than tropical (as is often claimed when convenient) then you will be squashing the true global temperature response down. If you have first calibrated individual series to temperature, then you should leave them alone and not variance match imo.

      • Paul_K
        Posted Dec 1, 2013 at 1:40 AM | Permalink

        Amen, Craig. At the least, some latitudinal adjustment of weighting seems to be called for – even if it is only crudely based on a functional model of temperature variance by latitude. Without this, it seems to me that the variance of the final reconstruction must be randomly influenced by the meridonial scatter of the selected proxies – even if they were to be “perfect” as proxies. I am only aware of one study that has taken this problem on board, and that was the Annan and Hargreaves reconstruction of LGM temperatures. Are there others?

      • Pat Frank
        Posted Dec 2, 2013 at 11:03 AM | Permalink

        calibrated individual series to temperature” Let’s be clear: that’s not calibration in any physical sense. “Calibration” in proxy thermometry is mere ad hoc re-scaling.

  20. Gaelan Clark
    Posted Nov 30, 2013 at 6:37 AM | Permalink

    I dont understand…..Way exclaimed in comments on Cowtan and Way thread that the evidence was within Shi et al 2013 AND that this evidence was “convincing” for the last 1000 years.

    Only after Steve Mc audited the paper does Way say he didnt read the whole thing and further that he doesnt understand the papers math…….

    Am I wrong here?

    • Robert Way
      Posted Nov 30, 2013 at 6:35 PM | Permalink

      Yes you are incorrect. I said I thought it was reasonably convincing because I liked the approach of using annual proxies, all covering the time period analyzed (e.g. no drop-off with time), with purportedly a relationship with local temperature* and testing the approach in the way they did (CPS, EIV, PCA). What I didn’t do was take their 45 individual proxies and look up every one of them for their peculiarities. I do not think it would be reasonable to be expected to do so with every paper one reads. I do not understand where you take that as meaning I did not read the whole paper.

      Secondly, if you had read the paper you would note that there is not a clear explanation of the regularized EIV in the paper. To understand the peculiarities of this method you have to piece together from several other papers with the McShane and Wyner paper (and its large volume of responses and comments) being a good source for doing so.

      Perhaps you’re letting your preconceptions about me impact the way you interpret the commentary.

      Steve: Robert, I quite agree that there is no adequate discussion of RegEM/EIV in the literature. I ported the methodology to R and, in doing so, made some huge simplifications by understanding the linear algebra. It is unbelievably inefficient in Mann’s implementation, with the inefficiency demonstrating Mann’s apparent lack of understanding of the algebra. My blogpost here
      http://climateaudit.org/2011/10/16/notes-on-regem/ contains notes on my implementation of the method, notes which lay out the linear algebra in a form not found elsewhere and which (IMHO) are superior to what you can find in the litchurchur. I have some other notes on the regularizing parameter that are quite interesting and which I haven’t put in a blogpost.

      • Posted Nov 30, 2013 at 11:27 PM | Permalink

        …and that’s why reviewers let dendro papers get published despite the papers’ extraordinary weaknesses…

      • Gaelan Clark
        Posted Dec 1, 2013 at 8:43 AM | Permalink

        Thank you for responding and I now understand your position.

        I have never heard of you prior to the recent posts and I have no preconception of you. I am very impressed with your being on site and answering questions.

        • tomdesabla
          Posted Dec 2, 2013 at 3:09 PM | Permalink

          Actually, I have to second your compliment. Robert Way’s engagement here is a real positive. I would have liked an apology to Steve for past comments, but I’d rather have this kind of consistent engagement than an apology and a disappearance. Bravo Robert Way. As they say in AA “keep coming back!”

      • Don Monfort
        Posted Dec 1, 2013 at 11:14 AM | Permalink

        Do you still think that Shi et al is reasonably convincing, Robert?

      • C Johnson
        Posted Dec 1, 2013 at 12:43 PM | Permalink

        Way, “I do not think it would be reasonable to be expected to do so with every paper one reads.”

        Honest question here Mr Way. What value is a lay man to place in the Peer Review Literature if the peers are not critically reviewing the results to check their validity? It is understandable not to do so with every paper published, however, time and again papers and results critical to the status quo are demonstrated to be thoroughly flawed.

        There appears to be an uncritical acceptance of papers because they were published, while the publishers themselves pass the responsibility of validating the results to the community. To say nothing of the difficulty in acquiring the data necessary to do so.

      • Posted Dec 1, 2013 at 2:54 PM | Permalink

        Robert Way: I do not think it would be reasonable to be expected to do so with every paper one reads.

        See, I agree with you. The problem is, Steve decided the would read the entirety of Mann’s paper and understand it. That was his original sin. Along the way, he ran into things that did not make sense.

        As Ross explains:

        In the Spring of 2003, Stephen McIntyre requested the MBH98 data set from Mann. He is not a scientist or an economist, he was just curious how the graph was made and wanted to see if the raw data looked like hockey sticks too. After some delay Mann arranged provision of a file which was represented as the one used for MBH98. One of the first things Stephen discovered was that the PCs used in MBH98 could not be replicated. In the process of looking up all the data sources and re-building Mann’s data set from scratch, Steve discovered a quite a few errors concerning location labels, use of obsolete editions, unexplained truncations of available series, etc. Some of these had small effects on the final results, but replacing the PCs had a big effect. I joined the project in the late summer of 2003 and we published a paper in October 2003 explaining the errors we found in Mann’s data. We showed that when these errors were corrected the famous hockey stick disappeared. (emphasis mine)

        Until I saw Steve’s work, my personal reaction to these papers had been one of intuitive dismissal … After all, I am used to seeing people who easily impress themselves with their ability to be fooled by spurious correlations.

        However, the reaction by “climate scientists” to Steve’s discoveries told me there was something else going on. I am grateful that Steve has been checking the Team’s assertions in the intervening decade. He is a scientist in the true sense of the word.

      • Steve McIntyre
        Posted Dec 1, 2013 at 5:13 PM | Permalink

        RObert wrote: “What I didn’t do was take their 45 individual proxies and look up every one of them for their peculiarities. I do not think it would be reasonable to be expected to do so with every paper one reads.”

        I cannot disagree more.

        As someone who know the multiproxy papers very well, it’s the VERY FIRST thing that I do when I see a new multiproxy paper: examine the proxy network, see what’s old and what’s new. In the case of Shi et al 2013, for example, their use of 10 Graybill stripbark chronologies is very important in any assessment.

        I also stratify the data to see which series are “active ingredients” in any HS-ness of the reconstruction, since too often results are dependent on a few series, despite protestations of authors otherwise. Thus, the contaminated Igaliku series in PAGES2K stood out immediately.

        I know the proxies well enough that I am familiar with peculiarities of many of them, though some I’m still learning.

        There are only a few “new” proxies in Shi et al. In my opinion, these are the most interesting aspects of any new study: do the new proxies shed any light on the field.

        I have become profoundly uninterested in novel multivariate methods applied to datasets with bristlecones and Yamal in them. I am prepared to stipulate that they have a HS shape.

        I am interested in whether proxies are consistent with one another in the same region.

        Nor do I think that it is “unreasonable” for interested readers to know the peculiarities of the proxies. Thousands of hockey fans know the peculiarities of the candidates for Team Canada or of NBA players. I therefore think that paleoclimate scientists should know the peculiarities of dozens, if not hundreds, of proxies.

        • Don Monfort
          Posted Dec 1, 2013 at 6:33 PM | Permalink

          Maybe Robert’s reading of Shi et al was cursory, because he had forgotten about the issues with multiproxy papers that you and others have been exposing for many years. Perhaps Robert assumed that you had not discussed Shi et al, because you couldn’t find anything wrong with it. He is probably sorry he brought it up.

        • Robert Way
          Posted Dec 1, 2013 at 10:06 PM | Permalink

          Yes and that is your niche and you’re welcome to find it interesting. You’re more than welcome to have your opinions on the matter and that is fine but if I read a paper and I see in a table of proxies that it lists a series of tree ring series as: ITRDB – ca606 (and other codes), then I feel it is not reasonable to expect that I have a responsibility to go check each individual series used from the ITRDB before commenting on whether a papers approach is interesting or not.

          Were I auditing or reviewing a particular paper then perhaps that would hold more weight but as a person whose primary research subject is not paleoclimate it is odd to see that you have taken issue with this comment.

          I think that the Shi paper presents many of things that have been asked for by people here and there should be some acknowledgement of this.

          However, if there are issues with the proxy selection then those are legitimate concerns and I hope that they are not the major driver of the results.

          As an example from your experiences – what would be a reconstruction paper that you feel would be an instance of reasonable statistical practice with reasonable proxies?

          Steve: robert, I wasn’t trying to moralize about your individual reading of the paper if you are reading the papers as a non-expert in the field. However, I believe that experts in the field, including anyone who agreed to be a peer reviewer, should be familiar with the peculiarities of individual proxies. Thinking about this some more, I think that the authors of Shi et al should have disclosed clearly that 10 of 45 proxies were Graybill stripbark chronologies, that these chronologies have been used in many of the reconstructions to which the Shi reconstruction was being compared and that these chronologies had been sharply criticized, including a statement by the NAS panel that they should be “avoided” in reconstructions. Can I ask you a question on this: would your attitude to the Shi reconstruction have been affected if they had disclosed this to you? If it would have affected your attitude (and, as someone knowledgeable about reconstructions, it affects mine), then it should be mandatory disclosure. In my opinion, the absence of such disclosure is negligence on the part of reviewers, who should have asked for such disclosure. I agree with your observation that mere readers should not be expected to check the underlying papers. On your question, I’ve long stated that the problem in the field is proxy inconsistency and that this is not a problem that can be solved by fancier multivariate statistics. Only by detailed reconciliation, area by area.

        • Brandon Shollenberger
          Posted Dec 1, 2013 at 10:29 PM | Permalink

          Robert Way, you say you shouldn’t have to do things prior to “commenting on whether a papers approach is interesting or not.” You were quoted in this blog post saying the paper in question is “fairly convincing.” Do you think the same should be true for both?

          I, personally, don’t think I’d be willing to say something is “fairly convincing” if I hadn’t applied any critical analysis to it. I certainly wouldn’t be willing to if the paper dealt with a contentious subject such as the hockey stick. I think that’s true of most people at this site, and I suspect it’s why you’re getting the reactions you’re getting.

        • CG
          Posted Dec 1, 2013 at 11:20 PM | Permalink

          Brandon,

          You can be convinced by something that is wrong, as it is impossible to thoroughly investigate everything. The important thing is being open to modifying your opinion based on subsequent findings or arguments. If this paper, based on its text and no further investigation, appeared to meet and handle all prior types of criticism, then it might be convincing. If someone digs in and finds that the paper’s text does not, after all, handle prior critics at all, then the paper would cease to be convincing at this time.

          As for whether or not Way should have investigated before opining that something was “convincing”, I would find little to no fault in that. It wasn’t as if Way was trumpeting the validity of this paper to the world without having dug into the nitty gritty, but rather, he commented (here on this blog) that it seemed to be valid and interesting. That’s fine, it’s just part of having a discussion. I would think somewhat differently if Way claimed to be a Paleo expert and recommended a paper without checking the data, but, that does not appear to be the case. Even if it was, it wouldn’t be a cardinal sin; it’d just be embarrassing.

        • Brandon Shollenberger
          Posted Dec 1, 2013 at 11:52 PM | Permalink

          CG, taking a paper solely at its word as resolving topics of dispute seems strange to me. I couldn’t do it. Even if I could though, this paper didn’t claim to do that. It didn’t come close to addressing most of the prior types of criticism. As such, I can’t see how one would come to believe it did.

          One of the major criticisms of temperature reconstructions has always been the limited data containing a signal similar to the final result. As we see in this post, not only did the paper not address that issue, it reused much of the data which has been previously discussed. I don’t see how one can read the paper and think it handles such a major criticism, and I think that would be a bare minimum for finding the paper “fairly convincing.”

        • William Larson
          Posted Dec 2, 2013 at 11:53 AM | Permalink

          Re Way and McIntyre, above (and wishing that “Reply” were an option for every comment here): Feynman used to argue that every paper ought to present not only the evidence FOR its findings, but should also include every argument AGAINST its findings. This only makes sense if scientific truth is the object. And so here–seconding SM’s argument–Shi et al. needed, absolutely, to include “caveats” about their use of Graybill strip-bark chronologies and any reviewers needed to insist upon it. It is appalling, this frequent lack of scientific rigor in climatology publishing.

        • Robert Way
          Posted Dec 2, 2013 at 1:01 PM | Permalink

          “Steve: However, I believe that experts in the field, including anyone who agreed to be a peer reviewer, should be familiar with the peculiarities of individual proxies. Thinking about this some more, I think that the authors of Shi et al should have disclosed clearly that 10 of 45 proxies were Graybill stripbark chronologies, that these chronologies have been used in many of the reconstructions to which the Shi reconstruction was being compared and that these chronologies had been sharply criticized, including a statement by the NAS panel that they should be “avoided” in reconstructions… Only by detailed reconciliation, area by area.”

          I don’t disagree with any of the above statements much really. I think that the authors probably should have disclosed that this conflicted type of proxy was used and they should have provided justification for having chosen to use it in a similar vein to how they justified the use of other proxies. I am not an expert on this type of tree ring chronology but if I recall the NAS statement on that proxy contains slightly more caveats than simply do not ever use. If there is recent literature on the topic which is able to counter the NAS suggestion regarding this type of proxy then it should have been placed in that context within the data section of the paper. Lacking that sort of counterargument it is problematic and probably something that should have been required. However using loaded words such as reviewer “negligence” is beyond what I think is acceptable.

          As for area by area – I agree with this approach and I believe it adds value to have actual experts on the given topic creating chronologies and producing regional reconstruction series. The alternative approach that we have seen has included downloading everything from a paleodatabase and running gridcell correlations etc… to build a reconstruction which I think of as being less convincing – even when done by statisticians.

          Steve: I wasn’t trying to be overly emotive in using the term “negligence”. In this context, I merely meant “neglect” by reviewers – would this seem more apt to you? From my perspective, I think that reviewers typically spend too much time gatekeeping and not enough time in ensuring that the record of the article is complete in terms of disclosure and data availability.

        • Steve McIntyre
          Posted Dec 2, 2013 at 3:53 PM | Permalink

          Robnert wrote: “The alternative approach that we have seen has included downloading everything from a paleodatabase and running gridcell correlations etc… to build a reconstruction which I think of as being less convincing – even when done by statisticians.”

          At the NAS panel workshop in 2006, Malcolm Hughes made an interesting comparison between two styles of reconstruction. (1) relying on multivariate methods to extract a “signal” from everything in a databank. Hughes called this the “Fritts method”, though more aptly the Mann method for most readers. Choosing small subsets from large collections is a lowbrow variation. (2) collecting data according to ex ante criteria and using simple averaging methods. Hughes called this the “Schweingruber method”.

          The Schweingruber method is almost never used in current paleoclimate studies. Briffa et al 1998, 2001 were based on the Schweingruber dataset. This is the dataset with the famous decline, which Briffa failed to show in some important graphics, notably, in conjunction with Mann and Osborn, in IPCC TAR.

          Mann’s reconstruction gave him fame and success, while Briffa’s occasioned zero interest and left Briffa near destitute for grant money. In my opinion, the contradiction between the Mann et al 1998 reconstruction and the Briffa et al 1998 reconstruction – both taken from large populations of tree ring sites – ought to have stimulated great interest among specialists to explain the discrepancy. Instead, in part because of Briffa’s decision to hide the decline, the divergence problem remained essentially unexplored for over a decade. Even today, the issues have mostly been papered over (through citation of the execrable article by Cook, normally a very capable scientist), rather than settled.

        • MrPete
          Posted Dec 2, 2013 at 1:06 PM | Permalink

          Re: Steve McIntyre (Dec 1 17:13), and Robert Way (Dec 1 22:06),

          So… I would hope that both readers here as well as Robert and his associates would recognize that while statements have been made here about “agree” and “disagree”… in reality my sense is there’s vigorous agreement among the principals here on this topic:

          * Any expert in the field, any reviewer, anyone with significant experience SHOULD be aware of the well-known datasets and SHOULD be checking such things in new papers, particularly because there is a long history of data reuse and data snooping.

          * Authors and reviewers should (in best Feynman tradition) explicitly declare the well known caveats about data sets.

          * Readers should not have to dig deep to discover those caveats and whether they apply.

          To me, this is quite similar to declaring CI’s, or whether a medical study was done using proper double-blind methods, etc.

      • Craig Loehle
        Posted Dec 1, 2013 at 10:47 PM | Permalink

        When assessing ANY study it is important to not be naïve, to use some critical thought. In toxicology, it is known that certain strains of inbred mice get cancer at multiples of the normal rates. In surveys it is well-known that people lie about their own behavior (they assert that they cheat on their wife or taxes less than reality, for example). It is well-known that data-dredging leads to spurious correlation, as does too small of a sample size. It is understood that people put away negative results and that journals don’t publish criticisms as much as deserved.
        In the case of paleoclimate, those doing work in the field need to be a little more critical of the data, and not just throw it all in the sausage maker.

        • C Johnson
          Posted Dec 1, 2013 at 11:34 PM | Permalink

          Way. “…if I read a paper and I see in a table of proxies that it lists a series of tree ring series as: ITRDB – ca606 (and other codes), then I feel it is not reasonable to expect that I have a responsibility to go check each individual series used from the ITRDB…”

          The peers in a scientific community have the responsibility to perform just such checks. If the formatting of the relevant information obscures fact checking, then I find it reasonable to expect the authors and journal editors to establish standards and practices which ease replication and verification. Turn-key code and open data would go a long way to helping this.

          When you introduced Shi et al, you marched down a check list. So you at least sought to ensure certain things were performed correctly. This is quite different from looking for a point of failure, and potential points of failure were found in roughly a week after bringing it to Steve’s attention. The paper was published in April. Where are the professionals whose duty it is to examine these results?

          Will be interesting to see the remainder of the dissection.

        • Fred Harwood
          Posted Dec 2, 2013 at 7:03 PM | Permalink

          Provided that they explain why the data were unsuitable?

        • Posted Dec 3, 2013 at 7:53 PM | Permalink

          Traditionally, journals competed heavily for getting attention from readers who were also researchers in the same or nearby fields. That put a premium on making articles interesting, reliable, and useful to others. Too many crap articles would wrongfoot credulous readers and hurt those readers’ own research output. Editors, not reviewers, would have the responsibility to ensure levels of disclosure that balanced readers’ desire for credibility and transparency with their desire for hot, interesting results–the tradeoff coming from the supply side, where transparency and disclosure impose costs on authors. It seems that the policy-oriented use of results in the climate field has disrupted this incentive structure, with articles getting published even in specialist journals to indirectly influence non-science audiences.

          Perhaps this creates a business opportunity for a line of high-priced subscription-only newsletter “hot sheet” for researchers in which the kind of thing Steve M does here is applied to all the “official” literatures. Given the competition among these researchers, they may not pass along the information very much to non-subscribers, so as to preserve their own advantages.

    • Selwyn
      Posted Nov 30, 2013 at 8:25 PM | Permalink

      I don’t know about wrong but probably irrelevant.

      This seems a fine example of science in action. Researcher reads paper and forms conclusions based on reasonable assumptions of compentence of original author(s), uses conclusions as part of further work and publishes, published work triggers further investigation and analysis impinging on researchers conclusions, researcher acknowledges issue exists and indicates further research.

      That, a little inspiration and a whole lot of drudgery is how we advance!

      A researcher who is prepared to attempt to explore ideas on a forum such as this deserves to be engaged on those terms. The discussion is about data and interpretation, not motive.

      • thisisnotgoodtogo
        Posted Dec 1, 2013 at 7:37 PM | Permalink

        Selwyn said:

        “This seems a fine example of science in action. Researcher reads paper and forms conclusions based on reasonable assumptions of competence of original author(s)…”

        It seems a fine example of exactly what referees in the field have been doing.

        • thisisnotgoodtogo
          Posted Dec 1, 2013 at 7:40 PM | Permalink

          Selwyn, apologies for not quoting properly, and appearing to disagree, when the wider scope of your comment showed the remedy Steve applied – which your comment was referring to.

  21. rwnj
    Posted Nov 30, 2013 at 7:11 AM | Permalink

    In Bayesian theory, the correct weights used for each observation would be the inverse of the variance of the accuracy of the observation (also called the “precision”). If you use PCA or some other method to infer weights, and if the resulting weights cannot reasonably interpreted and scientifically verified to estimate precision, then the Bayesian will pronounce these weights to be sub-optimal. (BTW, an explicit data model showing you assumption as to the parametric form of the error process would also be helpful in evaluating the method.)

  22. Ed Barbar
    Posted Nov 30, 2013 at 10:50 PM | Permalink

    What happened to the comments? They look odd.

    I realize this is a “weak” post in many respects. I’ve only dug in deep enough a few times to understand the math that Steve has been talking about, and mostly have taken for granted that you can’t tell much from small populations.

    I’ll say simply, if I were to create a study, what is not to like about having Steve M. Review it? As I recall, A. Watts did this once, perhaps without enough time, and there were good comments from Steve.

    Let me put it this way. I’m a skeptic. I’m skeptical because the world is a complex place, and it’s rare that anything with multitudinous variables, complex interactions, but one variable that does change over time dominates all others so much it’s “that variable,” unless the variable is something like “Smoke for 40 years, with hundreds of carcinogens,” and even then it’s not a certainty, as my 85 year old mother in law is find and kicking smoking her cigarettes.

    To me, not including the good statistical analysis is foolish: it only adds doubt. Why not run the clean statistics? Who is afraid of the results?

  23. observa
    Posted Nov 30, 2013 at 10:51 PM | Permalink

    ‘Substitute “McIntyre” for the British Infantry’

    Why does the invention of the Vickers machine gun suddenly spring to mind with its concomitant casualty rates? Not a pretty sight those battlefields with nary a tree stump, let alone any bark left on them.

  24. Jeff Alberts
    Posted Dec 1, 2013 at 1:25 AM | Permalink

    Hmm, the site seems to have lost all its formatting. No header image, no CSS, just plain HTML styling it seems.

    Anyone else seeing anything strange?

    • Green Sand
      Posted Dec 1, 2013 at 5:12 AM | Permalink

      Re: Jeff Alberts (Dec 1 01:25),

      Yup same here, Firefox, Windows 7

    • Steve McIntyre
      Posted Dec 1, 2013 at 10:26 AM | Permalink

      My subscription to wordpress’ Custom Design feature had expired. I just renewed it. I think that this fixes the problem – let me know if it hasn’t.

      • TerryMN
        Posted Dec 1, 2013 at 10:54 AM | Permalink

        Yep, it’s all good – have a great day!

      • Unscientific Lawyer
        Posted Dec 1, 2013 at 4:44 PM | Permalink

        Bummer, the mobile platform worked yesterday. Unfortunately that’s now fixed.

      • Jeff Alberts
        Posted Dec 1, 2013 at 5:33 PM | Permalink

        Good for me now.

      • AlmostCertainly
        Posted Dec 2, 2013 at 1:09 PM | Permalink

        Any chance you could put a clickable link to ‘mobile device viewer’ on the list on the left side? Someone once mentioned a (multistep)way to make the mobile view thing work.

        On my BlackBerry, the pages show the leftside list and then the articles are all shown with the left ~20% of body text ruthlessly truncated. I can’t fix it with ‘pinch-pull’ dragging or any of the usual trix… Too many missing words to guess what they were…
        TIA
        AC

        • AndyL
          Posted Dec 2, 2013 at 2:28 PM | Permalink

          On my Android, I scroll to the very bottom and there is a link saying something like ‘View Full Site’ – that works.

        • AlmostCertainly
          Posted Dec 2, 2013 at 2:58 PM | Permalink

          The VIEW FULL SITE link at the bottom resolves nothing on BB Z10.
          Just repeats issue of hard-truncation of leftmost 20% of text,
          but not the left-sidea list of stuff starting at

          Home
          About

          Subscribe to CA
          Tip Jar

          Maybe it is just a Canadian thing…
          AC

  25. jim2
    Posted Dec 1, 2013 at 2:06 PM | Permalink

    EIV/TLS Regression – Why Use It?

    http://statpad.wordpress.com/2010/12/19/eivtls-regression-why-use-it/

    • jim2
      Posted Dec 1, 2013 at 4:22 PM | Permalink

      In one of the comments in the linked blog post, someone questions why proxies aren’t each first converted to a temperature, then treated in with methods similar to those used for, as an example, thermometer records. Dr. M. then discusses the difficulties converting proxies into a temparature. If it is difficult to do this for a given proxy, how does a mash-up of such proxies improve the situation?

      • Beta Blocker
        Posted Dec 2, 2013 at 1:21 PM | Permalink

        Re: jim2 (Dec 1 16:22),

        Jim2, let’s ask the CA user community a question …..

        Is there a relatively concise process flow description out there on the Internet somewhere which contains a summarized step-by-step narrative of the Mannian process for creating a hockey-stick shaped paleoclimate temperature reconstruction?

        A summarized process flow description would begin with collection of the proxy data, then describe the initial analysis of that data, then describe how data aggregation and data transformation processes are performed, then how data infilling is performed, and would then end up with a short description of the statistical derivation approach used for constructing the hockey stick shape from the combined/transformed data sets.

        We would be particularly interested in knowing what kinds of data transformation processes occur along the way, including brief descriptions of how different classes of proxy source (trees, ice cores, sediments, etc.) are normalized into a common data analysis framework.

        We would also like to know if any given set of data, either raw or transformed, which is passing through a particular process step does, or does not, need to have any kind of physical meaning attached to it — either before or after a given process step occurs.

        • jim2
          Posted Dec 2, 2013 at 2:54 PM | Permalink

          I believe the code Mann used, or a reconstruction?, has already been dissected and found to be wanting. IIRC, basically, it filtered the proxies such that proxies that matched the instrumental record were retained, and these could be inverted to fit if necessary. The net result of this process is that the instrumental record will be at the end of the resulting chart, but all the data that comes before the instrumental period basically gets averaged. So the end result is the handle of the stick made up by the average of all selected proxies, followed by the instrumental record, which moved up to form the blade.

  26. Barry Wells
    Posted Dec 1, 2013 at 3:19 PM | Permalink

    Re post By Bernie 1815 –

    This is a classic example of corruption of fact by misguided belief – in climate change facts are often massaged to show what the presenter believes them to be and not what the facts actually show.

    Below are the facts of the British regiments that fought at Waterloo , all those who fought were heroes no matter what part of the British Isles they came from.

    British Regiments present at the battle:

    Scots Regiments
    Royal Scots Greys now the Royal Scots Dragoon Guards
    3rd Foot Guards now the Scots Guards
    1st Foot now the Royal Scots
    42nd Highlanders now the Black Watch (the Royal Highland Regiment)
    71st Highland Light Infantry now the Royal Highland Fusiliers
    73rd Highlanders the Black Watch
    79th Highlanders later the Queen’s Own Cameron Highlanders, then the Queen’s Own Highlanders and now the Highlanders
    92nd Highlanders the Gordon Highlanders and now the Highlanders

    British Regiments excluding the scots regiments

    1st Life Guards now the Life Guards
    2nd Life Guards now the Life Guards
    Royal Horse Guards now the Blues and Royals
    King’s Dragoon Guards now the Queen’s Dragoon Guards
    Royal Dragoons now the Blues and Royals
    6th Inniskilling Dragoons later the 5th Inniskilling Dragoon Guards and now the Royal Dragoon Guards
    7th Hussars later the Queen’s Own Hussars and now the Queen’s Royal Hussars
    10th Hussars later the Royal Hussars and now the King’s Royal Hussars
    11th Hussars later the Royal Hussars and now the King’s Royal Hussars
    12th Light Dragoons now the 9th/12th Lancers
    13th Light Dragoons later the 13th/18th King’s Royal Hussars and now the Light Dragoons 15th Light Dragoons later the 15th/19th Hussars and now the Light Dragoons
    16th Light Dragoons later the 16th/5th Lancers and now the Queen’s Royal Lancers
    18th Light Dragoons later the 13th/18th King’s Royal Hussars and now the Light Dragoons
    Royal Artillery
    Royal Engineers
    1st Foot Guards now the Grenadier Guards
    2nd Coldstream Guards
    4th King’s Own Regiment of Foot now the King’s Own Royal Border Regiment
    14th Foot later the West Yorkshire Regiment and now the Prince of Wales’s Own Regiment of Yorkshire
    23rd Royal Welch Fusiliers
    27th Foot, the Inniskilling Fusiliers and now the Royal Irish Regiment
    28th Foot later the Gloucestershire Regiment and now the Royal Gloucestershire, Berkshire and Wiltshire Regiment
    30th Foot later the East Lancashire Regiment and now the Queen’s Lancashire Regiment
    32nd Foot later the Duke of Cornwall’s Light Infantry and now the Light Infantry
    33rd Foot the Duke of Wellington’s Regiment
    40th Foot later the South Lancashire Regiment and now the Queen’s Lancashire Regiment
    44th Foot later the Essex Regiment and now the Royal Anglian Regiment
    51st Light Infantry later the King’s Own Yorkshire Light Infantry and now the Light Infantry
    52nd Light Infantry later the Oxfordshire and Buckinghamshire Light Infantry and now the Royal Green Jackets
    69th Foot later the Welsh Regiment and now the Royal Regiment of Wales
    95th Rifles later the Rifle Brigade and now the Royal Green Jackets

    largely Scots, anyway – I don’t think so!!!!!!!!!!!!!!!!!!!!!!!!!!

  27. Robin
    Posted Dec 2, 2013 at 7:25 AM | Permalink

    Another interesting post from Steve, who clearly knows what he’s talking about. That said,
    I’ve seen little evidence that Steve is in anyway a great statistician. He may well be. However, the issues he raises are, sadly, rather fundamental. In other fields in which there is a greater emphasis on mathematical competence they would never arise.

    There is a simple way for the paleo folk to neutralise Steve (and all the evidence suggests that this is exactly what they want to do). All they have to do is engage some competent outside statisticians to work on their papers. If I encountered several HS papers from independent groups which, as a matter of course, included professional statisticians (who are also prepared to openly discuss and defend the details of their findings) then I’d be convinced (and so, I suspect, would many other CA readers). The paleo community need to explain precisely why they haven’t done this since its such an obvious step. Sadly, the conclusion I draw is that the community is scared of the possibility/probability that their techniques simply aren’t fit for purpose and that their error bars will shoot up.

    • Gaelan Clark
      Posted Dec 2, 2013 at 8:25 AM | Permalink

      You mention that this “problem” is “fundamental” and should be easily dealt with by including statisticians!

      So…..all of the bad models and the tuning could easily be made correct with a little better math?……sorry, that doesnt add up.

      Why havent any of the “climate scientists” done this yet?

      The paleo group AND all other “climate scientists” need to explain why—based on sloppy stats and baseless modeled date—my 5 year old child hears scare stories on her children’s shows about rising waters and exploding temps. I cannot even let her watch discovery channel or pbs without her getting the routine brainwashing. This effort to force their meme has found itself pushing through every and any form of media and is being touted as fact……..and the poor scientists, so maligned and so misunderstood, all they need to do is get a few statisticians to work the math around and POOF, it will all then be real.

      Has it occured to you….1.that statiticians have already been asked and have already said that the maths are wrong?…..2.that the stats dept doesnt rely on scare mongering for their funding…..

      • miker613
        Posted Dec 2, 2013 at 8:39 AM | Permalink

        Sorry, but did you read what Robin wrote?

      • Robin
        Posted Dec 2, 2013 at 8:40 AM | Permalink

        Gaelan.
        I get the impression you’ve replied to something I didn’t write.

        I did not mention models; I referred explicity to the paleo work. Not did I write that a little better math would ameliorate the situation. Better maths may or may not lead to a credible hockey stick. I have no idea what the result would be in advance of such a study.

        Regarding the point about statisticians declaring the work is wrong, can you please point out the body of work by professional statisticians pointing out that the approaches used in paleo reconstruction are flawed. I would very much like to see this. Occasional critiques of individual studies isn’t really enough. Evidence-based reasoning is important, especially in discussions such as these.

        • Gaelan Clark
          Posted Dec 2, 2013 at 8:50 AM | Permalink

          “”There is a simple way for the paleo folk to neutralise Steve (and all the evidence suggests that this is exactly what they want to do). All they have to do is engage some competent outside statisticians to work on their papers. If I encountered several HS papers from independent groups which, as a matter of course, included professional statisticians (who are also prepared to openly discuss and defend the details of their findings) then I’d be convinced””

          From those words, I take this away: that you think the only thing these papers need is……..statisticians, and then you would be convinced. And that that particularly would end MrM’s challenges.

          Sorry for reading what you wrote

        • Robin
          Posted Dec 2, 2013 at 8:52 AM | Permalink

          Thanks.

          My words summarised that I would be convinced by a thorough analysis by professionals who are able to master and defend the work, regardless of the nature of that result.

        • Gaelan Clark
          Posted Dec 2, 2013 at 9:18 AM | Permalink

          And here you have undercut the value of MrM’s work in your original first paragraph AND during the last decadehe has been the ONLY person providing independent analysis on the STATS, and somehow you still want to be convinced that the climate scare mongering community is right.

          MrM’s work IS CONVINCING AND IS INDEPENDENT.

        • Robin
          Posted Dec 2, 2013 at 9:42 AM | Permalink

          Describing Steve as someone who points out fundamental issues is hardly “undercutting” him.

          Contrary to your assertion, its untrue to claim he is the only person doing independent studies. You should also give credit to Von Storch, Soon, Roman etc.

          I doubt that anything I’ve written misrepresents the work which has gone on or disrespects those that have done it.

        • Gaelan Clark
          Posted Dec 2, 2013 at 10:24 AM | Permalink

          “”I’ve seen little evidence that Steve is in anyway a great statistician””

          Save for his more than decades long battoe to show that the models are baseless…..upon the stats!

          Clearly, you dont know what an underhand sleight it.

        • Robin
          Posted Dec 2, 2013 at 12:01 PM | Permalink

          Yes, I’m sure if you ignore all of the positive things I’d written then it can be interpreted as a sleight.

          As I mentioned earlier, you seem to be responding to things I didn’t write and looking for meanings which aren’t there.

          Please, avoid cherry picking quotes – read them in context. Also, remember that others deserve credit – something Steve has never been reluctant to do.

        • Gaelan Clark
          Posted Dec 2, 2013 at 2:05 PM | Permalink

          I obviously took what you wrote differently than what you intended.

          I apologize for that.

    • Posted Dec 2, 2013 at 8:29 AM | Permalink

      The conclusion I draw is that they tried that already, discovered for the third time that paleo reconstruction is a field before its infancy, and proceeded forward with their old,safe, incorrect ways rather than change employment

      • Robin
        Posted Dec 2, 2013 at 8:45 AM | Permalink

        I certainly think there is a lot of fear. I rather suspect the main motivation now is the loss of face which would likely occur should they do things the right way.

        We can’t see behind the curtains but I’d bet a hefty sum of money that the paleo folk are regarded with some embarrassment by other physical scientists. Steve + various CA contributors have made fools of them too many times.

        • Posted Dec 3, 2013 at 8:06 PM | Permalink

          Robin, have you looked at McShane and Wyner “A STATISTICAL ANALYSIS OF MULTIPLE TEMPERATURE PROXIES: ARE RECONSTRUCTIONS OF SURFACE TEMPERATURES OVER THE LAST 1000 YEARS RELIABLE?” (2011)? Does that paper meet the criteria you have in mind?

          From their abstract:

          “In this paper, we assess the reliability of such reconstructions and their statistical significance against various null models. We find that the proxies do not predict temperature significantly better than ran- dom series generated independently of temperature. Furthermore, var- ious model specifications that perform similarly at predicting tem- perature produce extremely different historical backcasts. Finally, the proxies seem unable to forecast the high levels of and sharp run-up in temperature in the 1990s either in-sample or from contiguous holdout blocks, thus casting doubt on their ability to predict such phenomena if in fact they occurred several hundred years ago.”

    • Bob K.
      Posted Dec 2, 2013 at 12:31 PM | Permalink

      “I’ve seen little evidence that Steve is in anyway a great statistician. He may well be. However, the issues he raises are, sadly, rather fundamental. In other fields in which there is a greater emphasis on mathematical competence they would never arise”

      A few things. First, statistics is not a branch of mathematics. Second, I am a statistician and I work with many of my kind. The best of the field are those whose develop strong intuition about the way real data behave. This intuition is gained by years of experience and is guided by mathematical competence. It doesn’t take sophisticated mathematical models, as John Tukey’s techniques for exploratory data analysis demonstrate. The host of this site doesn’t tout himself as a “great statistician” or even as just a “statistician” but from what I’ve seen his sense about data ranks with the best in my field.

      “There is a simple way for the paleo folk to neutralise Steve (and all the evidence suggests that this is exactly what they want to do). All they have to do is engage some competent outside statisticians to work on their papers.”

      The problem is that a competent statistician likely would raise the same issues pointed out on Climate Audit. Or is it a team player that you’re looking for? We call those “rubber stamps”.

      Steve: everything that I do is technically pretty much within the scope of a university undergraduate in mathematics of my day. The corpus of a “great statistician” is completely different and well outside my accomplishments, such as they are. I think of my analyses as data analysis, rather than “statistics” as it is now done in academics. I do not regard anything that I do as showing anything more than a degree of talent. I think that I have a particular talent for seeing patterns and oddities in large data sets but this is practical rather than theoretic.

      • Robin
        Posted Dec 2, 2013 at 1:20 PM | Permalink

        Bob

        “The problem is that a competent statistician likely would raise the same issues pointed out on Climate Audit. Or is it a team player that you’re looking for? We call those “rubber stamps”.

        I’m somewhat surprised by your reaction. Why do you think I’m looking for a rubber stamp ? Can you please explain with reference to what I’ve written (as opposed to anything I haven’t written).

        • Bob K.
          Posted Dec 2, 2013 at 2:22 PM | Permalink

          Robin, I didn’t say that you were looking for a rubber stamp, but I wanted to show the dilemma raised by your suggestion. Putting a competent statistician on a team where statistical competence has been wanting presents a real danger of independent thinking on the part of the statistician.

    • MrPete
      Posted Dec 2, 2013 at 1:15 PM | Permalink

      Re: Robin (Dec 2 07:25),

      You’re correct that many of the problems are fundamental; that’s part of what is so galling about the obvious issues. Perhaps you’ve missed the long-running request here for exactly what you propose: more involvement of qualified statisticians in the work.

      In that sense, you’re also correct that in some ways the work done here usually doesn’t require world-class stats expertise.

      However, if you take the time to read the CA history, you will find three realities that perhaps speak to your question, and explain why it’s essentially impossible to “neutralize” what is said here:

      1) Quite a few high level statisticians (as well as experienced professionals in other related fields) contribute here. Steve is not walking alone.

      2) As has been noted by others, one mark of experience is an innate familiarity with the data sets. In the past, Steve sometimes took on a fun exercise: by examining only the output of a reconstruction, he was able to largely predict which data sets were used as input. That’s one of many metrics that garner respect.

      3) Steve is very careful. It’s quite difficult to catch him in a significant error.

      • Robin
        Posted Dec 2, 2013 at 1:29 PM | Permalink

        Mr Pete

        My first comment, which seems to enraged so many to the extent that they didn’t bother to read past the first few words, was pointing out that the mainstream paleo folk are most likely avoiding external experts for fear of the results.

        I shan’t take up any more space on Steve’s blog though I would be surprised if he would have any issues with anything I’ve written.

        The whiff of tribalism here is a little unfortunate.

        Steve: nor do I particularly disagree. During the NAS workshop in 2006, I discussed the consult-a-statistician recommendation with von Storch and Zorita. Von Storch, who knows a lot of statistics, said that such consultations had mostly been unproductive in the past since academic statisticians typically wanted to apply methods that were inappropriate to the data. Actually econometricians have more intuitive feel for problems of autocorrelation, heteroskedasticity and spurious regression than a typical academic statistician. This is one of Ross McKitrick’s strengths. But problems like data snooping are very lowbrow and not something that an academic statistician can really get interested in.

        • Posted Dec 2, 2013 at 1:36 PM | Permalink

          Robin -your words could and perhaps should be interpreted as a stiletto hit against Mannianism. I’m sure if the Great Scientist could’ve found a statistician to back up his pretences, we would’ve been told left right and center.

          As the controversy is 15 years old, the chance of that happening is negligible.

        • MrPete
          Posted Dec 2, 2013 at 2:23 PM | Permalink

          Re: Robin (Dec 2 13:29),
          By my count exactly one person responded a bit upset at how you said it.

          If one person makes a tribe, then we all have some deeper issues :-D

        • Robin
          Posted Dec 2, 2013 at 3:21 PM | Permalink

          Steve – thanks for responding to my post.

          Although such an endeavor may be doomed by gatekeepers, a general review paper pointing out how the body of paleo reconstructions uses biased methods and inappropriate datasets would be extremely useful. A lot of people would love to know how many reconstructions in the literature are left standing after basic common quality criteria are applied to the methodology. As you certainly know, there is a lot of expertise and familiarity possessed by you + various CA posters.

          Such a paper would continue to promote the usefulness of non-affiliated scientists and the role of blogs. It would also secretly be devoured by a generation of graduate students in the field. Most love to see egg on their supervisors’ faces (regardless of what they may claim on forums). This is how one affects change in science. There are too many vested interests (academics, funding agencies, universities) to expect change there (note to Lew – this *isn’t* a conspiracy). They have invested a lot of reputation and finance in the paleo debacle. Occasionally, strong leadership can turn things around but the “leaders” of the field seem intent on throwing abuse rather than engaging in the substance of the argument. To me this behaviour is one of the most depressing aspects of the whole thing.

          On another note, I also tend to regard the stuff you do as being data analysis rather than statistics, as I do in my job as a subatomic physicist. When we found the Higgs going to two photons it was with a simple weighted average. Nobody shouted for a more sophisticated treatment – this comes after the discovery.

        • Matt Skaggs
          Posted Dec 2, 2013 at 6:51 PM | Permalink

          Robin wrote:

          “a general review paper pointing out how the body of paleo reconstructions uses biased methods and inappropriate datasets would be extremely useful.”

          Try this one:

          http://www.uoguelph.ca/~rmckitri/research/WegmanReport.pdf

        • Brandon Shollenberger
          Posted Dec 2, 2013 at 10:06 PM | Permalink

          Robin, in addition to gatekeeping concerns, such a paper might be doomed simply because of the amount of material it’d require. I don’t think you could hope to discuss all the reconstructions. That’s why I’ve previously suggested it’d be nice to have a website which does exactly that. The site could have a page for each reconstruction, and that page would discuss the reconstruction’s data,* methodology and why those led to the paper’s results. With enough interest, it could even provide the data used in each paper, as well as any code which is available.

          That’d be a massive undertaking though. I don’t know who’d be willing to do it. I’ve spent a great deal of time following the debates over temperature reconstructions since before this site existed, but I’m sure I don’t even know half of what would be needed for a site like that.

          I’m also pretty sure you wouldn’t be able to get funding for it. That amazes me because a resource like that would be a far more useful contribution than yet another paper using overly-complicated methods on much of the same data as before.

          *Ideally, data would also have pages of its own. That’d allow people to easily check what concerns have been raised about what data.

        • Brandon Shollenberger
          Posted Dec 2, 2013 at 10:15 PM | Permalink

          It occurs to me even if authors didn’t provide code for their reconstructions, a site like that could still provide code for replications of those reconstructions. That means the site could provide packages for reconstructions that include data and turnkey code for those reconstructions.

          If global warming is the serious problem so many people portray it as, I think sites like that should already exist. In theory, a site like that could exist for the global warming issue as a whole. It probably wouldn’t even cost as much as one GCM, and I think it’d provide a lot more value.

        • Robin
          Posted Dec 3, 2013 at 6:06 AM | Permalink

          Brandon
          There is a role for both academic papers and web info. A review fits best in journals IMO.

          The undertaking would be fairly large. However, when writing review papers (I’ve written a few) the trick is to define and limit the remit of the paper appropriately. One remit could be, eg, to consider the top y papers (in terms of citation yields) for different regions and show whether or not they match basic methodological requirements. Such requirements could be, eg, reproducibility, methodological bias (eg does noise also produce the shape of the published result), statistical precision (does the conclusion rely on one tree… ) etc. Much of the info has already been published on CA in one form or other.

          Its certainly easy to suggest these things and not so trivial to do them. However, I don’t see it as a huge task should somebody competent and knowledge have the time and interest to do it.

          The gain would be, for the first time, an assessment of the body of work against the same common set of statistical standards. There is a famous result in medical sciences showing how the efficacy of drug X depends on the quality of a study – the true efficacy being found in the asymptotic limit of high quality. It would be very interesting to see if any such results emerge in this case. This is very much a “statistics” paper and I could even see it evading the gatekeepers and being published in a stats journal.

        • RomanM
          Posted Dec 3, 2013 at 8:22 AM | Permalink

          I have never seen a critical review such as you describe in any Statistics journal. Most reviews of a topic summarize the available material without serious criticism of failed methodologies which are simply omitted from the conversation.

          On the other hand, the climate paleo community seems to have attempted an organized project whose aim was to evaluate the characteristics of different techniques for paleo reconstructions. The Paleoclimate Reconstruction Challenge appeared in January, 2008. After indicating that there were indeed some differences in the various reconstructions, the announcement goes on to state:

          The last millennium Paleoclimate Reconstruction (PR) Challenge—run under the auspices of the PAGES-CLIVAR-Intersection and co-sponsored by the Electric Power Research Institute (EPRI)—will allow us to directly address these concerns and to establish objective reconstruction benchmarks. The idea is to use results from state-of-the-art coupled Atmosphere-Ocean-General Circulation Models (AOGCMs) in both open- and blind-test reconstruction exercises. Individual reconstruction groups (and anyone who would like to participate) will be brought together and issued a small set of realistic pseudo-proxy series and calibrated “instrumental data” drawn from the model output. They will be asked to reconstruct the simulated climate evolution to the best of their technique’s ability. By comparing reconstructions with the full, “true” model climates, each group can assess their performance in great detail. A key objective of this project is to document how much of the true climate can be described with the combined set of reconstruction results, to determine which aspects of the overall or regional climate are captured well, and whether important elements are being missed.

          Data sets were provided for the project (part of which was used for a post here on the possible effects of correlation screening of proxies) and I expected to see a good scientific evaluation of the current reconstruction methodologies. Five years later, nothing has appeared.

          There is currently a web page at NCDC which states:

          SUMMARY/ABSTRACT: NOTE: The Paleoclimate Reconstruction Challenge is not active at this time. The Paleoclimate Reconstruction (PR) Challenge provided a simulated proxy data framework for evaluating differing paleoclimate reconstruction methods during the last two millennia. The pseudo-proxy and pseudo-instrumental data sets produced for the PR Challenge are housed here so that scientists, students,and citizens interested in exploring paleoclimate reconstructions can continue to have access to this state-of-the-art set of simulations, which includes forward modeling of proxy properties driven by climate variables. The data can be downloaded from the DATA DOWNLOAD table above.

          and a quick google search brought up a blog site offering an explanation of possible reasons for the lack of results:

          As far as I can see, no group has so far participated in the challenge, i.e. the challenge so far remains unanswered. This is a pity from my point of view but may be explained by the PAGES-2K effort and the IPCC AR5 procedures. This implies that the challenge may now spur some method intercomparisons; a hope that is supported by the (subjectively perceived) increased interest in the forward modelling of tree-ring proxies for reconstruction-method intercomparison articulated at the Madrid 2K-reconstruction-simulation-intercomparison workshop.

          I have my doubts that the challenge will happen in the near future.

        • Brandon Shollenberger
          Posted Dec 3, 2013 at 11:35 PM | Permalink

          Robin, I’ve never seen a paper like that, and I can’t imagine how it’d work. I wouldn’t be opposed to anyone trying though. I just have little hope it’d succeed. I don’t see how one would compress the material enough without people deciding it was done inappropriately.

          As an aside, I’m going to look into how much trouble my idea would be. I think there are about three dozen papers which would need to be covered. That’s not too bad. It might even be viable as a hobby if one was willing to post it as a work in progress.

        • Robin
          Posted Dec 4, 2013 at 8:51 AM | Permalink

          Thanks Roman for the info.

          Brandon – I’ve written more complex reviews (in another field, subatomic physics) than that proposed here. Its not a huge undertaking, so long as someone (or a group) has the energy, interest and expertise to do this.

          In fact, in a better functioning field there would already be several such reviews in print.

    • Gerald Machnee
      Posted Dec 2, 2013 at 4:22 PM | Permalink

      Re Robin:
      ***I’ve seen little evidence that Steve is in anyway a great statistician. ***
      What you should have noticed is that he stands above most of the people in the climate weblogs who “claim” to know what they are doing. He does this for a hobby not daily living.

      ***There is a simple way for the paleo folk to neutralise Steve (and all the evidence suggests that this is exactly what they want to do). All they have to do is engage some competent outside statisticians to work on their papers.***
      You are dreaming. This has been suggested by Steve. This would cause a rejection of many of the papers reviewed here and also used by IPCC. You will also notice some of the warmist weblogs continue to justify bad papers including “independent” studies showing the “hockey stick”. Something like the 97 percent consensus paper would have died long ago.

    • Posted Dec 3, 2013 at 9:16 AM | Permalink

      Robin, your recommendation of working with competent external statisticians has been made repeatedly to the climate science community, for example by the Wegman report in 2006 and the Oxburgh report in 2010. These recommendations seem to have been ignored.

  28. michael palmer
    Posted Dec 2, 2013 at 7:55 AM | Permalink

    Climate etc has this link to an article about Climategate and Steve’s role. T think it’s brilliant.

    http://michaelkelly.artofeurope.com/cru.htm

    • kim
      Posted Dec 2, 2013 at 8:18 AM | Permalink

      What’s truly remarkable is that Michael Kelly wrote that four years ago, and the ignorant juggernaut rolls on.
      ===================

      • kim
        Posted Dec 2, 2013 at 8:19 AM | Permalink

        Rolls on in the same old way, I might add.
        =================

  29. observa
    Posted Dec 2, 2013 at 10:10 AM | Permalink

    Speaking of epic battles over trees I wonder if it’s still there-

    http://en.wikipedia.org/wiki/Battle_of_Lone_Pine

  30. Posted Dec 2, 2013 at 1:24 PM | Permalink

    This is a perfect example of science as a social -or perhaps even personal- construct. For Robert and presumably many professionals in the field, Shi et al is just another contribution of possible validity and importance. Things are moving in the paleo world etc.

    For me and probably many commenters here, the mere mention of Graybill is the signal of a bull***t extravaganza, given its usual accompaniment of reticence, forgetfulness and general inability by the authors to defend their ground from age-old questions. The whole paleo field is irredemeably poisoned and can’t provide any useful information etc.

    I surmise that an outsider would not associate science with reticence.

  31. Jeff Id
    Posted Dec 3, 2013 at 10:30 AM | Permalink

    I can’t add much that hasn’t already been said so thanks Steve for continuing to do the hard archeological work.

    I suppose that I could write that I’m surprised much of the variance loss concerns which have been published at this point haven’t eliminated hard correlation sorting in paleo-literature. Unfortunately, I’m not that surprised though.

  32. Kenneth Fritsch
    Posted Dec 3, 2013 at 11:00 AM | Permalink

    I suspect Robert Way made a quick judgment/comment on this paper because it supported his POV on the matter. This is a very human trait one can see on all sides of the AGW issue. I think such exercises should make it evident that a public/blog figure commenting, in what might appear to some to be in an authoritative manner, on a particular part of the science related to AGW may well not have the background knowledge to do so. I would hope by now those reasonably people interested in these issue know that they should read the papers and make their own judgments.

    I would also note that while it is of interest to many to see and make analyses of the methodologies, calculations and interpretations used by those authors doing temperature reconstructions, in almost all cases of which I am aware those authors continue to make the same basic mistake. The temperature proxies cannot be selected after the fact of measuring the correlations of the proxy responses to the instrumental record. Most proxies that respond to temperature also respond to other variables and have temperature signals that must be found in these relatively high levels of noise. If one can assume the noise is random then the only reasonable method of finding the temperature signal is to find a reasonable and physically based method of selecting proxies prior to any measurement and then using all proxy data. If the noise is not random then not even this method of averaging out the noise will work. All these relationships need to be determined prior to doing what could be considered a valid reconstruction. Unfortunately those authors doing temperature reconstructions do not start from the beginning and thus the shortcut they take of selecting only those proxies that show the recent warming in their overall response has to bias the final results.

  33. Kenneth Fritsch
    Posted Dec 3, 2013 at 11:26 AM | Permalink

    I just reread RomanM’s post from some time ago at CA from the link in his post above where he gives mathematical rigor to what I was attempting to say above. His analysis and the example of the fishing net with holes in it makes clear the very basic problem involved in almost all temperature reconstructions.

    Why this problem has not been addressed by climate science is a bit puzzling? It is as if it is a conceptual blind spot. I know there are many hard scientists, used to doing preliminary testing before making a confirming test, who evidently do not understand that confirming tests are not always possible in the softer sciences.

  34. Tony Mach
    Posted Dec 3, 2013 at 12:21 PM | Permalink

    There is a “rerun” of Calvin&Hobbes comics on the web. Somehow the last panel of “today’s” comic might amuse one or the other here:

    http://www.gocomics.com/calvinandhobbes/2013/12/03/

    Direct link to the larger image:

    http://assets.amuniversal.com/93c18e40172a01312ddf001dd8b71c47?width=900

  35. MrPete
    Posted Dec 3, 2013 at 5:35 PM | Permalink

    There’s plenty of reason to anticipate less-than-desirable results from almost any study… much more humility needed.

    Note sure if CA has already had a discussion of Why Most Published Research Findings Are False. Interestingly, this is one of the most-cited papers out there, and the author, John Ioannidis, is well accepted among peers.

    Here’s a popularized view.

    I’ll also note that Prof Ioannidis is the one who is initiating a project to investigate issues of invalid outcomes in science research and identify potential process changes. Watch for the “Meta-Research Innovation Center at Stanford (METRICS)…”

    The problem is far more pervasive than I ever imagined. Another paper showed that of the 49 most highly cited medical research results in the most prestigious medical journals, 34 had been later retested, and 14 of those retests either contradicted or had significantly smaller effectiveness than first reported. A 41% error rate among the “best” papers.

  36. Kenneth Fritsch
    Posted Dec 3, 2013 at 6:15 PM | Permalink

    For every over the top temperature response for proxies like bristlecones there are no doubt under the bottom responses – think divergence problems. These over/under responses may well “average out” if one were to use all the available data and not select after the fact like is the case in almost all temperature reconstructions.

  37. JamesG
    Posted Dec 4, 2013 at 8:15 AM | Permalink

    The trouble is that mainstream scientists don’t seem to care whether these papers are correct or not and most don’t bother to read stuff outside their own field. Any criticism is seen as just speading doubt about an obvious truth.

    In my own field I have wasted countless hours trying to replicate papers only to discover that the paper a) just doesn’t work, b) works only for very limited cases, c) is worse than knowledge already out there or d) is a pure academic exercise with no practical utility. I get maybe one paper out of fifty to be useful – hence I just skim most. I’m guessing but maybe that’s the case for most folk.

    I remember Michael Crichton mentioning the effect of criticising a newspaper for getting something utterly wrong about your own field but being still prepared to believe the rest of the newspaper on subjects you know less about. It changed the way I read the newspapers!

  38. Boris
    Posted Dec 7, 2013 at 11:20 AM | Permalink

    “When you remove the bristlecones and the Briffa chronologies, is there still a HS reconstruction?”

    Do you mean that after almost a decade you are finally going to get around to answering this question? I assume we can look for more on Salzer in…2019?

    Steve: the HS in earlier multiproxy reconstructions depended on problematic proxies. Shi et al 2013 is a relatively new paper and was not available 10 uears ago, so I can hardly be expected to comment on it before it was published. Because Shi et al used bristlecones and Briffa, it is not “independent” of the earlier reconstructions. It has a few proxies that I haven’t previously parsed; I’ve taken a preliminary look at them and they don’t appear to have a HS. But I want to doublecheck this before posting on it.

    • Boris
      Posted Dec 7, 2013 at 3:15 PM | Permalink

      Okay, but when you do, be sure to define “hockey stick shape” clearly–are you talking blade or shaft or what? Orientation? Magnitude? The definition seems to change quite a bit.

      Steve: for blog posts, I presume that readers are familiar with the concept. We used the following definition in McIntyre and McKitrick 2005 (GRL):

      For convenience, we define the ‘‘hockey stick index’’ of a series as the difference between the mean of the closing sub-segment (here 1902–1980) and the mean of the entire series (typically 1400–1980 in this discussion) in units of the long-term standard deviation (s), and a ‘‘hockey stick shaped’’ series is defined as one having a hockey stick index of at least 1 sigma. Such series may be either upside-up (i.e., the ‘‘blade’’ trends upwards) or upside-down.

      I don’t think that there’s anything magical about this definition, but it was serviceable for the purposes of that paper. I think that my usage has been consistent over time, though (and I presume that this is what you have in mind) the shape of the Mann et al 2008 reconstruction and some other recent reconstructions are not as HS-shaped as Mann et al 1999 and is more like Moberg 2005. Their main unifying feature is that the modern portion tends to be a fraction higher than the medieval portion, though the differences are generally razor-thin.

      • Brandon Shollenberger
        Posted Dec 7, 2013 at 4:02 PM | Permalink

        Wait, what? Since when does a hockey stick shape need to be defined? Can’t we just look at a hockey stick and see?

        I get some people may label reconstructions that are nowhere close to resembling hockey sticks, hockey sticks, but why should people misusing words require us define those words clearly?

        Shouldn’t Boris be taking this issue up with the people misusing the phrase?

        • Carrick
          Posted Dec 8, 2013 at 12:56 PM | Permalink

          Mann seems to have a bit of trouble with this. He thinks his 2008 publication confirms his earlier 1998 paper.

          It’s the only field I know where two completely dissimilar results can be seen as confirmation of each other.

      • TAG
        Posted Dec 8, 2013 at 5:52 PM | Permalink

        Could I suggest a more functional definition of a “hockey stick” curve as it applies to climate science. I would suggest that a climate science hockey stick temperature curve is a curve which shows a high degree of correlation with the atmospheric CO2 concentration. This definition relates to the ability to create a transfer function between them. a high degree of correlation would indicate a simple transfer funciton with little influence from other factors. Hence a diminished role for “natural variation”.

        So “hockey stickedness” would be the correlation between normalized CO2 and temperature curves.

        This measure would seem to reflect the reason why “hockey stick” curves became a subject of interest in the field.

  39. Kenneth Fritsch
    Posted Dec 8, 2013 at 2:56 PM | Permalink

    “Mann seems to have a bit of trouble with this. He thinks his 2008 publication confirms his earlier 1998 paper.
    It’s the only field I know where two completely dissimilar results can be seen as confirmation of each other.”

    Exactly, Carrick, I thought we were going to get a concession speech from Mann after Mann(08) – even though I totally disagree with the selection of proxies after the fact approach in that paper.

Follow

Get every new post delivered to your Inbox.

Join 3,423 other followers

%d bloggers like this: