An Interlude

I’ve obviously been in a quiet blogging patch. My wife and I were visiting our daughter who lives in western Canada.

I’m still amazed and flattered at my inclusion on the New Statesman list. I will post my reflections on this inclusion at some point, but I want to discuss the point more from a sociological than personal point of view. I also want to think about it for a while before commenting.

I spent some time re-visiting Yamal issues in the light of Climategate correspondence. I had hoped that the inquiries would have actually inquired into Yamal. I’ve got quite a bit of pending material on this which I’ll get to some time.

I’ve also spent a fair bit of time on Ljungqvist 2010 (and 2009). I’ve corresponded cordially with Ljungqvist and now have a complete collection of his 2010 proxies – a few of which, interestingly, were downloaded from Climate Audit (e.g. the Esper version of Polar Urals). Ljungqvist sent me a Supplementary Information listing the URLs from which data sets were downloaded, including some data sets in which the website was denoted only by its IP address, rather than its name – the IP address was Climate Audit before the move to wordpress. At this point, my interest is not with the “results”, but with the data. Ljungqvist’s collection includes many paleolimnological proxies, some of which overlap Kaufman et al 2009, but others that are not available elsewhere. Rather than getting too fussed about medieval-modern differentials in one squiggle rather than another, I would like to go a little deeper into the paelolimnological data as a whole – re-examining what, if anything, can be deduced from it. We discussed a number of data sets in connection with Kaufman et al 2009, but there are others that warrant examination.

Ljungqvist also introduced a new speleothem proxy. Speleothems have been playing an increasing role in recent multiproxy collections – Mann et al 2008 has a lot of speleothems in the AD1000 network. There have been interesting speleothem datasets archived at NCDC in the past 18 months – many involving Chinese speleothems, archiving practices for which seem to be much, shall we say, prompter than for speleothems from Raymond Bradley’s NSF-financed unit at the University of Massachusetts or Lonnie Thompson’s NSF-financed ice cores.

I’ve also spent time on McShane and Wyner – unfortunately, more time after submitting our discussion than before. The 93-proxy dataset that they use in their AD1000 reconstruction includes 24 strip bark series and 2 Korttajarvi series (Tiljander) without removing the contaminated segments. So it requires great caution in interpreting their results other than where they are, in effect, only mathematical. While I welcome their interest in the field, I wish that they hadn’t used things like “lasso” that aren’t actually in use in the field.

I also spent some time re-visiting boreholes – the interpretation of which seems very problematic. I’ve taken a look at some pre-inversion measurements in mining areas that I know a bit about and have some work in progress.

Also I noticed some interesting things about accumulation time series in long glacier holes – work in progress on that.

As regular readers know, as interesting as I find the statistical issues of reconstructions, I think that climatescientists spend too much time worrying about multivariate techniques – most of which is lost energy in the absence of adequate statistical theory anyway – and far too little time worrying about the data. A statistical model that encompasses 6-sigma strip bark excursions is not easy to develop, but I don’t see how the field advances without developing such a theory. I had some cordial correspondence with McShane and Wyner, who are very interested in the statistical aspects of the reconstruction projection and will hopefully be interested in these problems as well.

Speaking of which, I’ll also mention another article that discusses statistical aspects of climate reconstructions that, to my great shame, I didn’t attend to when it first came out – Loehle and McCulloch 2008, a situation that I also hope to remedy. I had presumed that this article was simply a corrigendum, but it has a very interesting discussion of “cross-sectional heteroscedasticity” in the context of CPS reconstructions. Hu McCulloch also has an excellent working paper on reconstruction methodology, attention to which is long overdue.


  1. TomRude
    Posted Oct 3, 2010 at 7:09 PM | Permalink

    From the tone of your note here, it looks like Ljungqvist, McShane and Wyner are authors who are interested in your comments, and that’s a welcome change no?

    • F. David Rounthwaite
      Posted Oct 21, 2010 at 7:23 PM | Permalink

      Many congrats, Steve. Lots of chat about this at 371 Bloor Street West when the local news appeared this week.


      • Steve McIntyre
        Posted Oct 21, 2010 at 11:10 PM | Permalink

        Hi, David. Great to hear from you. Cheers, Steve

  2. ben
    Posted Oct 3, 2010 at 8:19 PM | Permalink

    Steve it would be interesting, in future months when this settles down a little, to hear your perspective on any differences in your experience in your dealings with McShane and Wyner versus others in the climate community as regards to openness, attitude, skepticism, tolerance of criticism and so on. A priori I’d expect a big difference given the, er, troubles you’ve experienced previously. A comparison of median response times would be interesting, for example.

  3. pete m
    Posted Oct 3, 2010 at 9:27 PM | Permalink

    No matter what, family comes first. They’ll be at your bedside while we’re just moaning about the lack of posts lately!

    Wow that’s quite a to do list. We’ll be patient!

  4. Anthony Watts
    Posted Oct 3, 2010 at 9:35 PM | Permalink

    Hi Steve,

    I sent an email to your yahoo account a few days ago. Just a gentle prod of a question



  5. AusieDan
    Posted Oct 4, 2010 at 3:07 AM | Permalink

    I am amazed at the scale of your work load!
    I loook forward to hearing your reports as it progresses.

  6. Robin Edwards
    Posted Oct 4, 2010 at 8:18 AM | Permalink

    I too have been having some nice correspondence with Fredrik (Ljungqvist) about his paper, over the last couple of weeks, and he’s sent me the same stuff that you’ve got, Steve – as far as I can tell. I’ve not had time to access all the URL’s he has supplied in a .xls file, but have looked at some of the “personal Communication” files, with interesting outcomes. I’m not going to write about them here, since I’d rather get approval from Fredrik first.

    I must follow this thread closely!


  7. Kenneth Fritsch
    Posted Oct 4, 2010 at 9:52 AM | Permalink

    I hope your subject interest noted in the introduction to this thread is a preview of what we can expect for future threads here at CA.

  8. John Hekman
    Posted Oct 4, 2010 at 11:42 AM | Permalink

    Regarding your surprise at the New Statesman recognition, it seems that you really do not appreciate how important your contribution to understanding climate policy has been. There are many highly qualified climate policy sceptics out there, but none of them has your combination of mathematical ability, determination and creativity. Your old friend George Stigler was fond of saying that personalities don’t matter that much; if one person does not invent something, then the next guy will. But if you and Ross had not done your analysis and stuck to it through all the flak you took, I don’t know who else would have done it. Putting your work next to some of the others on the Statesman’s list, like some South American dictator, and there is no comparison.

    • Steve McIntyre
      Posted Oct 4, 2010 at 8:26 PM | Permalink

      Stigler wasn’t exactly an “old friend”. I only met him a couple of times – all on the golf course.

      I’m sure that I’ve told the story of my contact with Stigler when I was about 14. He threw my golf ball into the woods after I’d hit a great shot to be on the 11th green at Muskoka Lakes with a great chance for an eagle. (I annoyed the geezers ahead of me by hitting before they were off the green.) It’s funny what sticks in your mind. I remember the shot clearly though it was almost 50 years ago. I think that I knew who Stigler was because I’d caddied for my father in a match against Stigler in a club tournament the year before. I don’t remember who Stigler was playing with. I played Stigler in the club semis later that year and whipped him. :) I was a teenager and didn’t have the faintest interest in what he did. I knew that he was an American, I might have known that he was from Chicago.

      I played a lot of golf in my early teens, but haven’t kept it up. The other hole that I remember really vividly was my first birdie when I was about 8 or 9. The 12th hole at Muskoka was really beautiful, elevated tees looking over the lake with big Canadian Shield rocks. I was still too small to hit the green from the tee, but I bounced a ball off one of the rocks about 20 feet from the pin and got my first birdie.

      • justbeau
        Posted Oct 4, 2010 at 8:42 PM | Permalink

        George’s son Steve is a distinguished statistician. In fact, Milton Freidman and George were practicing statisticians during WWII.

  9. Hu McCulloch
    Posted Oct 4, 2010 at 12:51 PM | Permalink

    Hu McCulloch also has an excellent working paper on reconstruction methodology, attention to which is long overdue.

    Thanks for the mention — in case anyone is in Columbus this week, I will be presenting it in the OSU Statistics Colloquium on Thursday Oct 7 330-430, in Room 170 of the Eighteenth Avenue Bldg., 209 W. 18th Ave. Open to the public.

    A 9/12/10 revision of the paper is online via

    This is a corrected version of the one I mentioned last year on CA at, but the results are the same.

  10. Posted Oct 4, 2010 at 2:56 PM | Permalink

    fyi: is a dead link on

  11. Posted Oct 4, 2010 at 8:46 PM | Permalink


    As a loooong time visitor to this site who appreciates the science presented and debated here, and the diligent work you have taken on in a wholly honest and gentlemanly manner, I offer a heartfelt thank you and hearty congratulations, for your being recognized as #32 of the “50 People Who Matter” for 2010, by New Statesman.

    Personally, I believe you should have been listed as #1. But that just reflects my bias. Which is based on my own evaluation of the potential mess I believe you’ve spared us all from, had you never taken an interest in, nor gotten involved in “auditing” so much of climate science.

    Many people, including me, thank God you did get involved. And with that you’ve secured a large place for yourself in history. A hero to some, including me, and a spoiler to those that could have made enormous fortunes trading ~carbon credits~, at the rest of our expense. LOL

    All things considered, I’m sure it may take some time to unwind a bit, now that the battle appears to be on the downhill side of events. But I’ll caution you to stay sharp and at the ready, because as an American steeped in our politics I assure you congress will be calling on you sometime after the November elections.

    With all that said and sincerely meant, I hope you’ll take some time to enjoy yourself for a while. And if you ever feel compelled to come to this part of the world, if you don’t mind sharing a smallish condo with an aging wannabe polymath, and an ornery fat chihuahua, mi casa es su casa. Just bring a small toy for the dog. That way she might leave you alone.

    Waiting on standby here…

    Highest possible Regards,


  12. Alexander Harvey
    Posted Oct 5, 2010 at 8:05 AM | Permalink


    Some years ago I spent a lot of time looking into boreholes and it all turned into a counsel of despair.

    I did get some theoretical results, some of which I think I must have posted on your old site.

    Basically the eigenvalues that are evident in individual boreholes (which you wrote about) can be seen as arising from inherent eigenfunctions of the diffusion process.

    Also some work on temporal resolution (approximately {sqrt(e)*x,x/sqrt(e)} i.e if you are looking at a hump around 500 years ago it cannot be resolved beyond the range (300-825), it depends on the aperture and the number and distribution of data points but it is a guide figure. I also looked more generally at aperture effects (basically how the aperture [log of the ratio of max to min depth] is related to the eigenvalues. As I recall some of the borehole apertures were not sufficent to cover the span 1500-1950 without risking distorting the signal.

    Perhaps the bigest bugbear is due to the discontinuities in the slope due to the background vertical heatflux as it passes between strata with marginally different conductivities.

    The process used in the reconstructions is first to guess the background slope and then construct the anomalies and invert them. This process is completely unsound and small errors in guess give rise to large errors in output (also guessing wrong leads to the injection of a constant slope into the data, it just so happens that constant slopes are unphysical as they do not relate to any possible surface temperature history).

    The alternative is to allow the background slope emerge as a result from the inversion process not as an input but this means handling the data at one difference order less(not looking at the area under the slope but at the curvature of the slope only) effectively using the first differentials of the functions and making the inversion more rational but much less precise.

    Regarding the data itself some of the (I think included) wells are only logged to 1/10 the temperate resolution (.1C as opposed to .01C if my memroy serves).

    As I recall the metadata lists neither the difusion coefficient, specific heat, nor mineral clasification only the condictivity (or its inverse I don’t recall which) so a blanket specific heat has to be assumed. Alos some of the conductivities look convincing (lots of digits and vary between holes logged by the same researcher) others don’t, in particularly batches with the same value for each hole.

    I believe that there was a selection process that picked out the best holes by some criteria, unfortunately their seems to be nothing in the archived metadata to indicate how this could be achieved.

    I think that there is more but I gave up on this years ago and my memory is not what it was.

    I did write to the people that I thought should be interested but as it was basically a counsel of despair it was not well received, the best offer I got was come back when you have analysed all the boreholes, which was beyond my means (it would have taken decades without automation and a proper computer).

    I did at one statge divise a scan to look for strata lyer discontinuities (looking for tell-tale “V” like turns in the temps) but there was no way of fixing what was an acceptable amount of discontinuity and (as I recall) stripping out the most obvious duff holes left one with holes that contained little in the way of signal.

    Also if I recall the archived data does not necessarily include the complete log (topping and tailing has been performed to get rid of data corresponding to dates much before 1500 and the last decade prior to logging). This is a real pity as it makes the data conform to the prejudice of the model used (that such data would be unhelpful) and possibly disards that data on which the background sope estimation was predicated.

    Anyway I hope that you do revisit it.


  13. kim
    Posted Oct 6, 2010 at 10:01 AM | Permalink

    This business of ‘Steves’,
    Let’s touch upon the matter.
    Every field needs one.

  14. kim
    Posted Oct 6, 2010 at 10:02 AM | Permalink

    And climate science
    Got its ‘Steve’ and got him good,
    In the nick of time.

One Trackback

  1. By Top Posts — on Oct 4, 2010 at 7:21 PM

    [...] An Interlude I’ve obviously been in a quiet blogging patch. My wife and I were visiting our daughter who lives in western [...] [...]


Get every new post delivered to your Inbox.

Join 3,193 other followers

%d bloggers like this: