Antarctic Spatial Autocorrelation #1

“Noisy” covariance matrices have been discussed here on many occasions in a variety of contexts, largely because the underlying strategy of Mannian methods is to calculate the covariance of everything to everything else and then calculate verification stats using methods that ignore the data mining that effectively takes place with huge covariance matrices. Steig et al 2009 is no exception.

The “justification” in Steig is as follows:

In essence, we use the spatial covariance structure of the surface temperature field to guide interpolation of the sparse but reliable 50-year-long records of 2-m temperature from occupied weather stations. Although it has been suggested that such interpolation is unreliable owing to the distances involved, [1. Turner, J. et al. Antarctic climate change during the last 50 years. Int. J. Climatol. 25, 279–294 (2005)] large spatial scales are not inherently problematic if there is high spatial coherence, as is the case in continental Antarctica [Schneider, D. P., Steig, E. J. & Comiso, J. Recent climate variability in Antarctica from satellite-derived temperature data. J. Clim. 17, 1569–1583 (2004)]

Which raises the obvious question: is there “high spatial coherence”? The citation for this (Schneider et al 2004) is unsurprisingly self-referential. Although the “active ingredient” in the pre-1980 reconstruction is the surface station record, Schneider et al 2004 provides no evidence for “high spatial coherence” in these records; instead, it discusses the AVHRR records.

I thought that it would be interesting to do a scatter plot of the distance between surface stations (here only the surface stations, not the AWS stations) against the inter-station correlation – the sort of low brow analysis that the Team abhors. Here is a location map of the stations. There are several Southern Ocean stations (e.g. Campbell Island, Macquarrie Island near NZ and Grytviken near Argentina), that are included in the data set.

Here is a scatter plot of correlations to distance, evidencing a strong decay with distance. There is a highly significant fit to a simple relationship based on exponential decay with distance (r^2 of 0.76 and t =71. (At this time, I’m not trying to assert that this is the form of a “true” relationship, only that it is very evident that there is a strong spatial decorrelation in the first 2000 km.)

fm=lm(cor~ I( exp(-dist/1000)-1) ,data=station);

Here is a histogram of correlations for distances above 2500 km, showing that they are distributed on both sides of zero – on balance slightly negative. It seems to me that it would be uphill to demonstrate that high correlations in this histogram are “significant” “teleconnections”, as opposed to the sorts of correlation turned up in a population of nearly 1000 autocorrelated series.

Above 2500 km, the covariance between surface stations all just looks like noise to me. The above decay of correlation with distance strongly argues in favor of the “null” model being stations with spatial autocorrelation decaying to 0 by 2500 km or so.

Note: In comments below, readers have argued that this is too negative a characterization, Ryan pointing to the clump of positive correlations around 6000 km. In light of these comments, I re-examined the data set and found that the clump of positive long distance correlations involve Campbell and Macquarrie Islands to Grytvyken and the Antarctic Peninsula stations; correlations to the Antarctic continent are slightly negative. For now, I’m just struck at the obvious decorrelation with distance. So far I’ve not carried out statistical analyses against a null model. You’d think that Steig et al would have done this sort of thing.

UPDATE Feb 24: Here’s a variation of the above graphic limiting stations to “continental” Antarctica. First here are the stations used:

Location map of surface stations used in correlation-distance plot below. 17 stations are excluded, of which 5 are islands in the Southern Ocean and 12 are in the northern part of the Peninsula.

Now here is the same scatter plot for correlation vs distance, which perhaps shows a more coherent decorrelation than with the island and far peninsula stations included.

Scatter Plot of Correlation vs Distance, 25 Stations.

79 Comments

  1. Steve McIntyre
    Posted Feb 20, 2009 at 8:52 PM | Permalink | Reply

    source(“http://data.climateaudit.org/scripts/steig/collation.functions.txt”)
    ### circledist
    #calculates great circle distance with earth radius 6378.137 km
    #from http://en.wikipedia.org/wiki/Great-circle_distance 2nd formula
    circledist =function(x,R=6372.795) { #fromlat,fromlong, lat,long
    pi180=pi/180;
    y= abs(x[4] -x[2])
    y[y>180]=360-y[y>180]
    y[y< = -180]= 360+y[yF)
    #1 1583 59.98
    #2 1584 63.45 -1 -3.47 91.51 <2e-16 ***

    #SCATTER PLOT
    par(mar=c(4,4,2,1))
    plot(station$dist,station$cor,xlab="Dist (km)",ylab="Correlation",col="grey70",ylim=c(-.5,1))
    # 0.4958,
    a=seq(0,6000,100)
    lines(a,fm1$coef[1]*exp(-a/1000),col=2)
    abline(h=0,lty=2)
    title(“Antarctic Surface Stations”)
    text(4000,1,paste(“cor=”,round(fm1$coef[1],3),”*exp(-dist/1000)”),col=2,font=2)
    abline(v=2500,col=2,lty=3)

  2. Posted Feb 20, 2009 at 9:18 PM | Permalink | Reply

    “the sort of low brow analysis that the Team abhors”

    Sigh.

    Steve: Perhaps you can refer me to the place where Steig et al analyze the spatial covariance between Antarctic surface stations or where they refer to an article carrying out such an analysis.

    • Jonathan Schafer
      Posted Feb 20, 2009 at 9:37 PM | Permalink | Reply

      Re: Walt Bennett (#2),

      Did you have a point or are you just trolling? That’s a rhetorical question. No need to bother answering.

    • Terry
      Posted Feb 23, 2009 at 10:50 PM | Permalink | Reply

      Re: Walt Bennett (#2),

      “the sort of low brow analysis that the Team abhors”. Steve I totally agree with your approach In my line of atmospheric science the results of complex models must also pass the test when compared to “simple” basic first principles models. In that respect I am in Roy Spencer’s camp. If the results dont pass the basic test, then go look for the reasons why. You never know it may lead to re-evaluation of the complex model.

    • Posted Feb 24, 2009 at 10:35 AM | Permalink | Reply

      Re: Walt Bennett (#2), Steve, perhaps you can advise me why you have a penchant for characterizing certain scientists in such a gleefully ad-hom way.

      I’m the watchdog’s watchdog.

      • bender
        Posted Feb 24, 2009 at 10:38 AM | Permalink | Reply

        Re: Walt Bennett (#47),
        That’s an ad hom, Walt.

      • Steve McIntyre
        Posted Feb 24, 2009 at 10:57 AM | Permalink | Reply

        Re: Walt Bennett (#47),

        Are you disagreeing with any statement in the post?

        Do you disagree with this statement for example?

        the underlying strategy of Mannian methods is to calculate the covariance of everything to everything else and then calculate verification stats using methods that ignore the data mining that effectively takes place with huge covariance matrices.

        I’m definitely criticizng a prevalent style of doing these sort of studies. And it is a style, just as there are recognizable styles of art or clothing or pottery.

        An objectionable feature of this style is that key results are frequently asserted without proof. I believe that this is a correct characterization – do you disagree or do you merely find it unpleasant that I make the observation?

        Here they do not present any support for their claim that there is “high spatial coherence” between the surface stations. Again, if you believe that my opinion is incorrect, I invite you to provide evidence for this assertion within the article or in a reference.

        • bender
          Posted Feb 24, 2009 at 12:19 PM | Permalink

          Re: Steve McIntyre (#49),
          This IS a systematic pattern in Mann’s team work. It is EXACTLY why he needs to be working with real statisticians. Shake this bad habit and his agenda falls apart.

        • Posted Feb 24, 2009 at 5:37 PM | Permalink

          Re: Steve McIntyre (#49), Steve, I criticize your use of the term “the sort of low brow analysis that the Team abhors” and you know very well why I do.

          But if you insist on having me break it down: “low brow” is a self mocking reference to your belief that you find value in such methods that “they” do not; you group “the Team” together for this reference, and you assert that individually and as a group, they cannot be bothered to stoop to such methods, hence your use of the term “abhors”. You were being clever, playing to your panting crowd (and what a crowd they are, Steve, always ready to leap to your defense, swords at the ready).

          I’m not having it. Either this is a serious science site or your intent is to mock science. You cannot have it both ways.

          I’ll check back from time to time to see if you move your desk from the back of the class.

          I think you do serious work. I also think you bask in the glow from these adoring admirers who love your smirking approach to your work, whereas that is exactly what ruins the experience for me.

          These are very serious times, sir.

          Steve: Regardless of whether I can personally make a modest contribution to understanding particular issues, in my opinion – and precisely because matters are serious – it is important that climate scientists stop making assertions without providing their supporting evidence. I can’t personally re-work the analyses of dozens of people but I can take issue with this pernicious style of presenting scientific evidence.

        • Kenneth Fritsch
          Posted Feb 25, 2009 at 9:56 AM | Permalink

          Re: Walt Bennett (#60),

          You were being clever, playing to your panting crowd (and what a crowd they are, Steve, always ready to leap to your defense, swords at the ready).

          Walt Bennett, what you seem to lack in your attempt to emulate the auditor (by auditing the auditor) is that you have not provided any serious analysis of the subjects discussed here at CA. Your only effort to date seems to be that of baiting the people who post here while at the same time complaining about what you perceive as SteveM’s baiting of climate scientists.

          If you should decide to roll up your sleeves and do some honest to gosh tough analyses here at CA (in emulation the auditor(s)), I might take seriously your admonishments — or more likely ignore them in favor of any substantive statements or discussions you might have contributed.

      • Cliff Huston
        Posted Feb 24, 2009 at 11:13 AM | Permalink | Reply

        Re: Walt Bennett (#47),
        You seem a bit confused:

        ad hominem
        adverb & adjective
        1 Attacking an opponent’s motives or character rather than the policy or position they maintain : vicious ad hominem attacks.
        ORIGIN late 16th cent.: Latin, literally ‘to the person.’

        Steve is clearly attacking team’s methodology, not their character or motives.

        • Mark T
          Posted Feb 24, 2009 at 11:52 AM | Permalink

          Re: Cliff Huston (#50), Walt is not confused, his goals on this blog are quite clear and the distinction you point out is not something he cares to either understand or acknowledge.

          Mark

  3. Pat Frank
    Posted Feb 20, 2009 at 9:43 PM | Permalink | Reply

    Hansen did several similar surface station temperature covariance plots at selected latitudes, I think in his 1987 paper on global average temperature. I can find out which one later, if you like. Anyway, he got very similar results. As I recall, the covariance pretty uniformly reached about 0.5 at 1000 km or so.

    That little tip-up to positive correlation at 6000 km is interesting, isn’t it. Maybe a teleconnection resonance at quarter-global wavelength. :-)
    .
    Re: Walt Bennett (#2), The mannered sigh. Where’ve we seen that so often before?

  4. bender
    Posted Feb 20, 2009 at 9:51 PM | Permalink | Reply

    I’m glad you did this analysis. I was going to suggest it, but felt I was taking up too much bandwidth.
    .
    But look at the actual pattern there. After the correlation goes negative at 5000km it goes positive again at 6500km. (A quadratic would fit so much better than the exponential that the difference would surely be highly significant.) But the point is: what’s up with that? There actually does appear to be a long-distance teleconnection here!
    .
    One of the reasons I enjoy Steve’s work is he often explores the data starting in the same lowbrow manner that I would.

  5. bender
    Posted Feb 20, 2009 at 9:51 PM | Permalink | Reply

    crosspost.

  6. bender
    Posted Feb 20, 2009 at 9:55 PM | Permalink | Reply

    Actually, it looks like a hockey-stick. (Must get rid of the high correlation at 0km. Teamthink.)

    • Ryan O
      Posted Feb 20, 2009 at 9:58 PM | Permalink | Reply

      Re: bender (#8), Just flip the x-axis. Those numbers represent distances in the past.

  7. Ryan O
    Posted Feb 20, 2009 at 10:03 PM | Permalink | Reply

    Steve, TBH, a function decaying to zero like what you have doesn’t quite look right. It almost looks as though there is an honest-to-goodness negative correlation when the distances get far. One thing I noticed when plotting means on a map from 1957-2006 is that west and east Antarctica oscillated just about 180 degrees out of phase.
    .
    (Actually, it looked as if temperature waves went down the peninsula, through West Antarctica, and up East Antarctica)
    .
    Maybe the ocean is your teleconnection device.

  8. Jesper
    Posted Feb 20, 2009 at 10:29 PM | Permalink | Reply

    This is weak….eyeball the data. Why are you choosing a model that decays to zero? To match your thesis?

    A wave-like distance-correlation relationship is prima facie evidence of large-scale coherence.

    • bender
      Posted Feb 21, 2009 at 12:20 AM | Permalink | Reply

      Re: Jesper (#11),
      It has been shown many many times that weather & climate correlations DO in fact decay to zero over space … in the limit. It’s just that Antarctica ain’t “the limit”. Guess an exponential decay null model ain’t so bad after all.

  9. Steve McIntyre
    Posted Feb 20, 2009 at 10:34 PM | Permalink | Reply

    Hmmm, I was looking ahead a little and was maybe too quick. The positive correlations above 6000 km nearly all involve two Southern Ocean stations included in the data set (Campbell Island and Macquarrie Island). We previously ran into Campbell Island when we were examining GISTEMP Step 2, when we pondering NASA’s inability to locate Wellington NZ after its destruction by the Scythians in 1989 (or was it the Assyrians?)

    I guess that was before began to update station records. (Perhaps he’ll turn his attention to this vexing problem.)

    These correlate positively to the Antarctic Peninsula and nearby island stations also in the surface data set.

    Interestingly the negative long-distance correlations are to stations on the Antarctic continent itself.

  10. Jesper
    Posted Feb 20, 2009 at 10:45 PM | Permalink | Reply

    There are also dynamical reasons for expecting positive correlations at ~6000 km. On many (most?) timescales, the circulation over high Southern latitudes varies in a doughnut pattern, with pressure/temperature anomalies uniting areas of the Southern Ocean and Antartic coast, in contrast to the Antarctic core – the Antarctic Oscillation / Southern Annular Mode.

    6000 km is roughly the Antarctic diameter, and positive correlations at this length likely reflect coherent anomalies among stations on the coastal rim.

    It would be interesting to know where Steig’s analysis places the interpolated regions in terms of this prevalent pattern – whether they are representative of the Antarctic pole or rather the margin, and how the correlation structure accounts for this.

    • Steve McIntyre
      Posted Feb 21, 2009 at 12:09 AM | Permalink | Reply

      Re: Jesper (#13),

      I’m as interested as the next person in wave patterns and so on. The trouble with Steig’s Mannian covariance factory is that the patterns aren’t actually analyzed. The statements about physical interpretation of the three PCs aren’t supported or proven and, for the PC3, is demonstrably untrue.

      There’s a really good reason for starting with a null of simple spatial decorrelation: in a circular region, there are two equal eigenfunctions. Something like the Transantarctic MTs could plausibly give a structure, but surely the best way to show such a structure is through some sort of proper comparison to base case of no such structure. As I’ll show in a post or two, the Steig eigenvectors have a great deal in common with eigenvectors generated from null spatial decorrelation on a region shaped like Antarctica.

  11. Jeff C.
    Posted Feb 20, 2009 at 10:52 PM | Permalink | Reply

    When I was doing the gridding exercise I was struck by the incongruity of including Campbell and Macquarie Islands in the predictor data set. My atlas says they are both beyond the iceberg limit by 300 miles or so. Campbell is only 400 miles from Southern New Zealand.

    There is another questionable station on South Georgia Island (Grytviken) 1500 miles west of the Falklands. It is within the iceberg limit but outside the limits of pack ice.

  12. Steve McIntyre
    Posted Feb 20, 2009 at 11:15 PM | Permalink | Reply

    So far I’ve not carried out statistical analyses against a null model. For now, I’m just struck at the obvious decorrelation with distance. Maybe there are some bona fide relationships in here. But the obligation to demonstrate that relationships surely is the sort of thing that Steig should have done. For examples of why you should not just throw everything into a covariance factory, google “noisy covariance matrix” for interesting literature. I’ve edited the above text a little to clarify this point.

    After Ryan pointed to the clump of positive correlations around 6000 km, I re-examined the data set and found that the clump of positive long distance correlations involve Campbell and Macquarrie Islands to Grytvyken and the Antarctic Peninsula stations and added a location map to the above post showing these stations. Nearly all the long-distance positive correlations are between Southern Ocean stations, as noted above.

    • Ryan O
      Posted Feb 20, 2009 at 11:22 PM | Permalink | Reply

      Re: Steve McIntyre (#15),

      After Ryan pointed to the clump of positive correlations around 6000 km,

      .
      I think that was bender. ;) I was more interested in the negatively correlated ones, cause visually, there does seem to be a negative correlation between coastal stations on opposite sides of the continent. :)

      • bender
        Posted Feb 21, 2009 at 12:10 AM | Permalink | Reply

        Re: Ryan O (#16),

        there does seem to be a negative correlation between coastal stations on opposite sides of the continent.

        Want to bet that’s “not inconsistent” with predictions from GHG-forced GCMs? ;)

    • Geoff Sherrington
      Posted Feb 24, 2009 at 5:40 AM | Permalink | Reply

      Re: Steve McIntyre (#15),

      Steve, I have seen data only from Macquarie Island and it shows virtually no change in Tmax or Tmin in the last 40 years. So maybe the 6,000 km correlation is for stations with zero character.

      Macquarie Is Excel stats summary 1968-2008.
      Slope is -0.0032 deg/year Tmax. It is 0.0003 deg/year for Tmin, both linear least squares fits.
      Mean 4.918004
      Standard Error 0.054647
      Median 4.885397
      Standard Deviation 0.349909
      Sample Variance 0.122436
      Kurtosis -0.80288
      Skewness 0.146964
      Range 1.354164
      Minimum 4.213233
      Maximum 5.567397
      Sum 201.6382
      Count 41

  13. Hu McCulloch
    Posted Feb 20, 2009 at 11:26 PM | Permalink | Reply

    Antarctica is barely 3000K across at the thinnest, and except for the remotest interior, almost everthing is within 1000K of several stations. It therefore looks like interpolating Antarctic temperature would be a natural candidate for Kriging, a venerable mining tool Steve has discussed here on occasion, that incorporates this assumption about correlations. (I’m working on something else right now, so I’ll leave this to others to try.)

    By 3000-4000 KM, the correlations are all noise centered on zero. The distinctly negative group around 5000KM and the small positive group around 6000KM are quite interesting, but presumably not exploitable for the purpose at hand.

    While it’s possible for the 0-distance correlation to be less than unity (as when two stations, such as Enigma Lake and Terra Nova share essentially the same coordinates, but may have their own uncorrelated micrositing issues) it can’t be greater than unity. The 0-intercept should therefore be estimated subject to this inequality restriction, and the 1.04 Steve obtained by unrestricted correlation superceded by 1.00.

    Steve’s factor of 1000 looks close to optimal, but it would be good to estimate this with some sort of nonlinear model that also takes the unequal errors into account. I’m guessing the factor could be as high as 1500.

    • Steve McIntyre
      Posted Feb 20, 2009 at 11:59 PM | Permalink | Reply

      Re: Hu McCulloch (#17), Hu, I’d done an nls fit and the value is a bit above 1000 km. I was really just using this curve to sort of scale things, as among other things, the curve is too concave to meet a squint test.

  14. Hu McCulloch
    Posted Feb 20, 2009 at 11:42 PM | Permalink | Reply

    Note that 10° of latitude (the rings in Steve’s map) are roughly 1111 km apart, and that Steve’s map does not include the AWS stations, which for all their faults do fill in the gaps to some degree since their establishment.

    • Steve McIntyre
      Posted Feb 20, 2009 at 11:48 PM | Permalink | Reply

      Re: Hu McCulloch (#18),
      The AWS stations as a group had so much missing data that I ended up going to the surface station data. I should note this in the text.

  15. bender
    Posted Feb 20, 2009 at 11:47 PM | Permalink | Reply

    It is worth pointing out the implications (especially to Walt, who objects to Steve’s tone because he doesn’t get the substance of the issue). Using covariance only, RegEM would rather infill from a station 6500km away than a station 2000km away. If the two are both located in the donut of the southern annular mode this makes some sense. But it doesn’t necessarily always make sense. The correlation could be due to a spurious shared trend. (Did the team investigate this? If so, then why is it not in the SI?)
    .
    Leading to the question: what is the source of correlation at 6500km? Is it the early 20th c. shared warming “trend”? Or is it as jesper (and Steig’s abstract!) suggest: internal climatic variability having NOTHING TO DO with GHG forcings and “smoking guns”? (Why did this paper get such high billing and press? The results are anti-alarmist.)

  16. Steve McIntyre
    Posted Feb 20, 2009 at 11:56 PM | Permalink | Reply

    The clump of negative correlations over 5000 km are between Casey, Dumont and Mirny on one side of the continent and the Peninsula on the other. On a quick look, most of the negative correlations involve Peninsula sites. This would seem to be “consistent” with the old-fashioned view of Peninsula warming and continental cooling, tho I’m just experimenting right now.

    • bender
      Posted Feb 20, 2009 at 11:59 PM | Permalink | Reply

      Re: Steve McIntyre (#21),

      This would seem to be “consistent” with the old-fashioned view of Peninsula warming and continental cooling

      Peninsula warming in the 1950s-70s, but not past the 1980s?

  17. bender
    Posted Feb 20, 2009 at 11:57 PM | Permalink | Reply

    The other point, Walt, is that the zero mean correlation at 4000km is comprised of observations with both highly negative and positive correlations. All things being equal (which they aren’t, but anywho …) those positive correlations are likely to be spurious if the true mean correlation at 4000km is in fact 0. So why would you infill from them? To maximize the a priori effect for which you’re seeking confirmation? Ah, that confirmation bias creeping into team work again. That must have been Mann’s contribution.

  18. Jon
    Posted Feb 21, 2009 at 12:01 AM | Permalink | Reply

    The peasants are revolting.

    Steve: call in to quell the uprising.

    • bender
      Posted Feb 21, 2009 at 12:14 AM | Permalink | Reply

      Re: Supergrover (#25),
      Well-mannered sigh. [For Superwalt.]

      • schnoerkelman
        Posted Feb 21, 2009 at 2:57 AM | Permalink | Reply

        Re: bender (#28), I think that should be s/Supergrover/Supergavin/
        That’s regex (for the non-Unix speakers) not RegEM :-)

  19. Jon
    Posted Feb 21, 2009 at 12:18 AM | Permalink | Reply

    @25: Inside baseball, I guess.

    When are you going to “audit” Lomborg, Pielke Jr. and others that are self-described mainstreamers?

    Your prior handwaving is getting ever so weak.

    • bender
      Posted Feb 21, 2009 at 12:22 AM | Permalink | Reply

      Re: Jon (#29), When you audit the team I will audit Pielke Jr? Deal?

    • Posted Feb 24, 2009 at 12:09 AM | Permalink | Reply

      Re: Jon (#29),

      Neither Lomborg nor Pielke Jr. do climate, they do politics and economics.

      It’s Mr. McIntyre’s site, he can work on what he wants. Start a site and audit what you want.

  20. Allen63
    Posted Feb 23, 2009 at 10:16 PM | Permalink | Reply

    As alluded above, maybe the correlations are a function of two variables — latitude and distance — rather than distance alone (thus the improvement at 6000km). Perhaps a two variable correlation analysis? One could try latitude in lieu of distance — just to see what happens before trying a more complex analysis.

  21. david charlton
    Posted Feb 23, 2009 at 10:32 PM | Permalink | Reply

    I would be very interested in the correlation as a function of distance between the non-peninsular coastal stations and the interior. Isn’t that the foundational assertion that needs to be tested for the study to be valuable. Are the coastal stations histories enough like those of the interior to permit interpolation between sites?

  22. page48
    Posted Feb 24, 2009 at 1:32 AM | Permalink | Reply

    You’re back! Yeah!

  23. Geoff Sherrington
    Posted Feb 24, 2009 at 5:00 AM | Permalink | Reply

    On distance correlations with annual temperature averages, here is the main graph used for Australia. Reference “Updating Australia’s high-quality annual temperature dataset”. Della-Marta,Paul, Collins, Dean and Braganza, Karl. Aust. Met. Mag. 53 (2004) 75-93

    There seem to be quantitative differences cf. the Antarctic data. Note that virtually all Australian station pairs are included in the distances shown; and that it is rare for rhe correlation to exceed 0.8.

    • Steve McIntyre
      Posted Feb 24, 2009 at 9:34 AM | Permalink | Reply

      Re: Geoff Sherrington (#38),

      Geoff, that’s a nice graph. I re-did the Antarctic correlation-distance to exclude both the island (5) and northern Peninsula (12) stations, leaving 25 “continental” stations, as shown in the location map below in red (exclusions in blue):


      Location map of surface stations used in correlation-distance plot below. 17 stations are excluded, of which 5 are islands in the Southern Ocean and 12 are in the northern part of the Peninsula.

      Now here is the same scatter plot for correlation vs distance, which perhaps shows a more coherent decorrelation than with the island and far peninsula stations included. The majority of the remaining negative correlations are from two southern Peninsula stations remaining in the network (Rothera, San Martin).


      Scatter Plot of Correlation vs Distance, 25 Stations.

      • Martin Sidey
        Posted Feb 24, 2009 at 1:26 PM | Permalink | Reply

        Re: Steve McIntyre (#42),

        Using the graph of post 42, one can see that the distance between any two stations can be estimated from the correlation between them. Is this the answer to the ciricism that Steig’s method of infilling did not take into account distance but conly covariance. Distance is implicitly contained within the covariance?

        • bender
          Posted Feb 24, 2009 at 1:32 PM | Permalink

          Re: Martin Sidey (#56),
          Read the whole thread. That the two are generally correlated does not mean that the correlation holds steadfast for any given pair. i.e. You can have two stations that appear to be correlated to a much larger degree than their separation distance would indicate. So, hunt around enough and you will find the correlations that you are seeking. That’s confirmationist pseudo-science in action.

        • bender
          Posted Feb 24, 2009 at 1:34 PM | Permalink

          Re: bender (#57),
          Referring to #22 in particular.

        • Chris JH
          Posted Feb 24, 2009 at 5:30 PM | Permalink

          Re: Martin Sidey (#56),

          Using the graph of post 42, one can see that the distance between any two stations can be estimated from the correlation between them. Is this the answer to the ciricism that Steig’s method of infilling did not take into account distance but conly covariance. Distance is implicitly contained within the covariance?

          The locations of the stations is valuable data. Why discard it and hope that your algorithm partially recovers this discarded data?

      • Geoff Sherrington
        Posted Feb 24, 2009 at 7:18 PM | Permalink | Reply

        Re: Steve McIntyre (#42),
        I will do a discourteous thing and put this is upper case:

        THE GISS DATA ARE MANIPULATED AFTER BEING RECEIVED FROM THE SOURCE. THEY ARE PUT IN A MANGLER AND CAN EMERGE WITH LITTLE RESEMBLANCE TO THE ORIGINAL

        I DO NOT TRUST GISS DATA. I DO NOT USE GISS DATA.

        The data I reference come from the manager at Macquarie Island, the Bueau of Meteorology, Australia.

        Here is a graph of their data:

        The BOM data are sold as raw, but they are homogenised. In the Della-Marta 2004 paper of #38, read -

        “Despite the suitability of the dataset for national and regional-scale analyses, any individual station record should still be treated with caution. The subjectivity inherently involved in the homogeneity process means that two different adjustment schemes will not necessarily result in the same homogeneity adjustments being calculated for individual records.”

        The BOM have adjusted the “raw” data and GISS have had a further effort. Where is scientific rigor?

        I have posted elsewhere that my first look at the Dutch KNMI compilation showed errors of 1 deg C over several years. The Dutch responded by saying that NOAA mistakes were known but it was beyond their lifetime work to correct them.

        Re: Steve McIntyre (#42),
        It sure makes a differnce. Thank you for the exercise. My main concern is that I place little emphasis in cases like this on correlations much below 0.8. It’s a matter of what you’ve been used to.

        • Steve McIntyre
          Posted Feb 24, 2009 at 11:04 PM | Permalink

          Re: Geoff Sherrington (#61),

          The Dutch responded by saying that NOAA mistakes were known but it was beyond their lifetime work to correct them.

          Sounds like a job for:

        • Steve McIntyre
          Posted Feb 24, 2009 at 11:08 PM | Permalink

          Re: Geoff Sherrington (#61),

          GISS shows data before 1970. Do you know the provenance of this data?

        • Geoff Sherrington
          Posted Feb 25, 2009 at 12:22 AM | Permalink

          Re: Steve McIntyre (#64),
          The Bureau of Meteorology Australia report data collection at Macquarie Island about 1949 onwards, but there is much missing data. I have done a summary of a number of BOM stations and chose to start all of them in 1968 because of matters like missing data.

          In any case, I have plotted the annual average temps from 1951 onwards as below. There is a quadratic fit imposed, but I have not examined its significance nor do I make an interpretation.

          As for earlier data, I have part of a record of a whaling ship “Clarice” that called into Botany Bay in 1844…….

  24. tty
    Posted Feb 24, 2009 at 8:43 AM | Permalink | Reply

    It is not very surprising with positive correlations between stations in the subantarctic zone. Climate is very homogenous in the “roaring forties” (and fifties), and essentially the weather systems in this zone endlessly circle the Antarctic in an easterly direction.
    Nor is it surprising with a low correlation between these sites and the continent itself, since Antarctica is rather climatically isolated from the rest of the world by the Southern ocean. As a matter of fact it was probably the inception of this isolation which caused Antarctica to freeze over in the early Oligocene about 35 million years ago.

  25. Posted Feb 24, 2009 at 8:48 AM | Permalink | Reply

    Did Steig et al really use places like Grytviken Campbell and Macquarie? From their table S2 it looks like they did. At the risk of stating the obvious, these are islands in the southern oceans that have very little to do with Antarctica – to put it another way, if these are Antarctic stations then I live in the Arctic! So why did they use these stations? Could it be anything to do with the fact that they have positive trends? Has anyone checked what would be the effect if the islands were left out?

    Geoff, what data are you looking at? When I look at the GISS data for Macquarie it clearly shows warming.

    Jon, maybe when Pielke has a paper in Nature making false claims about temperature trends over a period and location where no data exists, with a misleading picture on the front cover, then he might be worth auditing.

    There is a tendency to miss what I think is the key point. The central claim of the paper is “West Antarctic warming exceeds 0.1 °C per decade over the past 50 years”. But there is NO DATA for W Antarctica for this period!

    • Geoff Sherrington
      Posted Feb 25, 2009 at 12:24 AM | Permalink | Reply

      Re: PaulM (#41),

      Is it possible for you to post a similar graph for the GISS data for Macquarie Island? Ref my #65.

  26. Bob North
    Posted Feb 24, 2009 at 9:49 AM | Permalink | Reply

    tty beat me to the punch. The weak to slightly moderate (all less than 0.5) correlations between Macquarie and Campbell and the peninsula and other island stations makes perfect sense given the fact that either oceanic or close to oceanic (in terms of the peninsula) and the weather patterns endlessly circle Antartica in that latitude zone.

    Another question for me is just how much should we be making of correlations that are in the 0.2 to 0.4 range. In my world, that isn’t much of a correlation

    • David Jay
      Posted Feb 24, 2009 at 10:32 AM | Permalink | Reply

      Re: Bob North (#43),
      Well, Bob – That is because this all takes place in the “South” world and you aren’t ;)

  27. Steve McIntyre
    Posted Feb 24, 2009 at 9:53 AM | Permalink | Reply

    Returning to the text that prompted this post. Steig said:

    large spatial scales are not inherently problematic if there is high spatial coherence, as is the case in continental Antarctica

    I do not see how the observed correlations between surface stations demonstrates especially “high spatial coherence” and the addition of island and far peninsula stations to the network doesn’t improve this claim as negative correlations get into the mix.

    Also, I’m getting really tired of arm-waving forms of expression such as “high spatial coherence” appearing in “scientific literature. Like “similar to” or Pielke Jr’s favorite “consistent with”. This is well within the purview of editors and referees. Editors should establish anti-armwaving policies and ask reviewers to consider that policy in their review. That would really save a lot of trouble for everybody.

  28. Posted Feb 24, 2009 at 11:38 AM | Permalink | Reply

    RE Paul M #41,
    Good point, Paul! clearly the following 5 oceanic stations did not belong among the “42 occupied weather stations used as predictor variables” (per SI table S2) in a study of “Warming of the Antarctic ice-sheet surface” (per Steig’s title)!

    Campbell (52S, 169E)
    Grytviken (54.3S, 323.5E)
    Macquarie (54.5S, 158.9E)
    Orcadas (60.7S, 315.3E)
    Signy* (60.7S, 314.4E)

    These points stand out in Steve’s map in #42 above, with Orcadas and Signy appearing as a single blob.

    The * on Signy indicates that it is one of the 15 singled out for the restricted 15-predictor reconstruction, so this restricted recon is contaminated with non-Antarctic data as well.

    I’m not so sure about the “far peninsula” stations Steve excludes from his last diagram in #42 — many of the other “peninsula” stations are in fact islands along the peninsula (as are many of the other coastal stations, I suspect), so if all islands were dropped (except perhaps the biggest ones like Ross), there might not be much left. But clearly the above 5 have no business in this study.

    Perhaps the criterion should have been whether or not the site is really on the “ice sheet” advertised in the paper’s title. I gather that some parts of the peninsula per se, and perhaps even other small areas, are not ice sheet.

    But at a minimum, the study should have been restricted to Antarctica and its adjacent islands.

  29. Posted Feb 24, 2009 at 11:44 AM | Permalink | Reply

    RE #51, if Campbell (52S) is Antarctic, then Amsterdam (52N) is Arctic!

  30. Mark T
    Posted Feb 24, 2009 at 11:50 AM | Permalink | Reply

    Wow, I had not caught (#41) until just now and I must admit, I’m a bit shocked that an island, i.e., a bit of land surrounded by water, would be used in a study about the continent itself. If I’m not mistaken, don’t islands tend to trend with the water that surrounds them? Granted, the similar argument can be made for coasts, but at least a coastal station partially butts up against land and Antarctica does not possess many interior weather stations.

    Mark

  31. Posted Feb 24, 2009 at 8:14 PM | Permalink | Reply

    The correlaton vs distance plot was very intriguing so I did the same spatial correlation as above, using the same code modified for the AWS RegEM data. The correlation vs distance was seriously degraded in the AWS reconstrucion.

    http://noconsensus.wordpress.com/2009/02/25/correlation-of-reconstructions/

  32. Posted Feb 25, 2009 at 10:38 AM | Permalink | Reply

    Geoff, I tried to reply but got stuck by a filter, perhaps for links and images. You can find the plot and data on the gisstemp station data website and check it against yours.

    • Geoff Sherrington
      Posted Feb 26, 2009 at 7:44 PM | Permalink | Reply

      Re: PaulM (#68),

      Re Macquarie Island.

      Here is the GISS version for average annual temp, after application of GISS corrections.

      What corrections are needed? It’s about as rural as you can get: it’s close to sea level: it has no other nearby relveant stations to corrupt its pure data; to my knowledge, it has has half-hourly automatic temp recording for some years; no lights. I can’t see ANY corections are needed.

      Giss shows about half a degree rise from 1951-2008. The Australia data show far less than this. (Data before 1951 are really full of missing values).

      Calling NASA Goddard – what did you do to the data from BOM?

      • bender
        Posted Feb 27, 2009 at 12:50 AM | Permalink | Reply

        Re: Geoff Sherrington (#74),

        Giss shows about half a degree rise from 1951-2008. The Australia data show far less than this.

        Would you be willing to fit a quadratic to each series and report the statistics?

        • bender
          Posted Feb 27, 2009 at 12:53 AM | Permalink

          Re: bender (#76),
          Or post the data and I’ll post an R script to do the analysis.

        • Geoff Sherrington
          Posted Feb 27, 2009 at 3:03 AM | Permalink

          Re: bender (#77),

          Don’t have the digital data for GISS. As explained elsewhere, there is some infilling in my data that negates sophisticated analysis.

          The need for infilling can be shown for some of the early years I rejected:

          YEAR MONTH DAY Tmax Tmin
          1949 11 6 5.6 1.7
          1949 11 7 3.9 0.6
          1949 11 8
          1949 11 9
          1949 11 10
          1949 11 11
          1949 11 12
          1949 11 13
          1949 11 14
          1949 11 15
          1949 11 16
          1949 11 17
          1949 11 18
          1949 11 19
          1949 11 20
          1949 11 21
          1949 11 22
          1949 11 23
          1949 11 24
          1949 11 25
          1949 11 26
          1949 11 27
          1949 11 28
          1949 11 29
          1949 11 30 7.8 4.4
          1949 12 1
          1949 12 2
          1949 12 3
          1949 12 4
          1949 12 5
          1949 12 6
          1949 12 7
          1949 12 8
          1949 12 9
          1949 12 10
          1949 12 11
          1949 12 12
          1949 12 13 7.8 4.4
          1949 12 14 6.7 3.9
          1949 12 15 6.7 2.8
          1949 12 16
          1949 12 17
          1949 12 18 8.9 2.8
          1949 12 19
          1949 12 20
          1949 12 21
          1949 12 22
          1949 12 23
          1949 12 24
          1949 12 25
          1949 12 26
          1949 12 27
          1949 12 28
          1949 12 29
          1949 12 30
          1949 12 31

        • Geoff Sherrington
          Posted Feb 27, 2009 at 5:11 PM | Permalink

          Re: bender (#77),

          In other words, I think that the inclusion of months of mostly missing data at non-randon times has depressed the early years of GISS and given a false impression of a following increase. (Remember December is summer here). That is part of why I started my studies in 1968. But, as well, there is a difference point-to point in later years between BOM and GISS that is probably not from missing data, especially post 1995 (AWS). I’d like GISS to explain how these changes arose as I can see no pressing reason to change the BOM data. I tend towards indiscriminate, overall use of the mangler.

  33. Hu McCulloch
    Posted Feb 25, 2009 at 10:40 AM | Permalink | Reply

    RE Jeff ID, #62,
    Thanks for the plots! However, these are for the RegEM reconstructions of the AWS data, not for the AWS data itself. What do the actual AWS correlations look like?

    Of course, since AWS is often sparse, many pairs won’t be calculable, and those that are won’t have many points. Still, it would be useful to see this. Perhaps pairs that have a reasonable number of points could be plotted with a different symbol than the others.

    The fact that each month has its own mean in the raw data further complicates the comparison. Perhaps the correlation could be computed for each month for which 3 or more pairs exist, and then the monthly correlations averaged, weighted by their number of observations. (Given that there are only n-2 DOF in a correlation, it would be meaningless to compute one with only 2 data pairs.)

  34. Hu McCulloch
    Posted Feb 25, 2009 at 10:46 AM | Permalink | Reply

    RE #69, or better yet, weight the monthly correlations by the number of DOF, n-2, so as to give monthly pairs with only 3 observations a weight of 1 rather than 3. Then give station pairs different symbols depending on their total weight counts.

  35. Hu McCulloch
    Posted Feb 25, 2009 at 10:59 AM | Permalink | Reply

    RE Steve #42,

    It might might be informative to compute manned station correlations by distance separately for Antarctic island pairs, continental pairs, and island/continental pairs. I suspect that island stations pick up a lot of correlation from ocean currents that would not affect continental pairs.

    (This would exclude, of course, the 5 temperate oceanic stations in #51 that didn’t belong in this study, or at least put them in a separate category.)

    RE #69, the same might be done for island vs continental AWS stations, but the data there is already so sparse this might not be informative.

  36. Posted Feb 25, 2009 at 11:03 AM | Permalink | Reply

    Let’s try that again – operator error.

    Re: Hu McCulloch (#69),

    I didn’t do the AWS correlation by itself because of the limited data. You are right though, it would be interesting to see, I’ll try to do something later.

    If it wasn’t clear, my point is that the non-reconstructed correlation of surface stations as plotted by SteveM shows reasonable de-correlation with distance. There is enough data that the RegEM versions of the surface station retain similar ‘appearing’ distance correlation to the original with a few outliers. The AWS stations after RegEM on the other hand don’t have enough data to retain a realistic correlation with distance resulting in what looks to me like a substantially blended trend.

  37. Alan Wilkinson
    Posted Feb 26, 2009 at 9:17 PM | Permalink | Reply

    Is there some reason to use an exponential decay for correlation over distance rather than the common inverse square relationship for flux dispersions?

  38. bender
    Posted Feb 20, 2009 at 9:40 PM | Permalink | Reply

    Re: Walt Bennett (#2),

    Sigh.

    Sigh.

One Trackback

  1. By Kriging on a Geoid « Climate Audit on Aug 26, 2010 at 1:24 PM

    [...] on CA. See, for starters, “Toeplitz Matrices and the Stahle Treering Network”, 3/22/08, “Antarctic Spatial Autocorrelation #1″, 2/20/09, “Steig Eigenvectors and Chladni Patterns”, and follow-up [...]

Post a Comment

Required fields are marked *

*
*

Follow

Get every new post delivered to your Inbox.

Join 2,898 other followers

%d bloggers like this: