Mann 2008 – Replication II

Let’s continue Mann 2008 – Replication with EIV. To run regreclow.m and regrechigh.m you’ll need files climate, eofnumb, and proxyN, where N runs from 1 to 19. I’ve run prepinputforrecon.m with required folder structure (C:\holocene\s1\zuz10\work1\temann\ etc.) in my computer. After that I did run regrechigh.m in folder

C:\holocene\s1\zuz10\work1\temann\zzrecon1209\nhnhscrihad\highf

and regreclow.m in folder

C:\holocene\s1\zuz10\work1\temann\zzrecon1209\nhnhscrihad\lowf

The regem options in high-version are

OPTIONS.regress = ‘ttls';
OPTIONS.regpar = eofnumb(istep)
OPTIONS.stagtol = 5e-2;
OPTIONS.maxit = 30;
OPTIONS.inflation = 1;
OPTIONS.disp = 1;
OPTIONS.relvar_res = 0.05;
OPTIONS.minvarfrac=0.85;

and in the low-version

OPTIONS.regress = ‘ttls';
OPTIONS.regpar = eofnumb(istep)
OPTIONS.stagtol = 5e-3;
OPTIONS.maxit = 100;
OPTIONS.inflation = 1;
OPTIONS.disp = 1;
OPTIONS.relvar_res = 0.05;
OPTIONS.minvarfrac=0.85;

Here maxit is 100, so this one takes about 10 minutes with my oldish laptop. Will try to find a way to upload the results here (two 3 MB zip files). High-frequency recon19 shows that I wasn’t completely lost with this comment suggesting RegEM is ICE-like calibration method :

Jean S figured out how the results (reconN -files) are related with archived nhnhscrihad_smxx (screened, NH, iHAD ):

  1. For each step, compute  recon=high_recon + low_recon.
  2. Proxies are standardized earlier in the process (high+low unity variance and zero mean in the calibration period). One needs to take std and mean of the target series, iHAD_NH_reform, and rescale the recon, recon= recon*sigma+mu
  3. There is a warning about over-fitting in the paper, so do this only for the steps N=11..19
  4. Combine the results of each step

Note that the number of proxies in highf and lowf inputs is not necessarily the same. Here’s the result and the archived nhnhscrihad_smxx:

As Jean S implied here , after 10 minutes of regem running, all you get is a linear combination of the proxies.

MAKEPROXY.m has lines

%@@@@@@@@@@@@@@@@@@@@@@@@
hhhigh=0; % 0 means screening on raw data  (old)
% 1 means screening on high-f data (new)
%@@@@@@@@@@@@@@@@@@@@@@@@@

so I tried the option hhhigh=1 as well. The results will be zipped here soon, don’t worry. Now I’ll just show the low-proxies(19) with high-f screening:

Hmmm. Well, there are only 6 proxies in this step, and the hockey stick is hiding. But Mann SI says:

As a further safeguard against potentially nonrobust results, a minimum of seven predictors in a given hemisphere was required in implementing the EIV procedure.

..so you’ll have to ignore this high-pass-correlation -screened proxy set ;)

Update (UC):


I added proxy names (hopefully correctly)  to the figures Jean S refers below, there’s something I don’t quite understand. See the AD500 step,

In the AD600 step, fisher-proxy enters the game, note how much the reconstruction changes:

Some 0.6 C change due to one added proxy. Weight of the curtis-proxy increases quite a lot, and there are many sign changes.

20 Comments

  1. Jean S
    Posted Mar 18, 2009 at 4:23 PM | Permalink

    all you get is a linear combination of the proxies

    Yes, and since the combination is the same for all time instances in each step, the weigth vector can be recovered even without running RegEM. Now the interesting part is that signs of the weights seem to vary from step to step as shown here.

    The claim that ‘‘upside down’ data were used is bizarre.

    MBH: Reply to McIntyre and McKitrick: Proxy-based temperature reconstructions are robust, Proc. Natl. Acad. Sci. 106, E11, 2009.

  2. Mark T
    Posted Mar 18, 2009 at 4:29 PM | Permalink

    It’s clear they either do not know what their algorithm is doing, or, well… you get the picture. You’d think after repeatedly being beaten over the head with their errors, they’d get better at making sure there were none.

    Good work, guys.

    Mark

    • Jean S
      Posted Mar 18, 2009 at 4:35 PM | Permalink

      Re: Mark T (#2),
      OT: UC informed me that you had finished your “project” last spring. Congratulations!

      • Mark T
        Posted Mar 18, 2009 at 5:10 PM | Permalink

        Re: Jean S (#3), Yes, I did. Thank you. It was, as I told UC, beyond painful. ICA, btw, with a PCA-like pre-processor! :)

        I’m actually toying with the idea of getting involved with this stuff again now that I’m not overwhelmed to the point of insanity. I need to brush up on some of the RegEM stuff that we discussed in the past. At one point I actually had myself convinced I knew what it was doing, hehe.

        Mark

    • Jean S
      Posted Mar 18, 2009 at 4:49 PM | Permalink

      Re: Mark T (#2),

      It’s clear they either do not know what their algorithm is doing, or, well…

      Here’s a prime example: eofnumb, which is estimated via PCA-type of analysis in the code. It controls the regpar -parameter in RegEM-TTLS, i.e., the rank of the linear combination “matrix” for the imputed values. However, now the target (GL/NH/SH temperature) is one dimensional (and there is nothing to infill in proxies), so it is completely meaningless! Maybe they didn’t realize that, or maybe, just maybe, they first tried “climate field” type of approach (ala Steig et al), but for some reason abandoned that but left the estimation of regpar parameter intact. ;)

      • Posted Mar 19, 2009 at 1:41 AM | Permalink

        Re: Jean S (#4),

        Maybe they didn’t realize that, or maybe, just maybe, they first tried “climate field” type of approach (ala Steig et al), but for some reason abandoned that but left the estimation of regpar parameter intact. ;)

        Mann08:

        Previous tests with synthetic proxy climate data derived from long model simulations demonstrate that this EIV procedure yields a very similar hemispheric mean reconstruction to that obtained from the application of the associated spatially-explicit CFR procedure when an estimate of only a single time series, e.g., the hemispheric mean temperature, is sought (32).

        ref 32: Mann ME, Rutherford S,Wahl E,Ammann C(2007) Robustness of proxy-based climate field reconstruction methods. J Geophys Res 112:D12109.

  3. Andrew
    Posted Mar 18, 2009 at 6:35 PM | Permalink

    Clearly the the dark blue series represents the dominant mode of climate variability. You just need to weight it 1000 times more than everything else. I mean, its got to pass verification, right? ;)

  4. Steve McIntyre
    Posted Mar 18, 2009 at 7:06 PM | Permalink

    Great stuff,. Look forward to getting back. Cheers, Steve

  5. Posted Mar 18, 2009 at 10:28 PM | Permalink

    You’d think after repeatedly being beaten over the head with their errors, they’d get better at making sure there were none.

    Problem is, they don’t recognize the errors as errors. We are the ones making the errors by pointing them out, remember.

  6. steven mosher
    Posted Mar 19, 2009 at 11:44 AM | Permalink

    hehe UC wrote…

    “To run regreclow.m ” ……….

    I ran regreclown.m

    haha. didn’t compute. cli mutt science

  7. JohnT
    Posted Mar 19, 2009 at 5:05 PM | Permalink

    I wonder how many of Mann’s peers secretly read CA, and look at threads like this and say “Yep, Mann is wrong. How could he have made such a huge mistake. Seems like he is losing his MoJo”.

    Mann’s statistical skills appear to be “nonrobust” compared to JeanS’s.

    Are thoughts like this being circulated in the climate community?

    • Michael Jankowski
      Posted Mar 19, 2009 at 5:24 PM | Permalink

      Re: JohnT (#12), it seems as if their heads are in the sand. He’s only one of many climate scientists who do not get statisticians involved in publications that are heavily dependent on statistical methods.

    • Mark T
      Posted Mar 19, 2009 at 10:33 PM | Permalink

      Re: JohnT (#12),

      “Yep, Mann is wrong. How could he have made such a huge mistake. Seems like he is losing his MoJo”

      Given that this started with Mann’s very first (or nearly) paper, I’m guessing none can really say he’s “losing” anything. Of course, this also assumes said peers understand the distinctions as well as Jean S or UC themselves.

      Are thoughts like this being circulated in the climate community?

      Not openly, at least, when was the last time you heard a headline in the MSM stating “statistical conclusions are shown to be unsupported in XXX09″ or similar?

      Mark

  8. andy
    Posted Mar 19, 2009 at 11:50 PM | Permalink

    Which proxy is this, AD600 step-in, with all upsidedown side effects?

    • Jean S
      Posted Mar 20, 2009 at 1:37 AM | Permalink

      Re: andy (#15),
      if you mean the one labeled “fisher_1996_cgreenland”, it is supposedly this one:

      http://www.gfy.ku.dk/~www-glac/papers/abstracts/183.htm

      It gets weights (AD600) -0.2079 (for the low split; shown above) and -0.1775 (for the high split). It is given ID 8000, which means that in order to pass the screening, it should have positive (pick-two) correlation with the local temperature. Indeed, it has, according to this file the reported (pick-two) correlations are
      full screening: 0.3311
      late: 0.3742
      early: 0.4976

      This amazing change between AD500 and AD600 (compare the lower right corners in UC’s update figures) in the reconstruction due to adding a single series is also clearly visible in the final reconstruction (see the second figure in this post). I guess this is what they mean by “robust”. ;)

      Notice also that there are FOUR Tiljander series (one switching sign between AD500 and AD600) out of total 12/13 proxies. I guess it must be “bizarre” to wonder how “robust” the reconstruction is for removing those series… ;)

    • Jean S
      Posted Mar 20, 2009 at 1:56 AM | Permalink

      Re: andy (#15),
      Oh, if you meant the one with the label “curtis_1996_d13cpyro” (which seems to be dominant in AD600 step), that’s one of Punta Laguna series (C13.P_coronatus) discussed already here in the context of infilling.

  9. andy
    Posted Mar 20, 2009 at 5:35 AM | Permalink

    Thanks, seems that at least some Fisher series have been discussed in CA as well; http://www.climateaudit.org/?p=883

  10. David L. Hagen
    Posted Mar 20, 2009 at 7:59 AM | Permalink

    For Freedom of Information Act (FOIA) requests for code and data, see new guidelines by Attorney General Holder:

    Attorney General Issues New FOIA Guidelines to Favor Disclosure and Transparency

    http://www.usdoj.gov/opa/pr/2009/March/09-ag-253.html

    Memorandum for Heads of Executive Departments and Agencies Re The Freedom of Information Act (FOIA)

    http://www.usdoj.gov/ag/foia-memo-march2009.pdf

  11. Posted Mar 20, 2009 at 12:39 PM | Permalink

    I guess Steve will be soon around, so I can email him the zip-files to upload. But for now I uploaded the steps we found most bizarre:

    Target temperature

    AD500 proxies, in the following order

    nv512
    tiljander_2003_darksum
    tiljander_2003_lightsum
    tiljander_2003_thicknessmm
    tiljander_2003_xraydenseave
    tornetrask
    tan_2003_recontemp
    curtis_1996_d13cpyro
    curtis_1996_d18o
    burns_2003_socotrad13c
    burns_2003_socotrad18o
    dongge

    AD500 reconstruction

    AD600 proxies, in the following order

    fisher_1996_cgreenland
    nv512
    tiljander_2003_darksum
    tiljander_2003_lightsum
    tiljander_2003_thicknessmm
    tiljander_2003_xraydenseave
    tornetrask
    tan_2003_recontemp
    curtis_1996_d13cpyro
    curtis_1996_d18o
    burns_2003_socotrad13c
    burns_2003_socotrad18o
    dongge

    AD600 reconstruction

    Proxies and reconstructions are as regreclow.m writes them.

One Trackback

  1. By Notes on RegEM « Climate Audit on Oct 16, 2011 at 10:08 PM

    [...] a period with some interesting properties previously observed some time ago by Jean S and UC here here, but not explained at the [...]

Follow

Get every new post delivered to your Inbox.

Join 3,203 other followers

%d bloggers like this: