As the date approached for the Mann-Steyn/Simberg libel trial, I’ve been reviewing my files on MBH98 and MBH99. It’s about 15 years since I last looked at these issues.
While revisiting these issues, I re-examined some data 
associated with the notorious “dirty laundry” Climategate email (link) – excerpt shown at right – that turns out to provide a glimpse of the long obfuscated results for AD1000, AD1400 and AD1600 steps over the 1902-1980 period. I don’t recall this being noticed at the time. Even by Jean S or UC. The identification proved interesting.
Attached are the calibration residual series for experiments based on available networks back to: AD 1000, AD 1400, AD 1600. I can’t find the one for the network back to 1820! But basically, you’ll see that the residuals are pretty red for the first 2 cases, and then not significantly red for the 3rd case–its even a bit better for the AD 1700 and 1820 cases, but I can’t seem to dig them up. In any case, the incremental changes are modest after 1600–its pretty clear that key predictors drop out before AD 1600, hence the redness of the residuals, and the notably larger uncertainties farther back… You only want to look at the first column (year) and second column (residual) of the files. I can’t even remember what the other columns are! Let me know if that helps.
The data referred to in this email were located in the Climategate I directory mbh98-osborn.zip as nh-ad1400-resid.dat etc and were discussed in a draft Osborn memorandum Mann uncertainty.doc. The columns are unlabelled. The second column is residuals. The AD1400 example is shown below:
An aside about Mann’s favorite “RE” statistic. While Mann (and Wahl-Ammann) hyperventilated about the supposedly unique validation imparted by an RE statistic, the RE statistic is extremely sensitive to choice of calibration and verification period – an issue that was never addressed by Mann or Wahl-Ammann. So, in the above example, if the calibration period were set at 1920-1960 and verification period at 1961-1980, the RE fails miserably. If the RE statistic is so extremely sensitive to the choice of calibration and verification periods, it entails that the underlying reconstruction does NOT possess the claimed “robustness” or “skill”.
Continued here.
Climate Audit 
18 Comments
Year 14 anniversary coming up. Maybe the 15th anniversary can inspire a major look back and update.
Steve:
Good work! I just wanted to let you know that there are some of us out here who value the work you’ve done over the years and now recently, in regard to Mann’s transgressions.
Donald Rapp http://www.drdrapp.com http://www.drdrapp.com/
>
As always, Steve, thanks for your keen eye and your bloodhound instincts. Most interesting and very valuable.
w.
Ditto, Willis.
I miss these posts a great deal.
Adding my comment to the rest of the chorus. I very much appreciate your efforts to understand and show the details of how the sausage was made!
To Jeff Alberts comment: Agreed! I miss the period of the early days of ClimateAudit when the posts and comments were happening multiple times per day multiple days per week.
Bruce
A curious detail in these files is that the “sparse” instrumental data in column 4, unlike the reconstruction in column 3, has nonzero mean. This has an unexpected explanation which I found through trial and error. In short, the “sparse” data was centered using grid box means calculated from the “dense” data, which was slightly corrupted by the inversion of all anomalies below -10 degrees Celsius. So -9 was unchanged while -11 became +11 etc, but only in the dense data.
!! That is a seriously hard-to-find mis-step but all too characteristic of Mann’s opus.
How did you find this out?
I implemented the data processing algorithm as described in the corrigendum SI (had to guess some details) and compared my grid box standard deviations with those archived. About 10% or so were irreproducible, and plotting some data I noticed that they corresponded precisely to those grid boxes that had at least one anomaly below -10 degrees. Then it was just a matter of figuring out what happened to these outliers. Frustrating but doable.
not on the scale of your amazing deconstruction of “signal free” algorithm, but very nice.
This was more of an Easter egg. 😉
Bravo hs! That is quite a find!
Bruce
Thanks. 🙂
I’ll bet that the error is due to a Fortran program reading fixed widths. In which only two characters were read. So the sign of -11 got lost.
That’s my theory, too. I notified Mann who confirmed the error but I didn’t ask him how it happened.
The MBH98 archive contains a Fortran program getdata.txt that reads anomalies-new (correctly) into an array DATA, but the array is only 72 cells, so it’s overwritten once per month and latitude. Then it just terminates.
here’s a puzzle for you. Many years ago, we (Jean S, UC and I) noticed that Mann’s list of proxies in the AD1400 network seemed to be missing a series. One could closely reverse engineer the first 50 years of his reconstruction with reported AD1400 proxies, but it should have been possible to do so exactly. It looks to me like there is one other series in the calculation, but not listed in the roster. It’s no big deal, but a strange puzzle. Maybe the NOAMER PC3?
I ran UC’s excellent script a couple of years ago and noticed this. Same with AD 1450 and an undocumented AD 1650 step.
The correct AD 1400 and 1450 proxy networks may be extracted by regressing the reconstruction on the full set of proxies (also available in Osborn’s archive) and checking for nonzero coefficients. The AD 1650 network is too large for this strategy (underdetermined system), but it was possible to work out by other means.
I put together a partial emulation and uploaded it to GitHub. Here is the corrected list of proxies for AD 1400: https://github.com/klimatsmart/mbh98emu/blob/main/mbh98emu/config/proxy/datalist1400.dat
PC3 is in there…
The Dirty Laundry dataset for AD1600 contains 1902-1980 values for AD1600 reconstruction. These could be additional datapoints.
3 Trackbacks
[…] « Mann’s “Dirty Laundry” […]
[…] Hockeystick-Jäger Steve McIntyre auf Climate Audit: […]
[…] few days ago, I posted a thread on MBH98 residuals (link) in which I observed that several datasets connected with notorious Dirty Laundry email contained […]