My notes on Luterbacher and Hegerl are not very good. Hegerl, in particular, was very difficult to follow and it would have been hard for the panel to assimilate. She made one nice observation – someone asked her about confidence intervals with low correlations. She said that they would be from the floor to the ceiling. Keep that in mind as we consider verification r2.
Luterbacher
Luterbacher presented the results of Luterbacher et al [2004] on European climate history from AD1500 on, which was based on documentary evidence. He started with some examples of what he meant by documentary evidence — e.g. the date of cherry blossoms. Records go back to 1700 in Europe and 2000 years in Japan and Korea. Another type is the # of times Venice lagoon froze over — where he showed an interesting graphic.
He showed results from 1500-2000, saying that comparable results were “not possible” prior to 1500. For earlier results, he mentioned a new tree ring study by BàÆà⻮tgen; a study by van Engeln from 1169-2004 and presented a graphic showing the advance/recession of Great Aletsch Glacier. The graphic had some question marks for past glacier recessions; Turekian asked him about these and for details of how the Aletsch Glacier study was constructed, how they quantified recessions; Luterbacher said that he “couldn’t give details” as someone else had done the glacier study, perhaps Holzhauser.
Luterbacher described their method briefly: PCA in the calibration period, calculation of transfer functions; assumed stationarity and an “independent” verification period. Despite parallels to Mann’s methodology, no one asked him about principal components or other aspects of his methodology.
Luterbacher showed a spaghetti graph and raised the issue of inconsistencies between them, proposing the following possible explanations: 1) differences between summer and annual target; 2) uncertainties; 3) differences in solar forcing in different regions. He acknowledged that the proxy records were strongly oriented towards summer temperature (where the change in the instrumental period was less than for winter temperatures.) In my opinion, none of the explanations were strongly argued (see my AGU PPT), but no one asked him about this.
No one asked Luterbacher (pace Boehlert) whether his data was available for replication — it isn’t; Luterbacher published in Science, has not archived his data and Science hasn’t taken any steps about it, although it has been brought to their attention.
As a side note, Luterbacher is cited by Mann at realclimate as “proof” that climatologists “prefer” the RE statistic. In fact, as far as I can tell, he is the only “authority” that Mann has been able to locate for this preference other than Mann’s own articles. (Last year, Mann attributed this to Cook et al [2004] as well, but, after I pointed out that Cook reported the r2 statistic as well, which was favourable in Cook’s case, Mann dropped Cook et al 2004 as authority and limited himself to Luterbacher.)
JàÆà⻲g Luterbacher, Daniel Dietrich, Elena Xoplaki, Martin Grosjean, Heinz Wanner, 2004 , European Seasonal and Annual Temperature Variability, Trends, and Extremes Since 1500, Science 303, 1499 –1503; DOI: 10.1126/science.1093877
Hegerl
My notes for Hegerl are even worse. She seemed very nice, but I dozed off a little during her presentation — just pacing myself. She mentioned someone named “Tom” a lot.
She presented the results of Hegerl et al [2006 submitted] which is a new reconstruction using 12 sites, apparently picked by this Tom fellow. I don’t know what the sites are; she didn’t mention them. I had previously tried to learn what the sites were as an IPCC reviewer, but was told that that was none of my business and that IPCC reviewers were not expected to check studies against data or to request data from authors. (I requested the information both from IPCC secretariat and later from Hegerl, and was told that if I made any further attempt to obtain data from authors, I would be expelled as an IPCC reviewer.) I mentioned this refusal (and a like refusal by D’Arrigo) in my presentation. Hegerl objected, saying that the paper was under review by Nature, implying that showing the site locations to an IPCC reviewer would be a breach of Nature embargo. (My retort to that was: make a choice, if you can’t supply the data to IPCC as part of its process, then withdraw the paper from use by IPCC.)
She presented a “new” multivariate method. I doubt that her presentation was intelligible to anyone without access to the article (which I’ve consulted for this report.) She cited (and Ross noticed this in one of her slides) an 1878 statistics reference for their “new” multivariate method (Adcock, R. J. A problem in least squares. The Analyst 5, 53-54 (1878))
The “new” method seems to have two main aspects: 1) weighting of proxies according to their correlation to temperature; 2) re-scaling of the proxy index by “total least squares”, relying on Adcock [1878], which was applied previously by coauthors Allen and Stott [2003]. I’m pretty sure that the first aspect actually embeds corresponding linear algebra to MBH98, which we mentioned in our handout.
She was asked about confidence intervals under low correlation (I think that was the condition) and she said that such confidence intervals would be from the floor to the ceiling.
Allen, M. R. & Stott, P.A. 2003. Estimating signal amplitudes in optimal fingerprinting, Part I: Theory. Clim. Dyn. 21, 477-491.
Hegerl, Gabrielle C. , Thomas J. Crowley, Myles R. Allen , William T. Hyde, Henry N. Pollack, Jason E. Smerdon & Eduardo Zorita. [2006 submitted]. A new 1500 yr climate reconstruction: enhanced lowfrequency variability and the fingerprint of anthropogenic warming
5 Comments
Looks like you are allowed to listen, applaud politely, but do not ask any questions.
Any responses from the Panel to your comments?
What Hegerl calls “Total Least Squares” is just the Errors-in-variables model. It’s covered in most undergrad econometrics texts. In Kmenta, for instance (1986 edition) the problem is explained and the various methods for handling it are discussed.
The assumption of the model is that the independent variables (on the right hand side) contain additive errors, in which case OLS estimates are inconsistent. OLS minimizes the vertical distances from observations to the regression line on the assumption that all the errors are additive on the dependent variable. If a portion of the error term is due to errors in independent variables, the angle of the line connecting the observation to the regression line needs to be adjusted prior to minimizing the sum of squared distances. Varying it from 90 degrees to 0 is equivalent to attributing the fraction of error variance attributed to the independent variable from 0 to 100%. A simple treatment of the problem is to estimate the slope coefficients under the full range of variance attribution, and if the coefficient doesn’t change then E-I-V isn’t likely a problem. But this is not a formal solution.
The approach used in Hegerl et al. assumes the variance of the error on the right hand side is known from other sources. Thus, having assumed the existence of the problem, it is then conveniently assumed away.
The formal solution to the problem is to use an Instrumental Variables estimator, where “Instrumental” is used in the econometrics sense, not as a fancy term for “thermometer”. IV estimation involves projecting the independent variables onto exogenous regressors that are asymptotically uncorrelated with the measurement error but asymptotically correlated with the regressor itself, then using the projection on the right hand side of the regression. If the instrument set is constructed to satisfy the asymptotic conditions, the least squares slope coefficients will be consistent.
I got the impression from the meeting that the “Total Least Squares” estimator, ca. 1878, is the method used in “signal detection” regressions that figure so prominently in the IPCC reports.
For those who have not been following the tales of hockeydom: ‘someone named “Tom”‘ just might be Hegerl’s husband, Thomas Crowley.
http://home.casema.nl/errenwijlens/co2/errenvsluterbacher.htm
My take on luterbacher is here, The dutch and the swiss temperatures from historical sources are a good predictor for central europe, no need to add poor quality proxies.
Esper:
One Trackback
[…] brought this refusal up at the NAS Panel discussed here. Hegerl explained to the NAS Panel the paper was under review by Nature, implying that showing the […]