Sometimes the Hockey Team baffles me. I was at Esper’s website and noticed Esper J, Neuwirth B, Treydte K (2001) A new parameter to evaluate temporal signal strength of tree ring chronologies. Dendrochronologia 19, 93-102. Something that looked pertinent. As a component of his estimate of signal strength, Esper applied a statistical procedure Gleichlàƒ⣵figkeit G which disregarded the number of cores in the sample in a calculation of signal strength. In an important example, the Esper G yields diametrically opposite results to results where the number of cores are considered.
The article itself is here. The Gleichlàƒ⣵figkeit G statistic in year t is defined as
G[t] = max {Nplus[t]/N[t], Nminus[t]/N[t]}, where Nplus is the number of tree rings with an increase in width; Nminus the number of tree rings with a decrease in width; N the number of tree rings in year t.
Dendrochronologists pay a lot of attention to "pointer" years, especially years with very narrow rings and use those to align the series. In a narrow pointer year, most of the cores will have a declining ring width and G will be near 100%. Remarkably Esper’s G (which seems to be traditional) is not adjusted for the number of cores. Thus, 23 of 23 cores changing in the same direction receives the same G as 2 out of 2 cores changing the direction. This seems incomprehensible, but it is so.
It’s hard to figure out what they wouldn’t adjust G for the number of samples (or "studentize" G, borrowing the term used the t-statistic). I tried the following definition for a "studentized G" statistic:
the proportion of cases in a 50-50 binomial draw of length N which are less informative than n out of N.
For example, if there is a 50-50 split between + and -, the studentized G would be 0 since you get at least that good a match 100% of the time. If you get 2 out of 2 going the same way, the studentized G is 0.5, since you get a less "informative" result (1 up, 1 down) 50% of the time. On the other hand, if you have 23 of 23 cores going the same way, the studentized G is nearly 1. This is a trivial modification but obviously a far more sensible measure of signal strength.
I tested this procedure on the Polar Urals data set (which I’ve written about before) with interesting results, as shown in Figure 1 below.
Figure 1. Polar Urals. Top – Gleichlàƒ⣵figkeit G; bottom – "studentized" G. Red – rolling 25-year gaussian smooth.
In other posts, I have questioned the crossdating and reliability of the early portion of the Polar Urals data set. Opposite stories are given by the Esper G and the studentized G: the Esper G indicates very high signal reliability in the early portion, while a studentized G shows extremely poor signal reliability in the early portion. The studentized G provided a much better estimate of signal reliability. It is hard to think of a good reason for using the Esper G. In fact, if the Esper G (which has been used in this form before Esper) had bee used by (say) Schweingruber on the Polar Urals, this could have contributed to a false sense of confidence in the early portion of the series,
I note in passing an interesting comment in Esper et al. [2001] about age adjustment. He mentions the following about the Karakorum junipers, which are very long-lived:
The series were standardized using negative exponential curves. These growth curve models are useful to estimate age-related trends in ring width series. The raw ring width series (Fig. 2a) clearly demonstrate that only the Alp data set contains age-related trends. The slow growing Juniper trees show no such long-term trends in the raw data (details in Esper 2000; Esper et al. 2000). Since the chosen standardization technique is adapted to data sets containing age-trends, the changes caused by standardization vary strongly between Alp and Kak.
If there are no age-related trends in the junipers, one wonders why they would be adjusted by applying an age-related negative exponential model. Surely this will introduce a modern upward bias (of the type noted by Cook and Peters.) I suspect that these junipers will have some properties in common with bristlecones – they are both high-altitude, very long-lived, arid species.
To my knowledge, the Esper data used in this study have not been publicly archived, but are probably at the S.O.& P site, password protected because of national security implications. (I’m not aware of Esper ever archiving any tree ring measurement data at WDCP.)
Climate Audit 
22 Comments
I wonder if this is a case of data being “processed” by statistical packages as a result of academic boredom?
Louis, I don’t think so. The G statistic is not in a canned package and wuold have to be programmed in. In fact, if it were a canned package, I doubt that any self-respecting writer of statistical software would not adjust for the number of samples. As long as the number of cores doesn’t change, this formula is a decent rule of thumb. I can see how it could develop in a pre-computer world by tree ring guys. The problem only comes when they try to stretch these chronologies to their limit with too few samples.
Steve, it seems that most of your criticism of MBH relates to misuse of statistics or claims of higher signinficance than justified. You seem to be coming to the table with heavy stats knowledge albiet some time away from academia. Mann seems to be coming to the table with middling stats knowledge, deeper experience in the field/literature, and with recent time in academia. How would a full-up, practicing, stats guru (but an academic) review Mann’s work and your criticisms of it? ONe from outside the climatology field, but with impeccable academic credentials and independance from the current spat?
TCO, my guess is that any unbiased top-notch expert would be even more scathing than me. It’s hard to explain why these sorts of people haven’t been involved to date. If I had a big policy job (putting a management perspective on things) and had to get an answer on this, I’d look around for exactly that type of guy, retain him for a couple of months, put him in touch with the protagonists and ask him to report on the situation.
I find it truly remarkable that no organization has bothered trying to resolve the issues. If this were an issue for Boeing or Pfizer about (say) O-rings or drug effectiveness statistics, they would tear the thing inside out and get to a definitive answer.
A
1. Big companies can suck too.
2. HAve you tried getting in touch with your (I know anonymous) reviewers who liked your critiques? Were they stats heavy guys or paleoclimate heavy guys?
In this day in age where it seem “the more co-authors, the merrier,” there seems little excuse for not having a statistician involved (at least as an auditor).
Mr Barton, are you listening ?
Re: #6 What? And rock the enormous gravy train?
They’re not rewarded for being statistically and scientifically thorough, they’re rewarded for bolstering a pre-determined “scientific consensus” through the mechanism of terminally weak “peer review”.
It’s up to the funding authority to appoint an independent audit of the work, then if the analysis is worthless (statistically or otherwise), the scientists get denied the opportunity to publish or receive further funding – its as simple as that. Exactly what happens in stock market offerings (except the money at stake is so much less) and private placement memoranda.
I think coming in with that kind of contentious attitude is not helpful, JohnA.
Main Entry: con·ten·tious
Pronunciation: k&n-‘ten(t)-sh&s
Function: adjective
1 : likely to cause contention
2 : exhibiting an often perverse and wearisome tendency to quarrels and disputes
synonym see BELLIGERENT
– con·ten·tious·ly adverb
– con·ten·tious·ness noun
says the troll…
source:
http://www.m-w.com/cgi-bin/dictionary?book=Dictionary&va=contentious&x=24&y=15
this is quite an important comment, because it reflects on the way science is done; it is not the responsibility of an organisation, but of individuals. No matter how good any individual, sometimes they do not realise the limits of their competence.
Which is fine in an abstract scientific field; given a couple of years, new individuals will come in with better training, realise the defects in the existing area, and make new science by pointing out the deficiencies in the old. It is clearly not so good when you are trying to make legislation from science which is incompletely understood.
I will also point out that organisations are not a panacea for correcting all evils ! The two examples cited, the pharmaceutical industry and aerospace industry, have had many decades to work out what the best practice for their industry is. They bring to the table the experience arising from many decades of getting their fingers burnt ! Part of the difficulty with the global warming issue is that it is so new, and there is very little organisation-style experience of this science.
cheers
per
Good insights, per.
hans, agreed, I’m an awful troll and you haven’t seen me at my worst.
all: I’m spending too much time here. Will be shifting to checking in once a week on weekend and commenting at that time. (note to self, try to be disciplined!!!)
TCO, don’t stay away too long, you’ve been pleasant company.
Re 3, 4 and 11: During the past three years I and a co-author (David Henderson, former Head of the Department of Economics and Statistics at OECD) have criticised the IPCC’s treatment of economic issues. Our main single criticism has been the Panel’s use of exchange rate converters to put the GDPs of different countries onto a common basis for purposes of estimating and projecting output, income, energy intensity, etc. This is not permissible under the internationally-agreed System of National Accounts which was unanimously approved by the UN Statistical Commission in 1993, and published later that year by the United Nations, the World Bank, the IMF, the OECD and the Commission of the European Communities, under cover of a Foreword which was personally signed by the Heads of the five organisations.
The practice of exchange rate conversion has been explicitly rejected by at least three Nobel Laureates in economics (Sir Richard Stone, Paul Samuelson and Amartya Sen) and three Distinguished Fellows of the American Economic Association (Irving Kravis, Robert Summers and Alan Heston). In the course of the current controversy, Sir Partha Dasgupta of Cambridge University has told Stephen Schneider of Stanford University, a leading IPCC figure, that “Castles is of course completely right” (in rejecting the use of “outmoded accounting practices”); William Nordhaus of Yale University advised an IPCC Expert Meeting on Emissions Scenarios last January that estimates of output or income using exchange rates are “simply wrong”, “constructed on an economically unsound basis”, “fundamentally wrong”, “highly misleading” and “precisely wrong”; Richard Tol of Hamburg University informed the recent UK House of Lords Committee inquiry into “The Economics of Climate Change” that the IPCC scenarios “essentially assume convergence based on market exchange rates, which is ludicrous”; Ross McKitrick of Guelph University drew the Committee’s attention to a statement by John Reilly of MIT that the IPCC scenarios exercise was “in my view, a kind of insult to science”; and the world’s leading expert on historical international comparisons of output and income, Angus Maddison, gave the Committee “an illustration of the implausibility of using exchange rate converters in historical analysis or futurology (as in the IPCC Special Report on Emissions Scenarios …)” In its unanimous report, the 13-member Select Committee on Economic Affairs of the House of Lords said that “We found no support for the use of MER in [long-term emissions scenarios], other than from Dr Nakicenovic of the IPCC.”
None of this criticism has moved the IPCC. Its Chairman, Dr Rajendra Pachauri of India, told the House of Lords Committee that the criticism of the emissions scenarios “only validates the methodology that the IPCC used earlier” and “does not invalidate it”. A press statement published on the IPCC website which is devoted specifically and exclusively to brushing aside the Castles and Henderson critique says that the IPCC “mobilises the best experts from all over the world”, and describes us as “so called ‘two independent commentators'”. The IPCC has selected Professor Nakicenovic and Brian Fisher, Director of the Australian Bureau of Agricultural and Resource Economics (ABARE), as the two Coordinating Lead Authors of the chapter in the Panel’s next assessment report (AR4) that is to assess criticism of the IPCC Special Report on Emissions Scenarios (SRES). In their first response to the Castles and Henderson critique, Professor Nakicenovic and 14 other lead authors of the SRES said that “Mr Castles and Mr Henderson have focused (at tedious length) on constructing a ‘problem’ that does not exist”, and in their “final” response Nakicenovic and 17 other lead authors persisted in affirming the “methodological soundness of the use of MER for developing long term emissions scenarios”. Brian Fisher, the other CLA of the chapter that is to assess the IPCC scenarios, is the co-author of an article published by ABARE (of which he is Director) which states that “The use of [market exchange rates] … in the SRES remains valid and the critique by Castles and Henderson cannot be sustained.”
Australia’s leading scientific research organisation, the CSIRO, appended an opinion piece by atmospheric scientist Kevin Hennessy to its submission to an Australian Senate Committee Inquiry into the Kyoto Protocol Ratification Bill 2003. According to Mr Hennessy,”Castles and Henderson have claimed that the IPCC warming projections are based on greenhouse emissions that are too high because market exchange rates (MER) were used rather than purchasing power parity (PPP) in calculating future economic growth. The claims have been reviewed and refuted by international experts (Nakicenovic and others).” The list of references to the opinion piece did not include any paper by “Nakicenovic and others”. Mr Hennessy subsequently declined an invitation to produce a paper on the CSIRO’s emissions scenarios for Energy & Environment (the journal in which Nakicenovic et al appeared), on the grounds that the CSIRO prefers to publish in peer-reviewed journals listed by the Institute for Scintific Information (ISI). Mr Hennessy has been selected as a Coordinating Lead Author of the “Australia and New Zealand” chapter of the next IPCC Report.
At the IPCC Expert Meetings on Emissions Scenarios last January, Professor John Weyant of Stanford University, Director of the Energy Modeling Forum, defended the use of exchange rate converters by the IPCC on the ground that “best practice can differ between making historical welfare comparisons and model projections of GDP, energy and carbon emissions.” According to the Report of the meeting “Weyant recommended using MERs or PPPs consistently.” It is of course a contradiction in terms to urge the use of MERs “consistently”.In a letter to Dr Pachauri three years ago I pointed out that an expert committee appointed by the UN Statistical Commission had found that there were “material errors” (that is, errors which left the reader with ‘a fundamentally distorted view of the phenomena being described’) in the UNDP’s Human Development Report 1999. I noted that the same statements had been repeated uncritically in IPCC reports. Two of these were in a chapter of the last assessment report of which Professor Weyant was Coordinating Lead Author. The IPCC has not acknowledged that any mistakes were made in the last assessment report and has selected Weyant as Review Editor of the Chapter in the next assessment report which is to review criticisms of the IPCC emissions scenarios.
Another Lead Author of the next IPCC Assessment Report whose country of residence appears on the IPCC lists as “Australia” is Bill Hare, who was one of the invited experts to the recent IPCC Workshop on Emissions Scenarios (as a representative of “Greenpeace, Environmental NGO”). Mr Hare argued for a strong role for the IPCC in the development of scenarios in the future, and asserted that the SRES had been a “big success”.
In my first letter to the Chairman of the IPCC three years ago, I said that it would “be desirable to seek the involvement of national statistical offices and of the International Statistical Institute (ISI) in the new emissions projections that I understand are to be prepared for the IPCC’s Fourth Assessment Report.” The IPCC subsequently decided that there would be no emissions projections. It has not invited any representatives of national statistical offices or of the ISI to any of its expert meetings, nor have any national accounts experts been included in the writing teams for the next assessment report.
The IPCC can ignore the world’s leading economists and statisticians with impunity, because it has the support of “the worldwide scientific community”. In its submission to the House of Lords Committee, the Royal Society (UK) explained that:
“The work of the IPCC is backed by the worldwide scientific community. A joint statement of support was issued in May 2001 by the science academies of Australia, Belgium, Brazil, Canada, the Caribbean, China, France, Germany, India, Indonesia, Ireland, Italy, Malaysia, New Zealand, Sweden and the UK. It stated: ‘We recognise the IPCC as the world’s most reliable source of information on climate change and its causes, and we endorse its methods of achieving consensus.”
The joint statement of support appeared in “Science”, the journal of the American Association for the Advancement of Science. In a news item published in the same issue of “Science”, the President of the Royal Society, Lord May, explained that the Royal Society had organised the petition because of resistance to the terms of the Kyoto Protocol by countries such as the US and Australia.
My comment relates to #3, #4 and #12 (not #11).
Statisticians and economists are outside the scientific community? That explains a great deal.
We can thank the Royal Society for clarifying things.
All kidding aside, we can thank Ian Castles for clarifying some things!
If I understand the IPCC’s definition of consensus, it is consensus by exclusion. The definition is
also a method. Both have the approval of the Royal Society? Isn’t the practice of science lovely.
Over long time, is it really relevant if one uses PPP or exchange rates?
Shouldn’t the economics drive them to the same place?
I remember reading some other paper by Esper that was thoughtful and even mildly “anti-GW” on the specific issue. Do you really need to go after each one of these guys like this, Steve? I’m on your side and all, but wonder if you could get more accomplished if trying the polite private notification first. cf ON SCIENTIFIC RESEARCH (very good book).
Sorry, TCO, but you’re wrong again in #19. Please see the references in my comment on your posting on the “Ian Castles…” site. Also relevant in this case is a paper presented to a Stanford Energy Modeling Workshop in February 2004 by Professor Alan Heston of the University of Philadelphia, Distinguished Fellow of the American Economic Association and Chairman of the Technical Advisory Group to the International Comparisons Project being managed by the World Bank. Heston said “… these [exchange rate-based] models are based on the assumption that prices and incomes in the future will converge to a world of one price, an assumption for which the past 50 years provides no support. But even accepting that assumption, MERs should converge to PPPs, and PPP based incomes to PPP based incomes, which in that never-never world will also be MER based incomes. Manne and Richels [the economists cited by the IPCC] and others [including the IPCC modellers], however, do it backwards starting with MER numbers.”
Well if they converge given enough time isn’t it better to use market values? I mean the market cap of Exxon is what it is worth. A squash is worth what someone would pay for it. Even if that gyrates maddeningly. P.s. reading your ref now.