I just learned (too late) about an interesting workshop sponsored by the American Statistical Association this weekend in Boulder.
The announcement last summer stated:
The American Statistical Association (ASA), the nation’s preeminent professional statistical society, today announced it will sponsor a two-day climate change workshop featuring 2025 leading statisticians and atmospheric scientists. The event, sponsored by the ASA’s Science and Public Affairs Advisory Committee (SPA) and the ASA Section on Statistics and the Environment (ENVR), will take place October 26-27 at the National Center for Atmospheric Research in Boulder, Colorado. David Marker, SPA chair, and Mary Christman, ENVR chair, will facilitate the workshop.
…
Understanding climate change requires the combined skills of atmospheric scientists and statisticians, Marker said. The former understand the physical relationships being investigated, while the latter know how to determine which hypotheses are strongly supported and which are still subject to uncertainty. By bringing together researchers from these two communities, we can identify where there is consensus and where future research needs to focus.
Caspar Ammann was invited, but not me. On the list were Ed Wegman, Gerry North, Doug Nychka.
I wasn’t previously familiar with the ASA section on Statistics and the Environment, but I plan to make some efforts in this direction. Their newsletter from earlier this year had a very interesting account by Richard Smith of North Carolina of a packed session at the 2006 ASA meeting discussing statistics and climate change, to which Wegman, Mike Wallace of the NAS panel and Smith himself spoke. Given the ignoring of Wegman by the climate change community, it’s interesting to read an account of the matter from an eminent statistician, who obviously could not ignore someoone of Wegman’s eminence. It’s also (and unusually) a balanced account which catches many, but not all of the nuances. Smith introduced the session as follows:
What is the Role of Statistics in Public Policy Debates about Climate Change? that was organized jointly by Edward Wegman (George Mason University) and myself at the 2006 Joint Statistical Meetings. The session took place in front of a standing-room-only audience and was chaired by Doug Nychka (National Center for Atmospheric Research).
Smith observed:
At the core of the controversy is an incorrect use by Mann et al. of principal components (PCs).
Note that there is no nuance here – Smith agrees with Wegman that the Mann et al method was incorrect. He then considered the argument that the error doesn’t not “matter” together with Wegman’s rebuttal:
A number of other commentators have acknowledged the flaws in the Mann reconstruction but have argued that this does not matter because the answers have been verified by other analyses. Ed’s own response to that was given in the equation:
Method Wrong + Answer Correct = Bad Science.
In other words, the fact that the answer may have been correct does not justify the use of an incorrect method in the first place.
Both Wegman’s talk and Smith’s account of it correctly noted that the issues with Mann et al were not just principal components, observing almost but not quite accurately:
Ed also touched on some of the other controversies in Mann’s work. Some of the proxies had been criticized as inappropriate. For example, bristlecone pines are known to be CO2 fertilized, creating a possible confounding problem if they are used in temperature reconstructionA figure from Mann’s own website suggested that the medieval warm period reappeared if bristlecone pines were excluded from the reconstruction. Other studies had shown a discomforting array of different results in the reconstructions obtained with minor methodological variations.
I presume that what was meant here is that Mann’s CENSORED directory (which we deduced contained calculations without bristlecones) did not have a HS shape. Based on our recent sampling in Colorado, it appears that the bristlecone problem may relate more directly to the strip bark phenomenon, as opposed to CO2 fertilization. In our discussion of bristlecones, while we noted that CO2 fertilization had been raised as an issue, we noted that the real issue was the non-robustness to a proxy known to be problematic – and that any use of this proxy as a worldwide thermometer should be preceded by a concerted effort to know everything possible about bristlecone pine growth. Smith then quickly reported on Wallace’s talk. On other occasions, I’ve quoted North’s answer to a question from Barton in which he stated that he agreed with the Wegman report. Wallace said the same thing:
In Mike’s view, the two reports were complementary, and to the extent that they overlapped, the conclusions were quite consistent.
The language in Wegman was much more straightforward, but Wallace’s statement, together with North’s statement, demonstrate that the NAS panel did not disagree with any of Wegman’s clearly stated findings. Wallace also said:
The NRC report reviewed a number of other reconstructions of the temperature record based on proxy observations and believed that the Mann et al. claim that the last two decades were the warmest of the last 1000 years was entirely plausible.
As I’ve observed elsewhere, the “review” carried out by the NRC panel does not seem to have constituted anything more substantive than simply reviewing the literature one more time. The “other” series that they used in their own spaghetti graph all used strip bark bristlecones/foxtails (Mann and Jones 2003; Hegerl et al 2006; Moberg et al 2005; and Esper et al 2002) – see below.
NRC Panel Figure O-5 (C).
It seemed bizarre to me at the time that the panel could recommend that strip bark trees be “avoided” in reconstructions and then use as evidence “supporting” the Mann result reconstructions that used strip bark trees. Worse, two of the 4 studies illustrated here (Mann and Jones 2003; Hegerl et al 2006) actually use the Mann PC1 that had been specifically rejected as incorrect methodology. You have to probe beneath the surface of Hegerl et al to determine that their “W USA” series was really Mann’s PC1, but it was. It still seems bizarre to me that the NRC panel could be so indifferent to performing any due diligence, given their charge. North said in a seminar that they just “winged it”, explaining that’s what you did in these panels. Given the fact that NRC reports are accorded great weight in legal proceedings, it would certainly be more re-assuring if NAS panel chairs at least gave lip service to due diligence requirements, and, even more reassuring, if they actually performed the due diligence that the public presumes that they did.
Smith observed:
while there is undoubtedly scope for statisticians to play a larger role in paleoclimate research, the large investment of time needed to become familiar with the scientific background is likely to deter most statisticians from entering this field.
In the end, it’s important not to lose sight of the forest for the trees, where the forest refers to the totality of scientific evidence for global warming.
As someone who’s actually made the “large investment of time” to become intimately familiar with all the proxy issues, you’d think that they might have invited me to this workshop. On the other hand, I’m not a member of the ASA, but it’s probably something that I should belong to. I’ll make an effort to introduce myself to the chairs of the workshop by email and see what happens.
As to the last sentence, I agree that it’s important not to lose sight of the forest for the trees. As a reviewer for AR4, it was my position that, if the paleoclimate issues were not relevant to the policy issues, then the Paleoclimate (and the hockey stick discussion) should be deleted from AR4 so that people could focus on what were the “real” arguments. The IPCC “consensus” was presumably that the paleoclimate arguments remained important and that’s why the chapter remained, despite my suggestions that it be deleted.
He concluded with the announcement of the workshop just held as follows:
Second, there will be an ASA workshop of invited participants whose purpose is to establish A Statistical Consensus on Global Warming, organized by Dr. David Marker, Chair of the ASA Science and Public Affairs Advisory Committee, and Dr. Mary Christman, Chair of the ASA Section on Statistics and the Environment, with the sponsorship of the ASA Board and the co-sponsorship of the Section on Statistics and the Environment. This workshop is planned for the fall of 2007 and should deliver its report by early 2008.
Given that the Richard Smith article specifically cites McIntyre and McKitrick in connection with this topic, it does seem peculiar that neither Ross McKitrick nor myself were invited.
