The National Academy of Sciences (NAS) has presumably been criticized in the past for the composition of panels (from the evidence of the mere existence of the 1997 law on committee balance and composition). This law and resulting policies provide for a comment period on proposed committees. Ross and I have exercised our rights under this policy and today sent the following letter to NAS.
We are writing to protest three of the appointments to the Panel because of bias, lack of objectivity and/or conflict of interest and to protest the failure of the Panel as presently constituted to meet policies of the National Academy of Sciences (NAS) regarding committee composition and balance. We have suggested several alternatives whose appointment would at least partly mitigate these problems.
The “Policy on Committee Composition and Balance and Conflicts of Interest for Committees Used in the Development of Reports”, a policy statement of the National Academy of Science (NAS) issued in compliance with section 15 of the federal Advisory Committee Act, provides explicit statements about the issues of bias, lack of objectivity and conflict of interest. It states, with respect to conflict of interest:
It is essential that the work of committees of the institution used in the development of reports not be compromised by any significant conflict of interest. For this purpose, the term "conflict of interest" means any financial or other interest which conflicts with the service of the individual because it (1) could significantly impair the individual’s objectivity or (2) could create an unfair competitive advantage for any person or organization. Except for those situations in which the institution determines that a conflict of interest is unavoidable and promptly and publicly discloses the conflict of interest, no individual can be appointed to serve (or continue to serve) on a committee of the institution used in the development of reports if the individual has a conflict of interest that is relevant to the functions to be performed. [bold in original]
and, with respect to bias and lack of objectivity:
Finally, it is essential that the work of committees that are used by the institution in the development of reports not be compromised by issues of bias and lack of objectivity. … Questions of lack of objectivity and bias ordinarily relate to views stated or positions taken that are largely intellectually motivated or that arise from the close identification or association of an individual with a particular point of view or the positions or perspectives of a particular group
The Panel is obviously going to have to consider our various criticisms of Mann et al. and will undoubtedly hear reference to a national Media Advisory by UCAR in May 2005 declaring that UCAR employee Caspar Ammann had shown that our various criticisms were “unfounded”. This press release has been relied upon in material presented to the U.S. Congress by Sir John Houghton of IPCC, by Dr Mann and by the European Geophysical Union. Ammann has advised one of us that he has used these two unpublished articles in his annual employment review at UCAR.
One of the proposed panellists, Dr Otto-Bliesner, has not only been a frequent coauthor and presenter with Ammann, but is Ammann’s immediate supervisor at UCAR (see http://www.cgd.ucar.edu/ccr/paleo/images/Bette1.jpg). As such, she has presumably considered Ammann’s articles on our work in the course of carrying out Ammann’s annual review. We presume that she would have been involved in preparing and/or approving the UCAR press release on Ammann’s work last May. In addition, last year, she co-authored an article with Bradley (of Mann, Bradley and Hughes) and served on a committee with him. It appears to us that her association with Ammann rises to a conflict of interest within NAS policy, but, in the alternative, her associations with Ammann and Bradley certainly rise to bias and lack of objectivity. While she is undoubtedly a meritorious person, the field of candidates is not so limited that her participation in the panel is necessary to its functioning and indeed her continued participation might well diminish the actual and/or perceived ability of the panel to provide objective advice. For example, *** would be an equally competent alternate without the accompanying problems of bias, lack of objectivity and conflict of interest.
Another proposed panellist, Dr Nychka, also a UCAR employee, is listed at Ammann’s webpage as presently collaborating not only with Ammann, but with Mann (see http://www.assessment.ucar.edu/paleo/past_stationarity.html). This ongoing collaboration certainly creates the appearance of a “close identification or association of an individual with a particular point of view or the positions or perspectives of a particular group”. Again, while Nychka is undoubtedly a meritorious person, the field of candidates is not so limited that he is irreplaceable on the panel and indeed his continued participation might well diminish both the actual ability and the perceived ability of the panel to provide objective advice.
We are also concerned about apparent bias and lack of objectivity in a third proposed panellist, Dr Cuffey, who in a newspaper op-ed recently wrote:
Mounting evidence has forced an end to any serious scientific debate on whether humans are causing global warming. This is an event of historical significance, but one obscured from public view by the arcane technical literature and the noise generated by perpetual partisans.
(see http://www.sfgate.com/cgi-bin/article.cgi?file=/chronicle/archive/2005/10/09/ING5FF2U031.DTL&type=printable )
The panel is being asked to consider the “historical significance” of present climate change. A panellist who has a priori dismissed questions on the matter, some of which are necessarily quite technical, as being “arcane” and “noise generated by perpetual partisans” can be “reasonably perceived to be unwilling, to consider other perspectives or relevant evidence to the contrary” as defined in NAS policy.
Lack of Appropriate Expertise on Proposed Panel
NAS policies require NAS committees to achieve standards of composition and balance. The brochure “Ensuring Independent, Objective Advice” advertises that NAS committees provide”
“An appropriate range of expertise for the task. The committee must include experts with the specific expertise and experience needed to address the study’s statement of task. One of the strengths of the National Academies is the tradition of bringing together recognized experts from diverse disciplines and backgrounds who might not otherwise collaborate. These diverse groups are encouraged to conceive new ways of thinking about a problem.”
The NAS policy statement “Policy On Committee Composition And Balance And Conflicts Of Interest” states:
For example, if a particular study requires the expertise of microbiologists, epidemiologists, statistical experts, and others with broader public health expertise, the significant omission of any required discipline from the committee might seriously compromise the quality of the committee’s analysis and judgments, even though it is clear to all that the committee is composed of highly qualified and distinguished individuals. Even within a particular discipline, there may be very important differences and distinctions within the field, or regarding the particular subject matter to be addressed, that require careful consideration in the committee composition and appointment process….
In our opinion, the committee as presently composed fails to comply with this policy on several counts:
1. Without implying that any of the panellists are not “qualified and distinguished individuals” within the meaning of NAS policy, to our knowledge, none of the panellists would be regarded as experts in assessing statistical significance in multivariate models using highly autocorrelated time series, a central topics in the debate. While the panellists have all published articles that pertain to, or use, climate statistics, the issues currently being disputed call for specialist input. NAS policy requires attention to “important differences and distinctions within the field”. We suggest that *** or *** would be qualified candidates in this respect.
2. To our knowledge, none of the panellists would be regarded as experts in the area of replication policy. The entire topic of replicability has been one of the most prominent aspects of disputes surrounding millennial paleoclimate studies. Indeed, it was only after Dr Mann was quoted on the front page of the Wall Street Journal as saying that he would not be “intimidated” into disclosing his algorithm that millennial reconstructions attracted the interest of the House Energy and Commerce Committee and, subsequently, the House Science Committee and National Academy of Science. Expertise in this area requires familiarity with journal policies, statistical methods, software evaluation, and the current literature on replication experiments. We suggest that *** would be a qualified candidate in this respect.
3. The issue of disclosure adequacy and possible omission of material results has also been one of the most prominent aspects of the debate. Last summer, the House Energy and Commerce Committee sent questions to Drs Mann, Bradley and Hughes regarding the omission of material results, such as the cross-validation R2 statistic and the impact of bristlecone pines. The President of the National Academy of Science wrote to the House Energy and Commerce Committee stating that a congressional committee was an inappropriate forum for the investigation of such matters and that a NAS expert panel would be more appropriate. We presume that the present panel has been composed at least in part in response to this initiative by the President of NAS. However, the panel as presently composed lacks any obvious expertise in this area. We suggest that NAS consider one or more of the members of the commission chaired by Kenneth Ryan on Integrity and Misconduct in Research as panellists.
We further refer to the following NAS policy:
A balance of perspectives. Having the right expertise is not sufficient for success. It is also essential to evaluate the overall composition of the committee in terms of different experiences and perspectives. The goal is to ensure that the relevant points of view are, in the National Academies’ judgment, reasonably balanced so that the committee can carry out its charge objectively and credibly.
For some studies, for example, it may be important to have an "industrial" perspective or an "environmental" perspective. This is not because such individuals are "representatives" of industrial or environmental interests, because no one is appointed by the institution to a study committee to represent a particular point of view or special interest. Rather it is because such individuals, through their particular knowledge and experience, are often vital to achieving an informed, comprehensive, and authoritative understanding and analysis of the specific problems and potential solutions to be considered by the committee.
Aside from the particular expertise of *** and ***, our own criticisms of paleoclimate practices and policies are very much influenced by our own experiences in handling economic and business data. Analysis of time series data is a common issue for economics and paleoclimatology. Many issues studied by econometricians are highly pertinent to paleoclimate applications and yet come from points of view that are different, and different in ways that the panel will find constructive to consider. In our view, econometrics has superior methodologies to paleoclimatology in addressing problems of spurious inference and data mining in the presence of strong autocorrelation and integrated processes. Paleoclimatologists, including even some of the panelists, have applied some econometric methods, but that is no substitute for the “point of view” or for up-to-date specialization.