Wegman observed last summer that climate scientists failed to involve statisticians to an appropriate degree in the work. Yesterday Simon Tett drew our attention to Brohan et al 2006 as an explanation of uncertainties in HadCRU3. Brohan et al, of which Tett is a coauthor, used the prominent statistician, Donald Rumsfeld, as an authority for their uncertainty model. Brohan et al:
A definitive assessment of uncertainties is impossible, because it is always possible that some unknown error has contaminated the data, and no quantitative allowance can be made for such unknowns. There are, however, several known limitations in the data, and estimates of the likely effects of these limitations can be made [Rumsfeld, 2004].
Rumsfeld, 2004 has the following reference (to a 2002 press conference):
The Acronym Institute. Disarmament documentation. Back to disarmament documentation, June 2002. Defense secretary Rumsfeld press conference, June 6. “Secretary of Defense Donald H. Rumsfeld, press conference at NATO headquarters, Brussels, Belgium, June 6, 2002,” US Department of Defense transcript. http://www.acronym.org.uk/docs/0206/doc04.htm
The salient quote appears to be:
Question: Regarding terrorism and weapons of mass destruction, you said something to the effect that the real situation is worse than the facts show. I wonder if you could tell us what is worse than is generally understood.
Rumsfeld: Sure. All of us in this business read intelligence information. And we read it daily and we think about it and it becomes, in our minds, essentially what exists. And that’s wrong. It is not what exists. I say that because I have had experiences where I have gone back and done a great deal of work and analysis on intelligence information and looked at important countries, target countries, looked at important subject matters with respect to those target countries and asked, probed deeper and deeper and kept probing until I found out what it is we knew, and when we learned it, and when it actually had existed. And I found that, not to my surprise, but I think anytime you look at it that way what you find is that there are very important pieces of intelligence information that countries, that spend a lot of money, and a lot of time with a lot of wonderful people trying to learn more about what’s going in the world, did not know some significant event for two years after it happened, for four years after it happened, for six years after it happened, in some cases 11 and 12 and 13 years after it happened. Now what is the message there?
The message is that there are no “knowns.” There are things we know that we know. There are known unknowns. That is to say there are things that we now know we don’t know. But there are also unknown unknowns. There are things we don’t know we don’t know. So when we do the best we can and we pull all this information together, and we then say well that’s basically what we see as the situation, that is really only the known knowns and the known unknowns. And each year, we discover a few more of those unknown unknowns. It sounds like a riddle. It isn’t a riddle. It is a very serious, important matter. There’s another way to phrase that and that is that the absence of evidence is not evidence of absence. It is basically saying the same thing in a different way. Simply because you do not have evidence that something exists does not mean that you have evidence that it doesn’t exist. And yet almost always, when we make our threat assessments, when we look at the world, we end up basing it on the first two pieces of that puzzle, rather than all three.
Now there is considerable practical sense in what Rumsfeld says here, but I don’t see that these comments justify Brohan (Tett) et al applying these comments as authority for the policy:
There are several known limitations in the data, and estimates of the likely effects of these limitations can be made [Rumsfeld, 2004].
Also, merely as a bit of practical advice to the authors of Brohan et al 2006, despite the renown of Rumsfeld as a statistician, there is a distinct possibility that some, if not most, readers may think that the “estimations of the limitations in the data” were not accurately made in this particular case. It’s a very odd citation. One wonders if anyone actually reads these papers/