I know this is well after the event, but Dr Wegman’s extensive and detailed responses to Rep Stupak‘s written questions are interesting reading. I’m not aware that Steve linked to this document before.
Beware though: the file is 10MB in size, so I’d recommend a right click followed by “Save Target as” – otherwise your browser will lock up waiting on the University of Guelph’s strangely low bandwidth Internet connection.
As if to demonstrate the “state of fear” in climate science – here is how some of the referees to Wegman’s Report reacted (page 8):
In addition, we had two other reviewers who asked that their
names not be revealed because of potential negative consequences
for them.
Climate Audit 
27 Comments
Just getting through it… but one thing I noticed that was very very interesting was the social network graphs. I think “team” is the right term for the current climate science “community.”
That bit was surprising as well. The comparison between Mann’s and Wegman’s social networks could not be more stark.
They must have upgraded their server, it took me less than 20 seconds to download the file.
Here are those pictures:
Dr Wegman –
Dr Mann –
Mathematically a fully connected set like that is called a “clique”
Wegman has something for everyone involved in statistics (my emphasis). Page 14:
One word.. “Brutal.”
When was this published/posted?
#5. I also thought that Wegman’s reponses were very apt. The bolded comment is an excellent quote. It’s a pretty obvious methodological point that climate scientists seem unaware of.
The usual suspects were all wound up about the use of social networks. Implication being, it’s something that “right wingers” of “the corporate-political establishment” would tend to favor. I’ve got news for them however – this approach was actually initially pioneered by groups such as the (anti CIA) Christic Institute during the 1970s. (Google that! …. )
re: 5 & 7
Statistically, it’s clear that mining of both data and methods is expressly verboten, duh; unfortunately, Mr Stupak may be too blindly protective (“obtuse” in Dr Motl’s parlance) of MBH’9x and AGW to realize that.
Wegman’s responses are both very powerful and very courteous.
Roger Bell
#4 While I agree that Dr. Mann appears to be far more ‘clique’ oriented than Dr Weber, could part of that be due to tthe relatively smaller set of associates between the two?
Reading this and the interplay between the AGW ‘believers’ and their reaction to anything that is perceived to be at odds with thier belief leads me to suggest that they are in classic ‘group think’ mode.
I think Wegman is being too kind here. It is not a matter of “enough principal components”, rather than a matter of precisely the right number of principal components.
I think that if you start adding more PCs than the critical number, then the hockey stick will start to dissolve again.
#12.
fFreddy, no you’re incorrect here. The bristlecones are primarily weighted in the covariance PC4. If this one series enters into the 2nd stage of the Mannomatic – the partial least squares regression – it will imprint the HS on the final series.
#4. I think that a more appropriate network analysis would have been based on the authorships of the various “independent” studies: Jones [Briffa et al], Briffa [Jones Schweingruber et al], Rutherford [MAnn Bradley Hughes Osborn Briffa Jones], etc. Also Bradley and Jones who go way back together and were really stood at the center of the Team in 1998.
Steve, what happens if you feed, say, 6 PCs into the Mannomatic ?
The paper analyzing Prof Wegman’s network describes him as a big boss – which he’s probably proud about 😉 – but the graphs of Mann and Wegman indeed look very different.
#15. IF the PC4 is in the regression network, you get a HS; if it isn’t, you don’t. It’s simply another way of saying: no bristlecones in the MBH network, no HS. That’s not to say that you can’t make a selection of proxies to get a HS without bristlecones – although there are very few such examples. But Briffa 2000 is one such example – a small network, where Yamal has a huge impact.
So an 8 PC regression looks the same as a 4 PC regression ?
Wow.
Dr. Wegman is an erudite spokesman for the measured response to the political movement of “Big Al”. We need more “reasonable men (people)” but, unfortunately, fame wins. Is there a “former next President of the United States” or a Cameron Diaz look-alike out there who has any actual knowledge regarding scientific process (or anything else) who is willing to stand up and become the next spokesperson for Climate Sanity? I think Steve’s comment regarding the 80% of political expenditures going to 30 second sound bites is all to telling.
I thought it was interesting how strongly and obviously he was being attacked through the questions. Obvious prodding for weaknesses — like a lawyer querying a witness.
How did those in the strong AGW camp but not immediately in Mann’s “clique” respond to Wegman? It seems to me
that they basically ignored both the original report of his committee and this testimony.
As #21 asks…I have seen hatchet jobs done on Steve and others coming from AGW in forums. That is how I found this site and the Wegman article. Have the AGW camp done their usual, attack their opponents (Wegman et al.) rather than answer the questions? If so could someone provide a link. The Wegman, especially the full version with all the different clique/network graphs, was astounding. I particularly laughed when he poked Mann by reminding the committee that their work was essentially “pro bono” and that they had real jobs that they had to keep.
“Democrats’ frustration at the hearing’s exclusive focus on two papers was apparent, as they stressed that the well-established scientific consensus on the reality of anthropocentric global warming would remain unaltered even if Mann had never written the papers in question. Rep. Bart Stupak (D-MI) exclaimed he was “stupefied” at the narrow scope of the hearings, and said that Congress was “particularly ill-suited to decide scientific debates.” Rep. Jay Inslee (D-WA) called the hearing “an exercise in doubt.” ” The materials given to Stupak Committee really underscore this quote from “Science and Technology Newsletter – July 2006”. I wonder what planet these people live on? Do they not realize how intricately man’s population increases and economies are linked to fuel use? This is old science established by the 1970’s by such scientists as Eugene Odum, and such new (at that time) fields such as Geographical Ecology: Patterns in the Distribution of Species.
Is there a good link where someone examines the costs of GW and compare it to economic loss from CO2 emission reductions? I was reading the IPCC summaries, but without cost basis, etc., it led me more to think it was an incredible waste of time rather than it served any purpose.
John,
For a strong critique of Wegman, you should go on Mann & co’s Realclimate.org
I remember they dismissed Wegman’s network analysis as UNscientific, which was a good excuse for them not to be specific on the statistic stuffs (hey, Wegman is no improvised statistician).
As to Kyoto and its cost, junkscience.com is the place to go. Just an off-topic update about it: the carbon trading scheme (ETS) set up by the Europeans to reduce carbon emissions is going down the drain, now. One ton of CO2 cost 30$ just eight monts ago. Right now, it costs 50 cents! Centralized & planned economy meeting the stock market!
Thanks, demesure. However, my request is somewhat more specific as indicated below.
Can someone explain just what is going on here? Did MM actually “remove” 80% of the proxy data for 15th century”? I would assume that in “Rutherford, S., Mann, M.E., Osborn, T.J., Bradley, R.S., Briffa, K.R., Hughes, M.K., Jones, P.D., Proxy-based Northern Hemisphere Surface Temperature Reconstructions: Sensitivity to Methodology, Predictor Network, Target Season and Target Domain, Journal of Climate, in press (2005)” in which RealClimate claims has debunked MM by peer reveiw, are actually the same guys who did the work MBH9X?? and other contested documents in this blog? I mean aren’t all these guys the ones who have worked together using similar assumptions and should not state “peer-review”, but say “in defense of” their work…etc.
“As detailed already on the pages of RealClimate, this so-called ‘correction’ was nothing more than a botched application of the MBH98 procedure, where the authors (MM) removed 80% of the proxy data actually used by MBH98 during the 15th century period (failing in the process to produce a reconstruction that passes standard “verification” procedures–an error that is oddly similar to that noted by Benestad (2004)”.
Sorry if this is a repeat, but as an environmental engineer who has to comply with regulations and should be invited to any “shareholder” meetings before the regulations are finalized, I would hope that MM or MBH would realize that if CO2 credits, fees, reduction goals, start being mandated, then engineers such as I are going to have a say.
In particular, each party should realize that even though there is a political process, there is also a regulatory process (which will include engineers and engineering constraints). Bad science equals bad regulations; and the regulated community will not be amused, nor silent about pointing out flaws, or problems with claims that have led to such regulations. Though the budgets of government are large, the budgets of the regulated community are also large. The regulated community also spends large sums of money for legal services since it has little, if any, choice. I don’t know if you have a thread about this.
I also do not know if realclimate or climateaudit realize this, but people in government are starting to propose regulations, fees, penalties, etc., that will effect the way humans interact with the environment. It has been publicly acknowledged in some cases. I know from a confidential source that this is occurring in areas not yet acknowledged. As someone who would have to comply with the regulations that are being proposed, I would like to note that it is the duty of the proponents of AGW to make their case and not the “contrarians” becuase the simple reason is the “contrarians” do not cost any money for compliance. No problem, no compliance needed.
The answer is no. The history of the dispute is obviously convoluted, but Rutherford et al 2005 is an intentional misrepresentation of the situation. Our first article observed that the MBH98 principal components were incorrectly calculated. MBH had stated that they had used conventional principal components but this was untrue on a variety of counts. Instead of using a principal components algorithm, they standardized the data on a short segment and used SVD on the short segment. They also used an unreported stepwise procedure. They claimed that their results were “robust” to the presence/absence of all dendroclimatic indicators.
In our 2003 article, we noted that there were many missing values in the tree ring networks and calculated principal components over the maximum period for which all records in the network were available. (Prior to doing so, we had asked Mann for clarification of the procedures used, which he refused to provide). We illustrated the problems with the PC calculations in our 2003 article on the Australian network where the changing availability of records was not as much of an issue as the North American network, showing that the Mann PC calculation yielded a HS, whereas a conventional calculation did not. In our 2003 article, we observed problems but were not able to fully diagnose them.
After our 2003 article, Mann made much information available and we were able to diagnose the biased PC methodology that had led to the HS shaped PC1s. This was subsequently described in our two 2005 articles. After our 2003 article, Mann also disclosed the stepwise PC methodology – a methodology that is hardly “conventional” and we implemented this methodology in our 2005 articles (published in January, February.) In our two articles, we describe a variety of permutations and combinations of results, all turning on the weighting of bristlecone pines in the various reconstructions. It’s not whether “80% of the North American” tree ring records are in or out, it’s whether bristlecones are heavily weighted or not. Mann’s PC methodology weighted bristlecones heavily so that the HS is essentially made of bristlecone.
Mann had done (but did not report) a CENSORED calculation in which he calculated the results without bristlecones, obtaining a non-HS result as we reported (and as Ammann and Wahl grudgibgly confirm under such circumstances). Knowing that their results were not robust to the presence/absence of bristlecones, they nonetheless claimed (see Mann et al 2000) that their results were robust to the presence/absence of all dendroclimatic indicators.
The Rutherford et al article that you mention was published in late 2005 and did not discuss either of our 2005 articles, which were in print long before Rutherford et al was published. In addition, the key findings of our 2005 articles were known to Mann in early 2004. I objected in writing to Andrew Weaver, editor of Journal of Climate, in January 2005 (based on an online version posted up a year before publication) that Rutherford et al was misleading on a number of counts, not least of which was their failure to consider the 2005 articles.
This is an abbreviated account. There’s lots of more detailed analysis throughout the blog.
Thank you. Having read both summaries and details of your articles, the claim in realclimate struck me as something that should be followed up on.
As someone who specifies equipment, trouble-shoots problems, and solves those problems(hopefully in an economical way), when there is a conflict in language which can result in the misunderstanding of a physical phenomena, I have found I had better research it carefully. As we also say in my part of the world “Don’t piss down my back and tell me it is raining outside.” The wording can be subtle. An example, the manufacturer of our boiler had a written guarentee that the efficiency of a proposed piece of equipment would meet a certain specification. In this guarentee was the wording “depending on wood quality”. So I required the manufacturer to provide the definition, since it was his boiler, and his specified equipment, he should state what the “quality” was. The manufacturer withdrew their proposal. Our relationship afterwards was not good to say the least, since the proposal had been submitted to regulatory staff, monies allotted, schedules agreed upon, etc.
The article in realclimate is then also an intentional misrepresentation. I find myself agreeing that even though Wegman was politely brutal, the quote of #5, and the apparent lack of ethical statisical founding, or perhaps just a lack of good statitics are at the heart of this issue. However, the potential changes that governments are considering is why it is so important that the data and methodology be transparent. One would hope that dozens of studies using differing methodologies and sources would be at the heart of AGW, not just the removal of some bristlecone data.
#26: John, just to reinforce a point of Steve’s. The claim that we got results by omitting 80% of the data is a pure misrepresentation. The germ of truth in it is this: in our 2003 E&E article we couldn’t figure out how Mann dealt with segments of missing data in his PC analysis, so we truncated some matrices at points that allowed the analysis to go through (though it was not 80% of the data). We later found out that Mann had used a splicing procedure where he recalculated principal components on different sub-networks and spliced the results together. There was no way to reproduce this methodology from his disclosure. However, after much experimentation with Mann’s newly-released archive (following our 2003 paper), Steve worked out the most likely splicing sequence, and as of early 2004 we have used the entire network in all replication work. Everything that has been debated since then has been on the basis of both teams using the entire data base. The best coverage of this is our 2005 E&E paper in which we explain the exact steps needed to go between the hockey stick of Mann and the non-hockey stick shape we had earlier obtained. As Steve mentions, to get a hockey stick, Mann needs to include the small group of bristlecone pine data, which (for reasons detailed in the 2005 E&E paper) are likely contaminated and do not belong in a temperature reconstruction model–a point upheld by the NAS panel. Remove the bristlecones, and there is no hockey stick shape left, regardless of any other methodological decision, regardless of data inclusions or exclusions, etc. Mann had discovered this in his own sensitivity analysis, which was posted on his FTP site but which hadn’t been reported in his paper. And with or without the bristlecones, the results lack statistical significance, a point we discuss in our 2005 GRL paper and exchanges with Huybers and von Storch-Zorita.
A chronological overview is in my “what is the HS Debate About?” paper linked along the left side of this page.