I’m trying to use "hissy fit" purely descriptively and not in an ad hominem sense. I can’t think of any other way to describe today’s postings at Daily Kos – scroll down. A reader posted up a pretty sensible and mundane question about RegEM – the new magic bullet from the nomadic Hockey Team.
I understand that MBH98 is out of date. Assuming now that RegEM is still up to date, it is, like MBH98, a parameter-intensive scheme with no defined error model (cf. Schneider 2001). Why should it be immune from the data processing and extrapolation issues raised by BàÆà⻲ger and Cubasch?
Which proxy study proves that RegEM is superior to EOF based approaches? Certainly not Schneider 2001 himself: “Hence, any claim that the regularized EM algorithm or any other technique for the imputation of missing values in climate data is “optimal” in some general sense would be unjustified. The performance of the regularized EM algorithm must be assessed in practice.” – I am unaware of such an assessment.
What is the agenda behind BàÆà⻲ger and Cubasch? [referring back to allegations from S&M that critics have “Agendas”]
Look at the outburst from Mann and Schmidt.
This is the first thing that they say to this reasonable question:
The Shills Have Arrived!
Where to start? A common tactic of shills (a good example is the stunt pulled by shill for hire Steve Milloy) is to truncate a quote so as to completely distort its original meaning
They go on to say:
The commenter is also either extremely misinformed (i.e., didn’t actually read Schneider 2001) or just plain dishonest when he/she claims that the Schneider (2001) “RegEM” algorithm provides “no defined error model”….
The Schneider (2001) paper (and algorithm) are publically available anyway. Or didn’t the commenter know that? Readers (warning, some background in statistics required) ought to take a look themselves, and decide who is giving them the straight story, and who might simply be lying to them….
But the agenda of the commenter–to disinform the readers of this thread–seems quite obvious.
Given prior observations in the thread about Mann’s ad hominems, the commenter raised his eyebrows slightly at this reply, but proceeded to ask some further technical and sensible questions. More hissy fit as follows:
the last time we’re going to discredit your claims
This is getting tiring, and we won’t encourage you on any further than this.
You now ask for a study that compares the MBH98 and RegEM approaches. Why don’t you take another look at the Mann et al (2005)paper provided above, and actually read it. What does figure 2 show? Ah yes, a comparison of applications of RegEM and MBH98 approach to the same precise data set, showing the two methods give very nearly the same result, well within the mutual uncertainties.
Now, we could pick apart everything else you just said, just as we did in the first round(especially your comment about the multiple contributions to the estimated error term which is a strength, not a weakness as you seem to imply, of RegEM). But this is now just getting semantic and boring.
We’d rather spend our time helping to educate the thousands of visitors a day that visit RealClimate who are often genuinely interested in learning about the science.
Your cherry-picking and deceptive quotation are probably not welcome by the DailyKos readers, and we’re not going to encourage you by responding further.
It is quite a spectacle.
Climate Audit 
45 Comments
If “the two methods give very nearly the same result, well within the mutual uncertainties.” then why is it asserted that the Hockey Team has moved on?
Also, did Mann & Schmidt actually answer the question concerning why RegEM doesn’t suffer from the same problems Bürger and Cubasch found in MBH98? I suppose I could go look but I’d hate being found anywhere near DailyKos.
WOW. That’s not a hissy fit, that’s a meltdown. Link please.
I’m trying to figure out exactly what they are doing in RegEM. I don’t have a lot of time right now as I’m tied up with some personal matters for a few days, but it’s pretty high on my priorites. Right now, I haven’t dissected Mann’s respnose. But if Mann had a good substantive response, I doubt that he’d have had such a hissy fit.
I wish that these guys were link their methods to statistical methods that statisticians have analysed.
The underlying RegEM concept as applied to proxy data troubles me. It’s one thing to use the method to estimating missing data here and there in a temperature data set. But I don’t see that it’s very helpful to use missing data methodology to backcast temperaures from grab-bags of "proxies" including contaminated records like bristlecones or screwy PCs given the lack of working knowledge of the properties of this methodology.
After all, the Rutherford, Mann et al paper is only half a year old; and Mann, Rutherford, Wahl and Ammann is only a couple of months old (and they blocked me from the site).
Steve,
http://www.gps.caltech.edu/~tapio/imputation/regem.m
is one reference to the statistic.
Cheers
Louis
If you let me know which site they’ve blocked you from (i.e. the base URL), I might be able to pull down the whole thing on to my web server and you can access it there. The only trick will be if they are using absolute references, but I can probably do a search/replace to fix it so you can view the data off-site.
I’ve saved them both – thanks anyway. Given that the FTP site can be accessedfrom the university, it’s hard to comprehend why one would be so petty on when the pettiness seems so ineffectual. Although, now that I think of it, without the blocking, I’d have probably looked through the file right when it was set up and they’ve gained 6 months that way.
Seems to me that the smartest thing that Mann and Schmidt could do would be to engage a truly independent statistical expert to review their work, and the M&M critique of it, and to explain (to us lay people) exactly which approach is statistically valid.
Re #7. Smart only if one were supremely confident of being vindicated, because the implications of being found in the wrong would be grave — or should that be “difficult to move on from”? — (for one party at least).
Cheers, — Jo Calder
[Mann et al (2005)] “tested the performance of the RegEM method in the context of paleoclimate reconstruction based on application to a long-term model simulation where the answer is known beforehand”
Ultimately, isn’t this circular reasoning? Our computer model confirms our tatistical methods which confirm our computer model?
I am deeply impressed by Mann and Schmidt’s professional response to a straightforward statistical query. This certainly gives me confidence in their results.
I look forward to an independent assessment of MBH98 and related studies and I think only Steve McIntyre could be worried about the results of such an inquiry.
I don’t find it hysterical, except for the two sentences written in bold face. But what I am impressed by is how Mann et al. are ready to leave the decisions what is a “deceptive quotation” or “cherry-picking” in climate science to the DailyKos readers. The same readers who wrote after the defeat of filibuster that America also did not give up when Germany [sic] attacked Pearl Harbor. And hundreds of other things. It makes it rather clear who is controlling the puppets.
The author of the question obviously knew what she or he was talking about, by the way.
Watch “Animal House” and then re-read the comment.
I did see the daily kos bit, and was amused. How about this:
Realclimate said:
Also:
This was in response to Jacks Back, who said:
I am amused by realclimate complaining about selective quotation, and then selectively quoting. Realclimate then state that it is incorrect to say that the WA paper was rejected, and then say (effectively) that it was rejected !
astonishing.
cheers
per
Substitute the phrase “the Mann, Rutherford, Jones, etc et al (yr)” with “our”, when appropriate, in the DailyKos thread and reread it.
Dr. Mann may be attending the American Meteorological Society meeting this week, though apparently he is not author or co-author on any papers, if he is attending. It may be inaccurate to attribute the direct authorship of the DailyKos replies to either Mann or Schmidt, though they are certainly from the RealClimate group.
Uncertainty in this regard is one of the drawbacks (and in other circumstances, virtues) of semi-anonymous posting to blogs and newsgroups. The unfortunate factor it also introduces — as evidenced by these exchanges — is that it also seems to raise the level of invective compared to discussions where the participants know each other by name. And face-to-face exchanges tend to be even more respectful, despite the participants having markedly different viewpoints.
Dear Paul Linsay,
granted, I did not watch Animal House and indeed, I am not planning to watch every comedy that has been created. And it is true that the silly comment about Pearl Harbor also appeared in that comedy – and it is remotely plausible that the original commenter knew the true answer (and “kos” himself knew it almost certainly). But do you question my general statement that the generic readers – and the owner – of DailyKos have absolutely no qualification of ability to judge what is an appropriate statistical evaluation of the climate?
You know that my goal was not to discuss off-topic things like the 1978 U.S. comedies but rather the overall intelligence and knowledge of the DailyKos readers which, I insist, is low.
Best
Lubos
Lubos
While I agree with your description of the Kosians there. the “When the Germans bombed Pearl Harbor” is pretty famous here, definitely part of the culture, and Animal House isn’t a second rate comedy that barely made money at the Box office. It was a huge movie here.
Having said that I’d have to see it in context, because it’s still possible that whoever posted that was being serious.
But the John Belushi Line is at least as famous as “We’ve got a half a pack of cigarettes a full tank of gas, it’s dark out and we’re wearing sunglasses.”
“Hit it”
Or
“Frankly Scarlet I don’t give a damn.”
Sid “KAAAAHHHNNNN!!!” Viscous
#16, Dear Lubos,
Unless I’m alerted to a truly outrageous post I never go to DailyKos, I always feel like I have to take a shower after reading more than five of the comments. The verbal filth is usually too disgusting to tolerate. I’m always amazed that they can write at all, so no, I don’t have any expectation that they understand the least bit of science or math over there. For that matter, when climate comes up at slashdot.org they don’t have a clue and they’re technically sophisticated.
I did go over to Kos when Steve mentioned Mann’s interview with DarkSyde. I read Mann’s bio and thought it was a bit strange but I couldn’t quite put my finger on it. There was something about that Masters in physics from Yale. Then it struck me. When I was studying for my PhD in physics, after we’d finished our course work but before we could do research, we had to pass a twenty four hour exam covering all of physics, plus make a one hour oral presentation on a topic of our choice. The exams were given twice a year and you had three chances to pass. If you failed on the third try, you were given a Masters in physics and shown the door. No one was admitted to the program as a Masters candidate. Out of curiosity I contacted the Yale Registrar of the Physics Graduate Program re a Masters degree. “We do not offer a terminal Master’s Degree; we have only[sic] a Ph.D. program only.” Hmm, so Mann decided that climate research was more interesting than physics after passing his exams?
Get some Pilsner Urquells and watch “Animal House”, you’ll have fun.
comedy gold.
mark
#18 I knew a Yale Masters in stats. Ran out of stem writing her dissertation, but *very* sharp. Dropping out with a Masters is neither a mark against nor an excuse for work done later.
I’m so confused … someone please help me.
RealClimate informs us that the science has moved beyond MBH98 and it is now a big ho-hum to discuss it further. But on DailyKos RealClimate lets us know that:
Why is Clmatic Change publishing a substantial paper on a topic of no interest anymore?
It doesn’t dismantle the criticisms; it isn’t “in press”; the reason why they are pushing is that they need it to launder MBH98 in AR4. It doesn’t matter a damn as far as IPCC is concerned if Ammann is just as bad as MBH; all they are worried about is getting something that is “peer reviewed”. Of course in this case, Ammann refused to produce cross-validation R2 statistics to a peer reviewer upon request so it’s going to be really tainted. Ammann’s website only says that it’s been “provisionally accepted” – I wonder whether that means that he gets to get the companion paper through GRL as well. Our Reply is pretty toughly worded and is not going to make it any easier for GRL to digest, even though Famiglietti’s got his finger on the scales.
Steve, you’ve met these guys and talked with them — Ammann, Famiglietti, and others. How do they rationalize sacrificing their professional scientific integrity and the content of the science itself, all on the altar of politics? Something is missing from the equation. What’s driving these guys? It almost seems like an unspoken conspiracy to cheat, which is very hard to credit. Is it blind righteousness, save-the-world, or what?
Re: #15
While I tend to agree with you on the differences in respect between semi-anonymous blogging and face-to-face interactions, I don’t think there’s much a difference when it comes to Mann. I’ve been in the same room with him and he’s every bit as arrogant and condescending in person as he is on the web.
I have to say I am bemused, and something doesn’t seem right to me. If an author submits a paper, it seems to me that it is the referees that are the people who judge the science. If a referee requests a statistic because they feel it to be essential, and the author refuses to provide that statistic, then it is up to the referee to advise the journal whether the paper can be accepted without this statistical information.
If the author wishes to argue for the validity of the statistic, or the journal ignores its referees, that is another matter. But I do not see how an author can refuse to provide information that is essential for the paper.
cheers
per
Re #24
Strikingly so. I have listened to him interviewed several times on the radio by fawnning adherents. Even in such a pleasant environment any hint that his methods and conclusions are in dispute send him into tirades. He would have us believe that only evil fools can find fault in his work. At this point the interviewer invariably backs off, least they be seen to join the dark side. He is never pressed. I suspect he would fare very badly in real debate.
Are Bürger and Cubasch evil or are they fools? Mann may want to expand his repertoire.
re #23
I’m going to give my own opinion: No, there is no grand conspiracy to pervert science by lots of scientists – no grand scheme or plan – no secret funding or backhanders.
In my opinion, the whole AGW affair is not a conspiracy. Yes, there are people who converse with each other around the world, but in truth these people are not capable of organizing a conspiracy.
What we have here is a classic contageous hysteria, and one that will not die down until another panic takes its place. Certainly hysterias do not in themselves disappear in response to facts or reason.
John,
The best advice I have ever seen for anyone anywhere is; “Don’t panic”…
Which can be found in large red letters…
re 21:
It’s like software vendors: “Our next release will be bug-free”
#27
John, agreed. Just as Catholicism is not a grand conspiracy to collect money for the Bishops, they truly do believe.
However there is the one bit of info, of which we sadly don’t have enough details, and that’s the “We have to get rid of the Medieval warm period” business. Without more info it’s hard to evaluate the veracity. But, if true, it shows a more malevolent intention.
I can’t find fault with their intentions “Save the world!” however I feel they have either consciously or sub-consciously created the problem. Just as they say about our (US) government that we had to re-invent an enemy after the fall of the Soviet Union , the 21st century luddites can’t get industry to shut/slow down without having a real fear to rally around.
Do you mean AGW alarmist hysteria, or sudden rock-star-type fame hysteria?
My pet theory is that when these green party academic types are suddenly thrust into the limelight as the saviors of the planet they can’t help but get all wrapped up in the glory and their ego starts to override their reasoning ability. What becomes more important, the hard work of science or the softball interviews, the regular mentions in the news media and the, ahem… new young google-eyed co-eds that are all trying to enroll in your classes of late?
ET –
“We have to get rid of the Medieval warm period”
This sounds like something that Henry Diaz would say. Maybe a google search on him would turn something up.
Re #27, John A.,
Would you expand on the “classic contageous hysteria” remark? Why don’t you think AGW is self perpetuating sociopolitical phenomenon? I see different groups with different interests. Some ordinary stuff… The false comforting belief that climate is stable and that only man can change it. A desire to see mother nature as just and rational. A moral play where mother nature punishes man for waste and greed. More a new age religious tale with prophets and doom and good and evil doers.
Perhaps in the end it’s all about a group of males seeking power and status.
Doug
The issue is that it was a comment made to, I believe, David Deming in response to an article he had written. But he won’t say who actually said it. I believe Steve has some suspects because I think he mentioned something like “So and so who is suspected of making the “We have to get rid off…””
http://www.henrythornton.com/article.asp?article_id=3165
http://www.cei.org/gencon/014,04484.cfm
“false comforting belief that climate is stable and that only man can change it.”
That’s the one that rally bugs me. If you want to talk consensus, there is a 100% scientific consensus that 10,000 years in the past (as well as other times) my house was under a kilometer of ice. That shows that the climate is naturally variable.
then they argue that it’s getting warmer than it ever has been, which is the crux of the argument at this point. There it’s a little fuzzy, but irrelevant really. the climate is variable, even if now it’s 0.6 degrees warmer than it has ever been before (it’s not I know, but try telling them) it’s irrelevant because the climate varies by 10’s of degrees naturally, anything that small is lost in the noise.
From Steve Milloy today
http://www.junkscience.com/MSU_Temps/Greenland_GISP2_long.htm
Look you can even see a teeny tiny hockey on the far left, that is absolutely dwarfed by the natural variation for all of the time period before that.
Damn warmers and there x-axis time scale limiting. Find the coolest time in the recent record, and use that as your 0 point, instant warming.
#25 – per, in the case at hand, Ammann and Wahl refused to provide the cross-validation statistics to the referee; the referee told the journal that the refusal was unacceptable and it appears that the journal (Stephen Schneider) has simply ignored the referee if the article has been “provisionally accepted”. Interestingly, Ammann and Wahl cited their GRL submission in support of their refusal to provide the cross-validation statistics, as supposedly containing lethal evidence of the defects in MM05 – at the time, they had already been rejected by GRL, which they concealed from the editor and referee. They had a little bad luck in that the referee knew about the rejection and so advised Schneider sharply reprimanding Ammann and Wahl for this concealment, but Schneider does not seem to have cared.
Stephen Schneider
That guy keep turning up. Talk about a Bad penny, but if he’s the one in charge of such things it starts to make much more sense.
I must admit that this does get quite curious. If an editor goes against the advice of the referees, then that raises interesting questions 🙂
It does happen legitimately; if the referees point is irrelevant to the paper, or if it has no substance, or if the editor gets conflicting advice from multiple referees on the same point. Starts to make it a rather sticky wicket if the editor is trying to justify his actions later on tho’, and given the political interest here, one might guess that only a foolhardy editor would make a clearly biased decision.
One of the issues here is that I don’t know what “provisionally accepted” means. There is a common practice, which goes like this:
“We will reconsider your manuscript for publication, providing that you can address the referees comments below:
referees points #1,2,3”
Under these circumstances, you would have to see what the stipulations of the journal were before you could come to a view about what is happening. I am sure we will be rapidly enlightened by the loud trumpeting of success if this paper is finally accepted !
cheers
per
Interesting how none of the criticism of various scientist these posts of ever also seems to refer to these doing the criticising. Odd that. But then again, you’re probably all perfect…And none of you show any signs of religious ferver, oh no! It’s just there is no doubt you’re right and Steve and John ‘A’s the word handed down truth…
So there!
See, we could have a real flame war here if anyone like me could be bothered 🙂
Or if we could understand what you are trying to say.
?
Yes, whatever are you trying to say, Peter? You criticize Steve all the time and we love to play whack-a-mole with you!
RE 35 ET Sid:
I’m sure you know, but for others, the Junkscience site has some extensive climate proxy info/graphs here:
http://www.junkscience.com/MSU_Temps/Warming_Proxies.htm
One concern might be cherry-picking (can’t tell offhand), but they include some of the HocTeam stuff. I know eyeballing is problematic, but guessing >50% clearly show the MWP & LIA. What else is interesting is that the long-timescale proxies like ice cores often show the highest temps 10000-14000 BP, then w/a slight but consistent downward trend since then (w/a secondary max ~5000 BP). What’s remarkable about that is that the LIA might’ve been the coldest global temps in 10000 or more yrs! That’s never mentioned….
Slightly off topic, slightly not ….
In terms of those who have hissy fits, of late, some readings have given me a chance to get more familiar with New Zealand’s Earth Sciences community. Talk about true believers in the Hockey Stick … they might as well be honorary members of the Hockey Team.
There must be something about those islands which fosters the Dr. Smith mentality. “Were DOOOOOOOoOooooooomed!” 🙂
Maybe it’s living on top of an 8.0 earthquake waiting to happen …. hmmm…. so do I, and I’m no Dr. Smith. Well so much for that theory …..
I have not tracked all the ins and outs of this fascinating hockey stick debate yet, but the notion that industrial greenhouse gases are driving the climate can be knocked independently, at least here in Australia for the rainfall, by considering the decadal variation in annual rainfall stats for various stations, of which we have many. The long rolling decadal waves cycle, so they at least cannot be industrially driven. They are also of differing wavelenghts, at different stations.
Also, as we have had a lot of interglacials back in the past three quarters of a million years (see the Dome C and Vostok ice-core data) when temperatures and CO2 levels have been higher than they are now, unless the good folk of Atlantis were very active many times, with long lunch-breaks, and were very meticulous about disposing thoughtfully of all their solid junk, it is perhaps a fair question to ask what is so special about this interglacial, except that it is ours? A slightly longer rear-view vision will I think, help bring a little more clarity. For instance, the Vostok data shows the temperature proxies climbing, and then, some 300 to 3,000 years later, the CO2 climbs or falls in step. In this universe cause is, I think, usually required to precede effect.
As an aside, I do wonder if some of the recent spikes in ice-core data are not there because whatever is being measured in fact dissipates rather rapidly when first buried only just below the surface of the icecap, and then much more slowly as the later snow layers bury that one ever deeper. That could leave several proxies in tatters, at least for the recent past. Like, say, the last thousand years or two.
Do not despair that you are alone in a sea of exceptionally slack science, earth climate folks. The story of the defence of the geologically impossible past rivers and oceans of Mars very closely matches the strategies used in the hockey stick war here. One time series, infinite variables, all either varying directly or inversely to each other. Result, an infinite number of possible Ph.D.s.
Re #44, Peter Ravenscroft
Who won ? And how ?