Dr UK writes in about an interesting article about ad hominems in the Climategate emails
Ad hominem arguments in the service of boundary work among climate scientists
By Lawrence Souder, Furrah QureshiIn their conclusions, Souder and Qureshi contrast the behaviour of climate scientists revealed in the Climategate emails with that of gravity wave scientists studied by H. M. Collins:
In his ethnography of gravity wave scientists, Collins fantasized: “[S]cience done with real integrity can provide a model for how we should live and how we should judge.” He makes this claim not because he finds perfection in the practice of science but because he found practitioners of science in a community who openly revealed their imperfections. This community, he boasted, gave him virtually complete access to their work. On account of this transparency he felt he could trust them implicitly.
I found the Souder and Qureshi paper very interesting. It analyses (in a qualitative way) the different forms of ad hominem attacks found in Climategate emails. It is getting short shrift in the comments at Bishop Hill, and indeed contains some factual errors and misunderstandings. But as commenter The Leopard in the Basement puts it, if one gets past the socio-speak and the references to ‘deniers’:
“I’m pretty sure the Team won’t like this one bit if they ever saw it it.”
Some factual errors in the article e.g. Real Climate was started before Climate Audit, but an unusual acknowledgement within the academic world.
Climate Audit 
55 Comments
Thank you! I saw it first at Bishop Hill:
http://bishophill.squarespace.com/blog/2012/3/16/irony-fail.html
There’s discussion of another similar paper by Marianne Ryghaug and Tomas Moe Skjølsvold at Klimazwiebel.
Is not the use of the term “climate change deniers” in itself ad hominem, assuming we accept the loose definition of the term ad hominem employed here?
theduke
Re: dpeaton (Mar 17 10:40), Better still, why are people missing this?:
Self-proclaimed climate deniers in the political arenas
Uh, yeah… Where have we heard that before? Is there competition for a certain designation? 😉
Yes it is. It’s comparing skeptics, the normal position of every scientist, to that of holocaust deniers. It shows just how politically charged the situation is. It’s never been about facts. In many ways, it reminds me when a grave incident happens when many people die and someone accuses a certain group without any evidence. If you dare say, “hold on, let’s wait until we have all the facts”, you will invariably hear “don’t you care that X number of people died? How heartless are you?”. It’s like the severity of the incident is more important than the truth. In this case, there are many parallels. The severity of global warming trumps the truth.
Well, that paper certainly formalizes one side of the argument. Apparently it was a hacker who released the climategate e-mails and all the scientists involved have been vindicated. But (as an engineer) what else would I expect from a bunch of artsies.
As Bishop Hill correctly points out, their use of the word “denier” throughout a paper speaking to the use of ad hominems is truly ironic. Actually beyond ironic and well into ignorant.
OK. I’m quoting from the paper,
The so-called Climategate email messages are publicly accessible now because in 2009 they were hacked or leaked [32] from an internet server at the Climatic Research Unit at the University of East Anglia.
[32] For an analysis of the various explanations for the appearance of the Climategate email messages see Leigh et al. (2010), Climate
emails: were they really hacked or just sitting in cyberspace?, The Guardian, Thursday 4 February 2010 12.14 EST,
http://www.guardian.co.uk/environment/2010/feb/04/climate-change-email-hacker-police-investigation.
I’m afraid that I find this kind of sophistry and sloppy writing unpersuasive. Apart from the easily checked factual and spelling errors, what they have done is to redefine ‘ad hominem’ to mean whatever they want it to mean and then create some new categories. It’s not exactly rigorous thinking or exposition.
No cigar, IMO.
When they start talking about “Meta-Ad hominem”, it’s time to summarily dismiss it as post-modern gibberish. If they had anything substantial to say, they could have said it in English.
ChE – “Summarily dismissing” is one of the main problems of the climate debate. Where else is there such a listing of the quotes and attitudes of the climate scientists in the academic literature as in this paper?
The authors apologise for the ad hominems revealed in the Climategate e-mails saying such familiarities are necessary for honest collaboration in science. And yet I cannot for the life of me imagine Christy and Spencer or Soon and Baliunas (say) ever stooping to such levels.
I agree with johanna above.
Speaking of ad hominem remarks, in my opinion the authors of the subject paper certainly dropped a bomb on this Website:
“McIntyre is not a scientist, but an engineer with special training in statistical methods.”
I believe that this statement would qualify as an “indirect ad hominem” that places Steve on the other side of the boundary from the “authentic climate scientists”.
I am a bad person. Once you get past page five things get very interesting. I cannot take back all of what I wrote but I now regret some of it. Forgive me please.
From the paper:
I’ve never been convinced of this (call me naive). When CG1 came out, it seemed to me the timing was more related to the FOI battle then Copenhagen. It also became clear that the Team desperately wanted to position CG1 as a politically motivated operation designed to disrupt Copenhagen rather than a whistle blower exposing misconduct. Recall the early and vigorous attempts by UEA and Mann to blame the Russians! But the Team could never get this idea to stick. They were lacking evidence and inference.
When CG2 happened, I was struck by the rather mundane nature of the emails. They were of a fundamentally different character from the first release. There were no bomb-shells and nothing memorable. Yes, they cast the scientists is an unfavorable light, but there was nothing new. And most importantly, unlike CG1, there was nothing that suggested serious (perhaps criminal) misconduct. In the end, CGII was a bit of a fizzle.
But now were seeing CGII offered as proof that the Climategate episodes were sophisticated political operations designed to disrupt international climate negotiations. CGII is the “proof” the Team needed.
Given the recent activities of Gleick, is it really all that unreasonable to posit that CGII was a false-flag operation? In other words, is it possible that someone from the pro-AGW side put out CGII to coincide with Durban to create a false impression that Climategate was all about sophisticated political efforts by “corporate interested” and the “denial industry” rather than a whistle blower from inside of UEA? IS CGII just another example of the Team manufacturing evidence to support a hypothesis?
I agree with Jeff Norman. The paper bears reading after you get by the rough spots. It has value after a full reading, although the distinctions between three types of ad hominems seem contrived to me. (What is a “meta-ad hominem” anyway?)
The picture painted is of a group of scientists hiding their data in order to protect the pre-determined narrative they were laboriously writing year after year as contributors to the IPCC process. I think they were obviously intimidated by someone whose acumen in statistical methods was either equal to or superior to their own. Here’s the paper quoting Santer: Santer in one of his few messages to McIntyre himself said: “I gather that you have appointed yourself as an independent arbiter of the appropriate use of statistical tools in climate research.
So they resorted to hiding the data and refusing to cooperate. They saw what M&M did to the hockey stick and it’s creator. They were afraid the whole edifice would come tumbling down.
The authors conclude: What distinguishes good from bad science is not the quality of scientists’ results but the candor of their reporting.
In other words– the words of Mosher– “Free the data, free the code.”
and open the debate
“Free the data, free the code, and open the debate.”
Steven Mosher, © 2008(?)
I thnk so. in 2007 it was free the code. hansen relented
then lukewarmer was born.. then the the tagline came.
The Emperor’s Striptease, in slow-mo.
At the same time, the author/s seem to betray a pro-Warmist bias. Whatever did they hope to accomplish?
Excruciating reading for the The Team.
depeaton, I agree that there’s an important pearl in your last quote, but the authors missed the AGW climatology bulls eye. It needs to be slightly amended, to this: ‘What distinguishes the good from the bad scientist is not the quality of scientists’ results but the candor of scientist.‘
AGW-promoting scientists lack candor, the sine qua non of professional integrity. They have jettisoned honesty. They have exalted sentiment above science. They have corrupted science with the opportunistic lie of politics. Their offense is unforgivable.
“McIntyre is not a scientist, but an engineer with special training in statistical methods.”
——————————————————
Since this is all about semantics, talk about damning with faint praise! Steve is not a (this, which is the authoritative source), but a (that, whatever). A neutral statement would have been something like – Steve McIntyre has qualifications and extensive experience in engineering and statistical methods.
Pea, meet thimble.
That’s actually not accurate. Steve isn’t an engineer. But that’s close enough for their purposes.
I stand corrected. He’s not an engineer. Regardless, I don’t think the description was meant to be disparaging, given the general thrust of the paper.
I don’t see it that way, Johanna. The description of Steve is accurate and I doubt it was meant to be disparaging. It comes under a section titled, “Context.” The authors were perhaps alluding to the fact that the “scientists” might have some cause to initially suspect McIntyre’s work.
The next paragraph describes Steve’s work as “trenchant,” which means “keen, penetrating, incisive.” (Websters New World).
I think Steve comes out rather well in the paper.
Scientists are people who follow the tenets of science. You don’t have to have a degree to be a scientist, or work at a university. You just need to perform research work in an open and honest manner and be willing to be challenged by your peers (and in turn, challenge them).
By that standard, Mr. McIntyre is most definitely a scientist. To suggest otherwise is not intellectually honest.
People who refuse to share their data and methods are not scientists because they are not following the tenets of science – no matter how much they claim the opposite.
Science and politics don’t really mix. Beware anybody who claims to be a scientist but operates in a partisan manner. They’re unlikely to be searching for truth.
Re: Nicholas (Mar 17 20:32),
Completely agree. This is why a young teenager was able to get a paper printed in JAMA (Journal of the American Medical Association. Her work was scientific; ie she was doing science.
Steve does science, and most certainly has expert knowledge in certain scientific fields.
@ Nicholas: “You don’t have to have a degree to be a scientist, or work at a university.”
Agreed. The greatest scientist/inventor in American history, Thomas A. Edison had only a few years of elementary education and did poorly.
Steve does science by auditing and correcting it. That helps it move forward. He’s indisputably a scientist and he’s proven he’s a better one than those who dismiss him and his work.
“Not a scientist” is a misnomer and should have been left out. I would have worded that sentence:
|
“McIntyre is a mathematician who has extensively used applied statistics throughout his career in mining exploration.”
|
…or something like that – but that’s just my take.
I found the paper amazing. Get past the denier ad hominom. Remember the author also used the term warmest and alarmists. The paper is the first scientific analysis of what many have been saying for years. That scientist became far to political and had too much invested in their positions to listen to legitimate criticism. I love that he picked up on the Hockey team so clearly without using those words. “Insiders” is that not the Hockey team. Steve wrote a great piece on the Peer review a while back. This paper has a very similar feel to me. It is a scientific paper that says these scientist behaved badly and not in the best interest of science. HHHHeellloooo, has this not been what has been shouted from the rooftops by many “deniers” for years. Hopefully peer pressure of the science community will start to focus on the unscientific behavior and scientists can get on with doing what real science is supposed to do. Seek the truth.
Second post in five years of reading so allow me to add this.
Thanks for your hard work Steve. And again, one day history will look back on this time and you will be seen as one of the heroes in this debate. Keep buggering on.
fortescue – very well said
The authors also botched the implications of “hide the decline” ascribing scientific legitimacy to what they actually did. Truncating Brifa’s dendrochronology results after 1960 (where it has a divergence from the temperature results) would bring into question the earlier periods of the Hockey Stick that rely heavily on Dendrochronology. Deleting known but inconvenient data to make your results fit your conclusions is not a legitimate scientific method.
But I have seen several Scientific papers with results not entirely consistent with the Team’s AGW narrative explicitly include a bow to the Team’s view. I was left wondering whether the early parts of the paper are a device to thwart having it dismissed for having a skeptic ‘camp’ bias.
I also noticed the use of denier evolved to skeptic and warmist to alarmist as the paper went on. But more over I was astounded that they presented as an accepted general fact that the Team had kept contrary points of view out of the ‘peer reviewed’ scientific literature and then criticized skeptics on that account. There was also a sense that in so doing, gives good reason to demure on their conclusions.
So I don’t put much weight into the early ‘friendly to the Team’ positioning.
re: hacked v. ?
Given the packaging of the email releases, what was selected, how attachments were handled, etc., it struck me at the time that someone with good unix sysadmin skills had done the work. And likely an insider that had knowledge of what was where on multiple systems.
In time after passions cool (and they’ve retired), I expect the whistleblower to self-identify and add some color in his/her reporting – tales of hallway conversations, water cooler talk, who-was-sleeping with whom, etc.
It wouldn’t surprise me to find someone on the staff offended by their behavior to the point they decided they needed to act so they could look themselves in the mirror everymorning.
We likely could get them to come out of the closet if we or some group could insure they wouldn’t be harming their family and dependents. Assuming they are 30 years old, they’ll need 30 years of income & health care protection (or to move someplace with an equivalent job with similar compensation).
The paper appeared to me to be bending over backwards to be neutral on the climate change question, while tut-tutting the poor behavior of the climategate emailers–sort of Judith Curry-style. Otherwise, though, the paper added little of value beyond their point of view. The idea of scientifically analyzing ad hominem attacks is a bit dubious from the start, and this paper did nothing to vindicate the concept.
The problem with the detached, neutral, scientific conduct-focused approach in general is that the scientific questions can’t be so cleanly separated from the conduct questions. It’s nice to think that even when one side in a scientific debate–whether the establishment or the dissenters–is the scientific equivalent of flat-earthers or creationists, the other side can and should stick to careful technical arguments, and will eventually win the day. But people (and institutions) are human, “eventually” is a long time, and a lot of damage can be done when flawed ideas are advanced by unscrupulous means, and resisted only by the most fastidiously scrupulous ones. In this respect, I think the “alarmists” have the better of the argument.
“Fakegate” has provided an excellent illustration of this point. Most of the discussion on both sides has carefully avoided the question of the authenticity of the “strategy memo”–one side emphasizing instead Glieck’s dishonest methods, and the other side, the contents of the acknowledged-authentic Heartland documents–for the understandable reason that the authenticity of the “strategy memo” cannot be proven one way or another, and probably never will be. Yet the final judgment on Glieck’s actions depends crucially on that question: if the memo is indeed authentic, then Glieck’s deception to expose an organization intent on undermining science education is at least understandable. And conversely, if the memo is fake, then Glieck isn’t simply an investigator with somewhat controversial methods–he’s at the very least a reckless slanderer, and at worst an outright forger.
Likewise, if opponents of the “consensus view” on AGW really are the equivalent of flat-earthers or creationists, trying to replace legitimate scientific consensus in the service of a patently unscientific agenda, then scheming to keep them out of peer-reviewed journals and science classrooms is a perfectly reasonable, even noble endeavor. It’s only if the climate skeptics have a legitimate scientific case that the shenanigans of Jones, Mann et al. start to look disturbing.
And this is where the real failure of the broader scientific community becomes clear. Academic research has become so specialized and compartmentalized–for political reasons as much as for scientific ones–that entire scientific fields of highly dubious merit have sprung up, keeping large numbers of “scientists” busy doing research that is at best useless and at worst downright bogus, with nary a complaint from the collective scientific establishment. In this environment, it’s simply expected that the scientific community will rubber-stamp the conclusions of any fairly small collection of scientists that comes under attack from without, regardless of the credibility of their conclusions and despite a complete lack of external scrutiny.
It should be obvious to any scientifically literate person that the claims of the AGW establishment are nowhere near iron-clad enough for their critics to be dismissed out of hand as cranks, lunatics and political hacks. Yet not only do I hear embarrassingly few scientists from outside the immediate field address this point, but nobody thinks it odd that these outside scientists should simply defer en masse to their specialist colleagues, no questions asked. If I didn’t know better, I’d think the scientific research community as a whole cared more about solidarity in the protection of its coddled, well-funded “expert” status than about the quality of its scientific research.
Climate science must deal with considerable complexity and uncertainty. As with most fields under these constraints, there tends to be a gentleman’s agreement that you get to assess your own hypotheses using the data most likely to provide support. This makes everyone’s life much easier. If someone comes along and tries to shoot down a hypothesis using the data most likely to refute, the ranks must close against that person. It is quite difficult to attack that data. You have to attack that scientist, or everyone’s life gets more difficult. This is the proper context for the two incendiary M&M papers, and it is the proper context for Soon and Baliunas. The same sort of strategy – outsiders attacking using the sharpest swords they can find – would draw the same sort of response in quite a few other fields that are operate at the fringe of our understanding.
Except in this case, the hypothesis was used as a pretext to stampede public opinion toward a new global economic and governing structure. Hardly gentlemanly.
Matt,
Nice try, but that’s not how science works. I agree that’s human nature you are describing, but a real scientist makes every effort to rise above that kind of primitive motivation. In contrast the team wallowed in it. Pointing at someone else and saying, in effect, “They would have done it too” is no defense.
I agree with this. The idea that what happened in climate science is characteristic of all science is grossly unfair.
And how many scientists want to purposely leave behind a legacy of flawed science characterized by cherry-picked data?
Paul and dpeaton,
That is three straw men between the two of you. Paul, I am not defending or attacking anyone. I am pointing out how scientists behave in quite a few fields where absolute (dis)proof is elusive. We are not talking particle physics or genetics. In those fields, you better test your hypothesis with the cruelest data or someone else damn sure will. Dpeaton, I did not say all science. See comment above. Nobody wants to leave a legacy of bad science. Using the data most likely to support your idea does not mean you are wrong. On the contrary, you are likely to be right. It is just not the best way to push the science forward. What us engineers like to think of as the “soft” sciences tend to build fragile webs that can be torn asunder. New theories add as long, tenuous threads. See the history of psychology, for example. Disciplines like particle physics are more like ants carrying one more tiny grain of sand to the top of the pile. Long, tenuous threads get ignored as crackpot in these “hard” science disciplines. Someone might kick a grain off the pile now and then, but the pile pretty much endures. Climate science is so acrimonious because scientists have demanded that everyone acknowledge their tall pile of sand. Skeptics take a close look and see a tenuous web largely held together by consensus. It is precisely because the edifice is weak that the climate scientists “protesteth too much” with all the ad homs.
Point taken. I noticed that you had originally qualified your assertion to include a subset of “all science” last night. As I am in no position to refute what you say, it stands.
First, if the paper is presented as a sort of back of the napkin idea of ideas to consider, and paths to think about, that’s one thing. If it is presented as conclusive settled science, it’s entirely another.
Second, the “data used to refute” was not some other data set selected by M&M, but was primarily the data sets that MHB selected, with the general methods MHB selected, with the math done correctly. On top of that, was an additional notation that some of the more prominent data sets weren’t as they were described, with others suffering from known and widely accepted flaws.
That M&M didn’t use their own data sets is one of the prominent complaints made of them by the Hocky Stick Team.
“No one should be surprised that scientists, when among their closest colleagues, will let down their
guard in the interests perhaps of conversational efficiency and say things like “Mike’s Nature trick” and
“to hide the decline” to refer to an acceptable method for combining different kinds of data sets.”
I never got by the sentence excerpted above when attempting to read the article posted by SteveM. I think I and other persons with some background on these issues tire of hearing these simplistic and innocent explanations for actions that need details of the implications for proper evaluation. Of course, most of us are not surprised to hear scientists talk to each other in private the way we saw it done in the climategate emails. The implications of “hide the decline” are more nuanced and far reaching than a mere informal exchange between scientists.
Ken: by the time I got to that point, I wasn’t expecting much and I just glided past it, intent on finishing it so I could drub it in a comment. But, by the end of the paper I felt they had redeemed themselves– if not entirely, then substantially.
You’ve quoted the most glaring error in the paper. There is also the unintended irony of referring to “climate change deniers” in a paper purporting to criticize ad hominem attacks. After that, you expect the thing to continue in a downward spiral. But upon completing the paper, I felt they had adequately exposed the depravity of the scientists in question and defended the work of individuals like Steve and Ross as valid contributions to the science.
It isn’t at all what they said. It is what they did. How can that be so difficult?
It is their use of the phrase, “acceptable method,” in your quote, Kenneth, that clearly indicates they either did not understand exactly what was involved with “hide the decline,” or they chose to ignore it.
There is one glaring omission in the paper; There is no reference to the Spencer & Bradwell 2011 episode at Remote Sensing [I think they were published at about the same time, so no blame to the authors]. The talk about getting rid of troublesome editors of journals appeared to become a reality in the case of Wolfgang Wagner.
Which may not the first time it has happened:
“Will no one get rid of this turbulent priest” – King Henry II [allegedly said of St Thomas-a-Becket” before Becket was murdered INSIDE Canterbury Cathedral.]
“‘Let us away, knights; this fellow will arise no more.”-Henry’s soldiers after they had murdered Becket.
My use of the word “omission” was not correct because it implies that the authors wilfully left the subject unmentioned, which was not my intention. “Glaring absence” would have been better.
“Ken: by the time I got to that point, I wasn’t expecting much and I just glided past it, intent on finishing it so I could drub it in a comment. But, by the end of the paper I felt they had redeemed themselves– if not entirely, then substantially.”
Even when intellectual lightweights might have good and true things to say about affairs they remain lightweights. I think we can all see beyond name calling and to concentrate on who does that detracts from the substantive issues of the science. I suspect that on a personal level climate scientists are no better or worse than other scientists and scientists in general are no better personally than your average Jane and Joe. The mixing of science and advocacy is another matter and well worth a few articles – even if those articles are by the softer scientists.
Hmm – isn’t Stephen McIntyre an “applied scientist”?
(I forget whether you are a mining engineer or geologist (both “applied scientist”) or some other educational background, but obviously from your work you perform far better than ivory tower “scientists”.)
Nothing wrong with self-taught either, in fact some of my profs liked to emphasize that an engineering education was primarily training in how to find information and use it (i.e. think). (Challenge with self-taught is cross-checking against reality. A good educator would give the advantage of having selected quality relevant sources, a self-taught person probably selects by focusing on actual problems – like a couple of bicycle mechanics did prior to 1903 when their machine flew at Kitty Hawk NC whereas not far away in in Washington DC the well-funded perfesser’s crashed.)
Narrowly defining “scientist” is a typical alarmist scam. A couple of dozen academics from UBC and UVic, including one Andrew Weaver, tried that one in a letter to the magazine of the Association of Professional Engineers of B.C. Its’ the classic “Argument from authority” tactic further corrupted, I suppose.
Applied scientists are good at validating theories against reality – the bridge has to stay up. (And at developing theories from reality – empirically-derived formulas, and recognizing their limits. Indeed, one engineer has taken a popular empirically-derived formula for heat transfer and adapted it to the atmosphere (for example, pressure and density vary with height). He concludes it supports the theoretical particle physics limit on the effect CO2 can have on climate temperature. (CO2 is present in popular combustion devices, have to keep the exhaust from burning out.)
Of course good scientists rigorously validate against reality. I was not close enough to research at UBC, and my memory has faded, to evaluate how well the various mathematics and physics professors were at validating, but most of them used real examples in teaching. (One math prof was a geophysicist, I found his example of vibration modes of the planet interesting. Another math prof had been an “applied mathematician” from some other place.) Of course the engineering profs were reality focussed. The problem profs I remember were graduate students, not selected for their teaching ability, probably given the job as income while they proceeded toward their Ph.D.
There’s an amusing but apt quote in Rob Flittons great book “Negotiation for Life and Business”, in the “Ninth Trait” section. Quoting someone named “Jack Kilby” saying “You could design a nuclear-powered baby bottle warmer, and it might work, but it’s not an engineering solution. It won’t make sense in terms of cost.” (Preceding one by Thomas Edison is also good.)
Sounds like most of the schemes to mitigate CO2, like the ship with tall chimneys to blow something into the air.
(Of course cost is no barrier in the minds of alarmists, as they believe catastrophe looms and they have little appreciation that the cost of their schemes takes away from many things that help humans.)
BTW, my engineering degree is conferred on me as “Bachelor of Applied Science” (in Engineering Physics).
– Keith Sketchley, B.A.Sc., P.Eng.
There was a Nobel laureate named Jack Kilby, co-inventor of the integrated circuit.
Yes, and if it was that Kilby it may have been said with feeling, in that in 1958 he made key breakthroughs needed for the IC but based impractically on Germanium. Robert Noyce created the digital revolution we have known ever since by making the ideas work with silicon, set out in his patent application of February 1959. Noyce of course formed Intel, setting many standards of Silicon Valley informality, and ten years later his close colleague Gordon Moore spelled out Moore’s Law to explain the rest of our lives:
http://upload.wikimedia.org/wikipedia/commons/0/00/Transistor_Count_and_Moore%27s_Law_-_2008.svg
Kilby was recognised with a Nobel Prize for Physics in 2000, which seems a tad late. But perhaps the key technical step was not Kilby’s but P–n junction isolation, patented by Kurt Lehovec at the Sprague Electric Company, who is reported to have said “I never got a dime out of [the patent].” And today for the first time I learn from Wikipedia that Lehovec came to the US from the Czech Republic in 1947 through Operation Paperclip. And died just a month ago. Wow. Thanks for mentioning.
Roger Pielke Jr’s blog has an interesting piece on “false positive science” that seems to hew very close to the point I was making about casting around for the data most likely to support the hypothesis:
“The culprit is a construct we refer to as researcher degrees of freedom. In the course of collecting and analyzing data, researchers have many decisions to make: Should more data be collected? Should some observations be excluded? Which conditions should be combined and which ones compared? Which control variables should be considered? Should specific measures be combined or transformed or both?
It is rare, and sometimes impractical, for researchers to make all these decisions beforehand. Rather, it is common (and accepted practice) for researchers to explore various analytic alternatives, to search for a combination that yields “statistical significance,” and to then report only what “worked.” The problem, of course, is that the likelihood of at least one (of many) analyses producing a falsely positive finding at the 5% level is necessarily greater than 5%.”
Re: degrees of freedom. Edward Leamer is an econometrician and empirical trade and macro economist whose most famous article was entitled “Let’s Take the Con Out of Econometrics.” He wrote a book called Ad-Hoc Specification Searches that covered the ground more thoroughly. His advice back then was to take a kind of inverse Bayesian approach–put in many different specifications, see how the results change, and then ask yourself how strong your priors would have to be to make you believe something given the range of results across all those specifications. Famous article title using this method (by Xavier Sala-i-Martin): “I Just Ran Two Million Regressions.”
I think this approach is not a bad one when dealing with lots of degrees of freedom in model choice.
Well, the paper has eked out a ranking
It’s achieved at least some “impact” franking
And is just enough warmist
To calm the alarmist
While giving the Team quite a spanking
Ah, of course there are errors of fact
Disappointing, and lacking in tact
To a skeptic, that’s odd
But at least the facade
Of “pure science” is shown to be cracked!
===|==============/ Keith DeHavelle
On Bishop Hill, thirty-two white stallions,
Gnash & Gnosh in astonishment.
============
One Trackback
[…] An unusual acknowledgement within the academic world « Climate … […]