Schmidt’s recent post on Yamal advocated the following “conspiracy theory”:
McIntyre got the erroneous idea that studies were being done, but were being suppressed if they showed something ‘inconvenient’. This is of course a classic conspiracy theory and one that can’t be easily disproved. Accusation: you did something and then hid it. Response: No I didn’t, take a look. Accusation: You just hid it somewhere else.
One aspect of Schmidt’s response is beyond laughable. I agree that the best way of disarming suspicion is to show data: “take a look”, as Schmidt says. However, if Schmidt thinks that the conduct of the scientists involved in the various data refusals, obstructions and FOI refusals constitutes “take a look”, then he’s seriously in tin foil country. Comical Gav indeed.
Although I find it hard to believe that Schmidt is unfamiliar with the past incidents that gave rise to suspicion that adverse results and data have been withheld or not reported, I’ll review a couple of important ones. These do not, in any sense, constitute an inventory of incidents. They are ones that are either familiar in part to CA readers or which illustrate an important aspect of the problem.
“Dirty Laundry” and Verification r2
In December 2003, despite a number of prior data refusals, I asked Mann for the residual series from the individual steps (termed “experiments”) in MBH98. Unknown to me at the time, Briffa and Osborn had made an almost identical request three months earlier (which Mann had complied with). Residual series permit a reader to carry out standard statistical tests (verification r2, RE, etc) without having to re-do the entire calculation from scratch. I copied David Verardo of NSF on the request. Without waiting for Mann’s refusal (this surprised me), Verardo said that Mann was not required to provide this data to me. Verardo’s letter was later cited in Mann’s evidence to the House Energy and Commerce Committee and in Stephen Schneider’s book.:
His research is published in the peer-reviewed literature which has passed muster with the editors of those journals and other scientists who have reviewed his manuscripts. You are free to your analysis of climate data and he is free to his. The passing of time and evolving new knowledge about Earth’s climate will eventually tell the full story of changing climate. I would expect that you would respect the views of the US NSF on the issue of data access and intellectual property for US investigators as articulated by me to you in my last message under the advisement of the US NSF’s Office of General Counsel.
In response to the identical inquiry from CRU, Mann immediately sent the residual series to Osborn, warning him that the residual series were his “dirty laundry”, provided to Osborn only because he was a “trusted colleague”. Mann asked Osborn to ensure that the “dirty laundry” didn’t fall into the wrong hands, an assurance that Osborn readily gave.
None of the so-called “inquiries” delved into why Mann regarded the residual series as his “dirty laundry” and why he was so anxious to prevent this (apparently “inconvenient”) information from falling into the wrong hands.
One reason might, of course, have been that the residual series would immediately permit the calculation of the verification r2 for each step. (Favorable) verification r2 results for the AD1820 step were illustrated in MBH98 Figure 3; elsewhere MBH98 said that the verification r2 statistic had been considered. But in the SI to MBH98, Mann had archived the RE for each step but not verification r2.
In MM2005, we reported that the verification r2 for the AD1400 step was approximately zero – a very surprising result given the “skill” claimed for MBH98. (Zorita, for one, was surprised by this result and thought less of MBH98 accordingly.) In MM2005 (EE), we expressed our surprise that the results of such a central verification statistic had not either not been calculated or reported.
In Mann’s testimony at the NAS panel in March 2006, Mann was directly asked whether he had calculated the verification r2 for the AD1400 step; Mann flatly denied doing the calculation, saying that such a calculation would have been a “foolish and incorrect thing to do”. However, by that time, Mann had archived part of his source code in response to the House Committee and that code showed conclusively that the verification r2 values had been calculated for all steps.
That Mann had calculated verification r2 results is beyond dispute. That they were “inconvenient” is beyond dispute. That they were not reported is beyond dispute.
Wahl and Ammann announced in May 2005 that all our claims were “unfounded”. Since our codes were very close and I reconciled them almost immediately, I knew that their verification r2 results would be identical to ours. Again, I was asked to review the paper (though my review was disregarded.) As a reviewer, I asked for the verification r2 results. Wahl and Ammann refused. Rather than rejecting the paper, Schneider terminated me as a reviewer. At AGU in December 2005, I asked Ammann what the verification r2 for their AD1400 step was. He refused to answer – a refusal noted by Eduardo Zorita and others.
I asked Ammann out to lunch after the paleo session (I bought). Since our codes reconciled, it should have been possible to clarify the dispute. I offered to jointly (with our coauthors) write a paper stating what we agreed on and what we disagreed on. He refused, saying that this would be “bad for his career”. To this day, I remain dismayed at this answer. I urged him to report the verification r2 results; he refused. I told him that I would not simply stand by while he refused to report the adverse verification r2 results that confirmed ours; he shrugged. I therefore filed an academic misconduct complaint at UCAR; while the complaint was shrugged off without investigation, the verification r2 results appeared in the final article, confirming our point.
The Climategate emails show that Phil Jones, also a reviewer of the paper, was outraged that we had complained about Wahl and Ammann suppressing the inconvenient data, not by them trying suppressing the data.
Jacoby and D’Arrigo
Another equally disquieting incident occurred before Climate Audit and may not be familiar to all readers. This incident also illuminates issues about when data is “used” – an issue that I believe to be relevant to the Yamal incident.
Jacoby and D’Arrigo (Clim Chg 1989), a study of northern North American tree rings, was extremely influential in expanding the application of tree rings to temperature reconstructions (as opposed to precipitation.) (See CA tag Jacoby for prior posts that have been tagged.) The Jacoby-d’Arrigo reconstruction was used in Jones et al 1998 and its components (especially Gaspe) were used in MBH98. It is used to “bodge” of Mann PC1 in MBH99; Mann’s “Milankowitch” argument rests almost entirely on this bodge – ably deconstructed by Jean S here.
Jacoby and D’Arrigo stated that they had selected the 10 most “temperature-influenced” sites from the 36 northern North America (boreal) sites that they had sampled in the previous decade, to which they added Cook’s (very HS) cedar series from Gaspe “because of the scarcity of data in the eastern region”. However, if you pick the 10 most “temperature-sensitive” series from a network of 36 autocorrelated red noise, you will get a HS. This phenomenon has been more or less independently reported by me, Jeff Id, Lucia, Lubos Motl and David Stockwell. We noted this phenomenon in our PNAS Comment on Mann et al 2008, taking some amusement in citing AIG News (Stockwell) since it was unreported in the Peer Reviewed Litchurchur. The phenomenon seems to baffle climate scientists.
Not only did Jacoby and d’Arrigo pick only 10 of 36 sites, they only archived these 10 sites. I asked for the data from D’Arrigo but got nowhere. At the same time, I had also learned that a new Gaspe version had been calculated – one which did not have a HS. (See CA here). I asked D’Arrigo to archive or provide me the data. She refused, saying that the version on file (cana036), which had a huge HS, was a better guide to NH temperature.
In early 2004, Climatic Change did not have a data policy when I was asked to review a submission by Mann et al savaging us. In my capacity of “reviewer”, I asked for the supporting data and code that Mann had previously refused. The late Stephen Schneider, then editor of Climatic Change, said that no one had ever previously asked for supporting data or code in the 28 years that he had edited the journal and that such a request would require a change in editorial policy. The progress of my request is documented in Climategate letters, since Phil Jones and Ben Santer were on the Climatic Change editorial board and both opposed the proposal. (Peter Gleick made his first cameo appearance at this time, also supporting obstruction.) Eventually Schneider adopted a policy requiring supporting data, but not code. Under the new policy, I asked for supporting data for the new submission, which Mann withdrew. (Osborn expresses some annoyance at this in a CG2 email.) This was the first academic paper that I had been asked to review. While I think that Mann knew that I was the reviewer, if I were doing it again, I would only act as an identified reviewer so that any adverse interest was clearly disclosed to the author.
With the benefit of Climatic Change’s newly minted data policy, I asked them to request the missing measurement data for the “other” 26 Jacoby sites. Jacoby refused in a truly remarkable letter, reported on in one of the very first CA posts here. The following is a lengthy excerpt, see the link for the full letter):
The inquiry is not asking for the data used in the paper (which is available), they are asking for the data that we did not use. We have received several requests of this sort and I guess it is time to provide a full explanation of our operating system to try to bring the question to closure…
We strive to develop and use the best data possible. The criteria are good common low and high-frequency variation, absence of evidence of disturbance (either observed at the site or in the data), and correspondence or correlation with local or regional temperature. If a chronology does not satisfy these criteria, we do not use it. The quality can be evaluated at various steps in the development process. As we are mission oriented, we do not waste time on further analyses if it is apparent that the resulting chronology would be of inferior quality.
If we get a good climatic story from a chronology, we write a paper using it. That is our funded mission. It does not make sense to expend efforts on marginal or poor data and it is a waste of funding agency and taxpayer dollars. The rejected data are set aside and not archived.
As we progress through the years from one computer medium to another, the unused data may be neglected. Some [researchers] feel that if you gather enough data and n approaches infinity, all noise will cancel out and a true signal will come through. That is not true. I maintain that one should not add data without signal. It only increases error bars and obscures signal.
As an ex- marine I refer to the concept of a few good men.
…
I was dumbfounded by Jacoby’s response. At the time, I expressed my disbelief as follows:
Imagine this argument in the hands of a drug trial. Let’s suppose that they studied 36 patients and picked the patients with the 10 “best” responses, and then refused to produce data on the other 26 patients on the grounds that they didn’t discuss these other patients in their study. It’s too ridiculous for words.
The incident also sheds light on the question of when data is “used”. I plan to cite this incident in another forthcoming post. No statistician would accept Jacoby’s argument for a minute. By examining 36 series and picking 10, all 36 series were “used”. I find it hard to believe that Jacoby’s position has any traction whatever, but I was unsuccessful in persuading Schneider.
Jacoby’s practices came up unexpectedly in the NAS panel workshop, when Rosanne D’Arrigo told an astonished panel that “you had to pick cherries if you want to make cherry pie” (see here). Although this evidence was highly relevant to the subject of the NAS inquiry and occasioned a flurry CLimategate emails, the NAS panel report avoided the issue entirely.
A related issue arises in respect to the Yamal-Urals regional chronology where CRU examined several versions before reverting back to the very HS-shaped chronology arising from the very small dataset used in Briffa 2000. I’ll discuss this in another post.
Bona-Churchill
In the mining exploration business, investors who trade mining stocks know that “late” results are almost never “good” results. The reason is human nature. In public stocks, you’re legally obligated to report results promptly, but there is some play in timing. If promoters have “bad” results in the first part of the program, there is a great temptation to delay the bad news in the hope that later results will bail out the program. The best and only way to deal with temptations to delay bad results is to establish an announcement schedule ahead of time and stick to it.
In 2006, I noticed that Thompson had swiftly reported Kilimanjaro results, but results from Bona Churchill had not been reported with the same alacrity. (Six years later, Bona Churchill results still have not been published, let alone archived.) My surmise at the time was that the results were “bad” (i.e. did not have elevated O18 values in the 20th century.)
By saying this, I am not saying that climate scientists are less honorable than mining promoters. Only that there are great human temptations to delay reporting “bad” results. And, after a while, delay can turn into neglect, without any explicit decision ever having been made not to report the “bad” results.
In the mining business, promoters are bursting to report good results. I presume that this temptation also affects climate scientists, who, as Schmidt tells us from time to time, are human and subject to human frailties. I remain convinced that climate scientists are more eager to publish “good” results than “uninteresting” ones. Especially when Nature and Science have such an appetite for “worse than we thought” articles to be a mild object of satire even within the “community”.
Six years later, Bona Churchill results remain not only unarchived, but unpublished. At this point, one cannot say that the Bona Churchill results have been “suppressed” for good; but they have clearly been delayed. A graphic in a workshop shows that my surmise was correct: contrary to Thompson’s expectation, 20th century O18 values were not elevated. They are “inconvenient”.
Conclusion
Back to Gavin’s point.
It’s not tin foil country to say that, in respect to Jacoby and D’Arrigo 1989, Jacoby failed to report or archive data that didn’t show what he was expecting. I typically describe this sort of phenomenon in less charged terms than Schmidt: “failed to report” as opposed to “suppressed”. But there’s nothing tin foil about saying that Jacoby was selective in what data he archived – Jacoby said so. One of the many problems in this field is that so many “real” scientists see nothing wrong with this.
Similarly with Mann’s residuals (“dirty laundry”) and verification r2. And Thompon’s Bona-Churchill.
I find it hard to believe that Schmidt seriously believes that climate scientists have reacted to suspicion by saying “take a look”. The pages of Climate Audit are replete with one incident after another, where climate scientists have taken precisely the opposite attitude.
That includes the present case. The sane resolution of the present FOI request would have been for East Anglia to just send me the data, even if they felt that they didn’t “have” to. The best way to remove suspicion would have been to say: “Here, Steve, take a look”. If Schmidt thinks that that’s what CRU has done in this case, then he’s the one in tin foil country.
Climate Audit 
152 Comments
“McIntyre got the erroneous idea that studies were being done, but were being suppressed if they showed something ‘inconvenient’. This is of course a classic conspiracy theory and one that can’t be easily disproved. Accusation: you did something and then hid it. Response: No I didn’t, take a look. Accusation: You just hid it somewhere else.”
So just show your sensitivity tests…if you did any. And iy you did not do any sensitivity tests, at least allow us to say that you failed to perform sensitivity tests.
Gavin is losing his marbles.
Comical Gav isn’t even able to tell what is and is not a “conspiracy theory”. Conspiracy implies active participation in a secret endeavor. The CG and CG2 emails have demonstrated instead that a group of activist climate scientists simply share the same “forma mentis” (“circling the wagons”), probably naturally self-selected as anybody with an inquiring mind would be quickly isolated from the group.
OTOH Gav’s friend Mike has been repeating his own belief in the Big Oil Conspiracy, for years and years with even less evidence than a HS. Mann’s is of course a classic conspiracy theory and one that can’t be easily disproved. Accusation: you are paid by Exxon. Response: No I am not, take a look. Accusation: You are such a Big Oil brainwashed puppet, you work for Exxon for free.
The drug trial analogy from the events of the Jacoby and D’Arrigo 89 paper is very apt:
(My paraphrase) – “I analyzed 36, and picked 10. Therefore – I only used 10 and so the other 26 were not used, relevant, or available for review”.
I cannot think of any other field in academia or industry where this would be judged as either acceptable or normal practice.
Efficacy in clinical trials is judged from two points of view: 1) The outcome for patients taking drug or placebo who complete the full course of treatment. 2) The outcome for the patients one “intended to treat” because they met the qualifications for beginning the trial. Patients who are failing a course of treatment and suffering side effects are more likely to drop out of a clinical than those who assume they are receiving some benefit in return for suffering side effects. The FDA requires both types of analysis so that the influence of dropouts is clearly identified.
If early work at a site shows an absence of unprecedented warmth or divergence with modern samples, researchers may choose to not go back to find enough samples to adequately characterize the possibility of unprecedented warmth of the MWP. The 36 sites where work was completed and published could be the survivors of 50 or 100 sites where work was begun.
Gavin and Team made their hat-choices neat
But isn’t it odd at this junction
That “foiled again!” for them is not defeat
But more of a wardrobe malfunction?
The data and code would make this a non-issue
We’re reasonable people! Just try us!
But all this behavior seems thin as a tissue
That wraps cherry-flavored source bias
This could be quite funny if so very much
Was not at stake for populations
“To see what we did? Oh, no, you cannot touch!
Just have faith in our proclamations!”
===|==============/ Keith DeHavelle
It is extremely helpful to have such summaries provided. Mahalo nui loa!
You may be the only one or one of the very few on the planet that is sufficiently knowledgeable and competent to provide such insight, certainly with such alacrity. We all owe you big time. Please enjoy some Squash this weekend.
As a lowly independent analytical chemist, this type of stuff just floors me again and again. In our little corner of the science world, even we have policies and procedures and also formal ethics codes to prevent this type of crap non-science arising from massaging the data. The data is the data. All of it! It really burns my backside. And they wonder why there is such a backlash.
As a political scientist I can tell you that you would not have gotten away with the nonsense like this even in our glorious discipline. 🙂
Ivan –
“Glorious” is tongue-in-cheek, I do presume.
Having seen the results of enough pre-election polls, I would disagree somewhat with you. Essentially, none of the polling firms presents their algorithms for taking raw polling data and turning it into their published numbers. Granted, those algorithms are their “trade secrets”, and they don’t want to reveal their methods. At the same time, there is a good amount of competition for polls, so the public has multiple polls to refer to. At the same, time, scores of billions of public dollars are not being expended directly as a result of any single poll. Bias is always suspected in election polls, so skepticism is part of the polling industry, and no polling company shrieks and whines when someone challenges their output.
In many polls I’ve seen
The process is exposed
The weighting (fat or lean)
By ratios proposed
And justified by count
Or detailed in assumptions
(Though some of those amount
To “in-your-face” presumptions)
It’s in their full reports
(Equivalent to papers)
Unlike the Climate sports
Who obfuscate their capers
===|==============/ Keith DeHavelle
What makes this a bit tricky is that there can be valid reasons for excluding some data from statistical analyses. Even in the drug trial case, a mistake might have been made in determining the dose for a patient, or they may be some other legitimate error. In non-experimental settings, of course, there may be many more reasons for excluding some data simply because one does not have as much control over how the data were generated. We do have “outliers” that should be excluded if we are to have a more reliable analysis. The standard practice, however, is to explain the grounds for excluding data and report the results with and without the data included so the reader can make an informed judgement about whether the results are reasonably supported by the evidence.
Peter, Hypothetical – it might be an important addition to note in the pharmacy trial that 5 results were discarded – because the test subjects had died from the test. Yes, negative responses and explanations should be documented.
“We do have “outliers” that should be excluded if …”.
That is of course a normal procedure as the outlier might exhibit some errors. In the tree ring community they specifically use these outliers from a large data set to enhance a specific feature: encreasing temperature. Then they expect us to rely on their judgement that the vast majority av measurements were wrong and should not be used (archived). What we have left is in IPCC words: overwhelming majority of scientific articles strengthens the view that treerings show warming.
As an ex-medical rep, I’ve read through dozens of clinical papers.
If a patient’s data has been excluded from the final analysis, for whatever reason (Non-compliance, unrelated illness, “lost to follow-up”, etc) these reasons are stated in the paper.
Ideally, selections should be made while blind
And, quite early on, be given exclusion
A sudden ring spike, or stats of this kind:
Do near cores or near trees show bark extrusion?
Do trees in this local spot all trend together?
This call should be made without knowing the temp
Show the data reversed, unlink time and weather
So no “mismatch to theory” could make them exempt
So if these selections are made by blind judge
Based solely on match to site-synchronous style
And then used, we’re happy. ‘Till then we won’t budge
For only transparency here makes us smile
===|==============/ Keith DeHavelle
Isn’t Gavin’s point that the data that you say shows an ‘adverse result’ or a ‘bad signal’ is actually just bad data and cutting-room floor material… Stuff that would not make selection muster? I think in this regard isn’t it possible to see both sides of the coin for what it looks like to you and looks like to them?
So then, the key would be to assess the dendro literature for the credence to the idea that you simply get a bunch of core samples from wherever you deem appropriate, then look at the data, and simply throw out any one that doesn’t equivocate the modern instrumental record for the area. If this indeed is the way the literature accepts– then what are they doing wrong if they don’t report the crap left-over (which undoubtedly occurs in other experiments across all fields of science)?
Perhaps this means that the literature itself should change to reflect a study that says, “Hey… if just 10% of regional cores happen to fit the instrumental temperature record in the last century (the rest of their trail being immaterial), then what does this actually say about the validity of the exercise in the first place? Isn’t it possible (if not more likely) that these 10% may actually be false-positives randomly ascending?” …What in the literature proves that they aren’t, other than just saying so?
I have this feeling that the regional Yamal/Ural dendro study is d-e-f-i-n-i-t-e-l-y going to employ this same methodology (because it’s in the literature with a track record of general acceptance). They’ll comb through all the records and identify all cores that equivocate the last centuries instrument record in the region, and then produce that chronology…It’s impossible that those cores will not ‘confirm’ the recent temperature rise, and virtually impossible that their pre-instrument records will be anything other than a spaghetti-paste of cancelled out mediocrity– they certainly couldn’t select cores that simply confirm some diminished MWP or something, because they don’t even have an instrument record to back it up and it would just be a circular-proof exercise.
Since this methodology is out there in the dendro literature, it’s going to be acceptable practice, until there are papers out there that challenge it. It looks like blog posts aren’t going to do it when it comes to reforming the selection methodology of dendrochronology.
The negatives of a test are as important if not more important than the positives in science – or any rigorous discipline. Failure to consider these data points sets the table for spurious results – in this case hockey sticks. The tactic of ignoring all unconforming data Jacoby and D’Arrigo study appears to be the core of detroclimatology.
Indeed. It’s understandable not to show negative results in a paper as you can only cover so much, but there is no justification for not making the data those results came from available.
The flaw in the argument is that they are not excluding a tree because the patient died or was noncompliant, but because it did not show a correlation with temperature (or maybe a negative one), but the question at issue is DO TREES RESPOND TO TEMPERATURE IN A CONSISTENT WAY? So they are presuming the hypothesis they are testing and artificially inflating their good results in this way.
Very correct, Craig. Such post-hoc selection criteria are the equivalent of putting the cart before the horse, or, logically, a circular argument. The conclusion is part of the premiss.
Mark
Craig –
Correct. There are TWO levels of weak/bad science here. On one hand is the insufficiently vetted underlying correlation. That is the big picture non-solidity. On the other – more specific – hand is the pre-filtering of data in a non-transparent way.
It is 100% intellectual dishonest to present a paper that leaves out 26 of 36 available data sets without informing the editors and the public of those 26 and why they were left out. There is nothing else to call it but dishonesty. The public ASSUMES that all the data is included; why would they think otherwise? Ditto the editor, who may be from another discipline altogether and not even know of the existence of such additonal data sets.
But the reviewers are really the crux of this. It is the reviewers who should be adequately informed about the existence of other data sets and who should be asking, “Where are those other data sets?” Review is supposed to be a gauntlet. It is the reviewers who should be a true gauntlet. So, at bottom, what we have here is a failure of the peer-review gauntlet. If the gauntlet was working, Jacoby and Osborne and the rest would KNOW that they can’t get away with it.
It is important to point this out and keep on pointing this out: It is the journals who are letting the scientific world down. Yes, it is the authors who are trying to cheat, but if the umpires see them cheating (or refuse to pay attention), the players will cheat till the cows come home. If they simply made everyone follow the rules, none of this would be happening. But when there is overlap between the reviewers/umpires and the scientists/players, this sort of thing was going to happen in some discipline. And the smaller the population in any discipline, the more overlap – meaning the more potential for corruption.
The obvious argument about this is to make active scientists ineligible as reviewers. Outsiders cannot know enough to review properly, so the general public can’t do it. But there is one knowledgeable group that could definitely do it: Retired scientists in the field. Being much more out of the politics of academia, they would be much freer to disagree; their careers would not be at risk.
Isn’t Gavin’s point that the data that you say shows an ‘adverse result’ or a ‘bad signal’ is actually just bad data and cutting-room floor material… Stuff that would not make selection muster? I think in this regard isn’t it possible to see both sides of the coin for what it looks like to you and looks like to them?
That may be his point – but it seems to be via “trust me”.
In order to show it is “bad data” you want to preserve both the data and selection method to demonstrate why and how you determined that.
snip – no need to editorialize about dendro
I think the point that Salamano is missing is that the decision to exclude certain of the series was made on the statistical analysis alone. Of course there is “just bad data and cutting-room floor material” (as Salamano puts it), but determining what this bad data is based on a statistical calibration is inappropriate. To extend Steve’s analogy to a drug trial: it is proper to exclude patients from a clinical trial based on observed physical characteristics, e.g. too old, too young, preexisting heart condition, and even in rare cases, not the target ethnic group. However, it is not acceptable to exclude the patients based on a statistical analysis of the data. To try get that past the FDA / FDA advisory committee is just laughable. Salamano is just wrong when he claims this approach “undoubtedly occurs in other experiments across all fields of science”.
I sincerely hope Salamano misspoke when he claimed that this methodology is in the literature with a track record of general acceptance.
Let’s not confuse “null” data and “bad” data like Jacoby did. If I have series of say, 20 data points, and 2 of them are outliers, I might be justified in tossing them out (with proper written justification, of course). But if I give my statistical treatment to 36 data sets and kick out all but 10 as being “bad” or outliers, that has to tell me something about the assumptions used in the data analysis, shouldn’t it? If I found over 10 of my original 20 data points to be bad, then I would have to assume soemthing was wroing with my technique. I would call those other 26 “nulls” that did not prove my hypothesis instead of calling them “bad.” As nulls they should be reviewed to yield information about the veracity of the method. As a reviewer, I would feel that I was not getting the entire story if that information were not being provided.
Damn, I thought I had written something about proper sampling techniques before I went overboard and sniped at what I thought would be considered horrible practices even at undergrad level.
Sampling is critical in statistical analysis because the statistician is trying to infer properties of the whole based on relatively few peeks at the data. Usually, sampling is done either periodically or randomly such that the analysis picks a population or the samples that probabilistically is close to representative of the whole. The larger the sample size the greater the chances of picking a representative sample. Selection criteria is usually applied to create a representative sample of a subset of the whole. Again, it is crucial to create representative population/samples.
For Jacoby, the selection criteria is to create a temperature record from data points selected based on temperature record. It’s like seeing hockey stick after trying their best to pick out the hockey blades.
Excluding outliers is sometimes attempt to reduce error from the possibility of including instrument error driven extreme data points. The number of these points should be small and even if the statistical analysis were to include these data points, the outcome shouldn’t move so much. In typical statistical analysis, the conclusion would not be sensitive to one or two data points regardless its absurd value. Exclusion of few outliers is not the case when removing 26 of 36 data samples. Those outnumber the 10 used by Jacoby and would have dominated his outcome had they been included.
Briffa has demonstrated that they will used outliers in their reconstruction with YAD06 as a good example of an extreme hockey stick outlier sample that was selected to be part of the Yarmal reconstruction.
It would indeed be nice if treering chronologies were chosen that reflect local temperature records.
But the fact is, Briffa’s Yamal treerings were flagrantly not reflecting local temperatures.
OK, but then you hit the “Divergence issue”, as was done a decade ago, where the proxies selected are checked going forward and found to have no correlation after the original selection point…at which point we hit “Hide the Decline”.
“However, if you pick the 10 most “temperature-sensitive” series from a network of 36 autocorrelated red noise, you will get a HS. This phenomenon has been more or less independently reported by me, Jeff Id, Lucia, Lubos Motl and David Stockwell.”
Is this the situation where you select those signals that end with rising values (as per the instrumental record) but the rest of the signal is not validated. Averaging these non-validated sections tends to produce a flat line which added to the validated rising end gives a hockey-stick?
If it’s something else, could someone point me to an explanation that is aimed at laymen?
You’ve got the right situation.
The Air Vent has explanations about hockey sticks from noise in a series of clear, detailed, yet concise posts.
Ouch! That had to hurt. One of these days the hockey team is going to learn:
“You don’t tug on Superman’s cape
You don’t spit into the wind
You don’t pull the mask off that old Lone Ranger
And you don’t mess around with Steve Mc”
The only part that isn’t bloody is the soles of the GISS man’s feet.
Breaking news in Oz. A new proxy study shows that the last 5 decades in Australia are the warmest of the millenium. It’s promoted as Australia’s contribution to the 5th IPCC report in 2014. I wonder if all of the dirty laundry, err data, will be archived.
None of the proxies are on the actual Australian continent.
All the data is up at the PAGES site. Further it is Australasia not australia strictly speaking
This is just the first wack at the data. see the web page
I think the word Australia and Australian appears over 30 times in the first part of the manuscript even though none of the proxies are actually on the continent.
A lot of the proxies are NZ tree rings. I’ve looked at the paper briefly, and it looks like mannian hockey-o-matic to me, they even use the same period (1921-1990) for calibration and verification (!?). They add an additional “early verification period” 1900-1929, but the pre-1600 data doesn’t seem to verify at all against that period (they report low RE scores in figure 3).
I hope you have some time to take a look at this, Steve, my alarmists friends are already celebrating the second coming of the hockey stick…
Steve – I hope that you would issue a press release on the death of the Hockey Stick.
So, 26 out of 36 samples were allegedly worthless (although they don’t explain why for each one) and they expect us to believe them?
Steve’s analogy with how the shadier portions of the mining industry operate is very apt. We had a long-standing column in the Australian Financial Review about Blue Sky Mining N/L written by Pierpont (Trevor Sykes) that is eerily familiar. It featured Bottle, the geologist; Sir Mark Time, the clueless but superficially respectable company chairman; Spender, the accountant – you get the idea.
Steve, you would have enjoyed this column, which lifted the lid on every mining scam known to speculators.
Here is the final column, linked to the archive of other articles:
http://www.pierpont.com.au/article.php?Ten-final-tips-from-Blue-Sky-Mines-34
Another pearler is “Blue sky gives birth to a biotech” here:
http://www.pierpont.com.au/article.php?Blue-Sky-gives-birth-to-a-biotech-23
Quote from the biotech column:
What happened was that Spender discovered Professor Algae, a microbiologist who has spent 40 years peering down a microscope and tends to squint a lot when you bring him into daylight. The downside is that after half a century looking at laboratory slides, all he had found was a molecule that might cure cancer in tapeworms. This means that if you take a drug based on his molecule, your tapeworms will be healthier and live longer, but we plan to play that down in the prospectus.
The important word in there was cancer. It is no use discovering something that cures rare diseases. To succeed on the stockmarket, a biotech has to offer a cure for one of the big market glamour ailments such as cancer, heart, stroke or AIDS. Hair restorer would be better still. So Spender had to do some in-depth research of his own.
Spender: Now this molecule you’ve discovered might just possibly might cure cancer?
Algae: Honestly, I don’t think . . .
Spender: . . . Cure cancer in human beings?
Algae: . . . There’s no chance . . .
Spender: Would you like to drive a red Porsche?
(Pause)
Algae: . . . I’m sure it just needs a little more research funding.
————-
Many of the techniques have resonance to those following another field characterised by lots of puffery, speculative investment and money for nothing to unscrupulous operators.
“What makes this a bit tricky is that there can be valid reasons for excluding some data from statistical analyses. Even in the drug trial case, a mistake might have been made in determining the dose for a patient, or they may be some other legitimate error. In non-experimental settings, of course, there may be many more reasons for excluding some data simply because one does not have as much control over how the data were generated. We do have “outliers” that should be excluded if we are to have a more reliable analysis. The standard practice, however, is to explain the grounds for excluding data and report the results with and without the data included so the reader can make an informed judgement about whether the results are reasonably supported by the evidence.”
Exactly. But You/We/They are ethically bound to publish ALL of the data. No hide and seek allowed. And provide a full narrative along with the full data set on what your methods did or did not include and WHY. Had this concept drilled into me in a no-name undergrad school, and pounded into me at the professional level. I have ZERO respect for the “climatology” field based on this selective and secretive approach. Unfair, I know, due to a handful of unethical rogues.
I’ve just had a look at data for the tempertature station closest to Yamal with long term data. Salehard is at 67N 67E (Yamal is at 71N 71E) and has data from 1883 to the 2011. The observed temperature shows no sign of a ‘hockey stick’ and is much closer to Steve McIntyre’s reconstruction.
http://www.climatedata.info/Discussions/Discussions/opinions.php?id=3283134733594426905
Ron:
Doc Martyn found this very interesting paper yesterday.
“Posted May 15, 2012 at 7:41 PM | Permalink | Reply
Steve. This is a paper on the temperature, precipitation, snowfall and much else for Yamalo-Nenets AO, Sibiria,
…
Click to access Ealat_Yamal_climaterep_dvs-1.pdf
The figures are mind blowing, especially Figure 7.”
Thanks for that.
For me the part of figure 23 showing annual temperature anomalies for 4 stations was the most interesting. In terms of highs and lows they are all synchronous with global temperature trends but in this very rural region the late 20th century and early 1940s peaks are almost identical whereas in the global record the later peak is much higher than the earlier one.
Could this be due to the absence of an urban hear island effect?
“I am not saying climate scientists are less honorble than mining promoters.”
It is apparent some climate scientists have the same ethics as mining promoters.
Don B,
I read your comment as honorable to both climate scientists and mining promoters. If, however, you meant a derogatory statement, then you are ethically bound to hand back all of your property that was derived from mining. Please don’t rubbish without proof/examples.
I should not have insulted all mining promoters. I had in mind the rough and tumble days of the Vancouver Stock Exchange, and the promotion of penny mining stocks. A book of interest is “Fleecing the Lambs.” 🙂
Don, if my comment above about mining speculators ever gets out of moderation (it contains links to Pierpont’s superb columns about scams in the Australian mining industry), one of them, ‘The slowest oil well in Australasia’ (Oct. 2003), says inter alia:
“The main shares in InterOil are traded on the Toronto Venture Exchange (similar to the old Second Board here) and on the Nasdaq pink sheets, an unregulated market.
The volumes traded overseas are not great but they appear to be stronger than in Australia and so the movements in Toronto and New York look as though they drive the share price on the ASX.
In other words, the price of InterOil seems to be driven by Canadians, whom Pierpont regards as the West Australians of the northern hemisphere.”
Both Pierpont and Steve know that the mining industry has plenty of cowboys. Pierponts’s own company, Blue Sky Mining N/L (no liability except at gunpoint) vowed never to mine dirt, only the stock market. Cleaner, more profitable, and very little chance of going to jail for theft. Sound familiar?
snip OT
Geoff, one of the main points of Pierpont’s columns is that the regulatory framework enables grand theft. By the time the cops arrive, there is nothing left but tumbleweeds. Almost no-one ever goes to jail. That is why I think that Steve’s background in mining makes him singularly attuned to misrepresentation and pea-and-thimble tricks.
The story is the same whether you are claiming you’re going to pull gold out of a new mine or stuff ‘carbon’ down an old one. The market price of carbon is a sure indicator of confidence in the operators. That’s a decline which can’t be hidden.
Young Steve
I see you’re back in fine form. This is gritty stuff! Since you’ve started this thread, beginning with Yamal, I’ve been forwarding your posts to Lawrence Solomon, Donna LaFramboise, Lorne Gunter, Lorrie Goldstein as well as others.
It’s difficult to imagine that these folks aren’t reading your posts regardless of my efforts, but still I see nothing in the papers or on the tube.
Fellow readers, PLEASE get off your butts and encourage sympathetic editors/columnists to report on young Steve’s brilliant auditing, or more to the point, his results.
snip
This must be keeping Gavin busy from 8 to 4. He is not giving up his day job.
The analogy with mining is a prescient one with the usual provisos. A well established mining industry with proven methods of exploration when along comes a new bunch of mining entrepreneurs who reckon they know better how to discover gold in them thar hills and they’ve got the core samples to prove it, so pile on on investment folks. Hmmm…interesting, go the mining establishment, busily eking out a living on proven mines and exploration techniques, where and how exactly did you data miners get those impressive core results from? Oh ye of little faith they cry holding up their glittering core samples. Don’t you worry about that and pile on everybody.
If you don’t believe in sampling you wouldn’t want your doc to drain all your blood for a blood test, but by the same token I don’t want my doc draining most of my blood to scientifically extract all the cancerous cells to proudly announce that I definitely had a new form of cancer but luckily for me he had the cure.
Typo: there should be an active link for “here” in:
“At the same time, I had also learned that a new Gaspe version had been calculated – one which did not have a HS. (See CA here).”
“Jacoby failed to report or archive data that didn’t show what he was expecting.”
Climate scientific method appears to be quite a bit different from the generic scientific method. In climate scientific method the hypothesis is determined and then data that does not support the hypothesis is discarded. In The Model we trust.
Anyhow, back to the tree ring circus. Fantastic work.
Here’s a quote I just stumbled across, from Bertrand Russell’s Understanding History (1957), p. 37:
“There is nothing genuinely scientific in a premature attempt to seem scientific.”
It’s called “Accentuate the Positive”
http://www.nature.com/news/replication-studies-bad-copy-1.10634
Good article. I thought this was amusing:
Re: theduke (May 17 09:16),
That tongue-in-cheek paper is quite serious. They make the proof, then show what the process failure is and make recommendations for modifications to the modern science process!
It’s available online here.
The paper is by Simmons, Nelson and Simonsohn. The line from the acknowledgements section near the end:
“All three authors contributed equally to this article. Author order is
alphabetical, controlling for father’s age (reverse-coded).”
made me chortle, and is a subtle demonstration of the point of the paper. Especially since they could have claimed sorting by first name to get the same result.
Re: theduke (May 17 09:16),
Actually, the conclusions of that article may well be worth a separate, thoughtful thread:
Table 2. Simple Solution to the Problem of False-Positive Publications
Requirements for authors
1. Authors must decide the rule for terminating data collection before data collection begins and report this rule in the article.
2. Authors must collect at least 20 observations per cell or else provide a compelling cost-of-data-collection justification.
3. Authors must list all variables collected in a study.
4. Authors must report all experimental conditions, including failed manipulations.
5. If observations are eliminated, authors must also report what the statistical results are if those observations are included.
6. If an analysis includes a covariate, authors must report the statistical results of the analysis without the covariate.
Guidelines for reviewers
1. Reviewers should ensure that authors follow the requirements.
2. Reviewers should be more tolerant of imperfections in results.
3. Reviewers should require authors to demonstrate that their results do not hinge on arbitrary analytic decisions.
4. If justifications of data collection or analysis are not compelling, reviewers should require the authors to conduct an exact replication.
Global Warming: Forecasts by Scientists versus Scientific Forecasts* Green & Armstrong, Energy and Environment, Version 43 – July 10, 2007
“Accentuate the Positive” was excellent. Thanks for posting the link.
Re: Sera (May 17 00:02),
It looks like “Accentuate the Positive” is describing a phenomenon in publication journals and the peer review system. It’s describing a situation where publication of scholarly papers tilts towards positive findings. This creates pressure for publication of dubious results, promotes uses of dubious methodology that increases the chances of positive findings, and provides temptation to engage in research fraud. If you look at the “Accentuate the Positive” chart, Environment/Ecology & Geosciences don’t have too much of the case of positive results.
The flip side of “Accentuate the Positive” is that over time a culture develops has an aversion to replications, confirmations, or refutations. Scratch beneath the surface of a published paper and too often you will find poor experimental design or non-replicable results. This does sound oddly familiar to the stonewalling reception to queries for data during Steve’s attempt to replicate the hockey stick reconstructions.
The most important discussion is of experimental methods and experimenter bias. It is ironic that psychology which first exposed experimenter biases to the scientific world is so afflicted by such practices. Of course, there are cases of borderline misconduct (see my bold.)
I’m confused. Why the hubbub over tree rings? As near as I can tell, their ability to archive temp data to anything resembling the needed precision to generate/extrapolate a model designed to look decades into the future, is impossible, there are simply too many variables into what makes a tree grow…moisture, temp, competition, disease and I harbor the same doubts about ice cores, corals, and tarot cards…
As one of my profs said. “Your answer had better not have more precision than your least precise measurement….”
Keep up the good work Steve.
Like a surgeon delicately carving away the dead flesh to expose the diseased organ !
As I understand the situation from the admittedly small number of academics I know the standing of an academic, and the instution they are aligned with, depends greatly on the number of published papers.
As such, I can understand to a certain extent the reluctance to do anything that might adversely affect one’s ability to publish in the future (like publish data sets that may reveal useful new information).
But (and this is the big one for me), if Mann, Jones, Briffa et al honestly, genuinely believed in the quality and integrity of their work and honestly, genuinely believed that it demonstrated dire outcomes for the planet they would publish EVERYTHING.
Hell… they would have their data tattooed to their backsides and run naked in front of the White House/Houses of Parliament to make sure they got the greatest attention for their work.
Instead they obfuscate, prevaricate and flat-out lie to avoid releasing their raw data, metadata & code. To me, at least, that speaks volumes.
Right on, Hamster!
Maybe O/T: has anyone ever compiled a list of the really influential climate scientists/alarmists that includes the members of the Team and, perhaps a few others? I am under the impression that it’s probably less than twenty individuals.
Angry Hamster says, “But (and this is the big one for me), if Mann, Jones, Briffa et al honestly, genuinely believed in the quality and integrity of their work and honestly, genuinely believed that it demonstrated dire outcomes for the planet they would publish EVERYTHING.”
We know, from the climate gate e-mails, that the team DOES NOT “honestly, genuinely believed in the quality and integrity of their work”
Bradley:
I’m sure you agree–the Mann/Jones GRL paper was truly pathetic and should never have been published. I don’t want to be associated with that 2000 year “reconstruction”.
Cook:
I am afraid that Mike is defending something that increasingly cannot be defended. He is investing too much personal stuff in this and not letting the science move ahead.”
Cook goes on to propose a single unifying GRAND climate reconstuction paper by the entire team of “climate scientist” and then states how little he thinks they will learn from it.
“…Without trying to prejudice this work, but also because of what I
almost think I know to be the case, the results of this study will
show that we can probably say a fair bit about 100 year variability was like with any certainty i.e. we know with certainty that we know fuck-all.
Of course, none of what I have proposed has addressed the issue of
seasonality of response. So what I am suggesting is strictly an
empirical comparison of published 1000 year NH reconstructions
because many of the same tree-ring proxies get used in both seasonal
and annual recons anyway. So all I care about is how the recons
differ and where they differ most in frequency and time without any
direct consideration of their TRUE association with observed
temperatures.”
Wow, now having trashed all past and future reconstructions as junk, and stating he does not care about there “true” association with temperatures, Cook the desribes how to make these useless reconstuctions appear credible in the next IPCC report…
” I think this is exactly the kind of study that needs to be done
before the next IPCC assessment. But to give it credibility, it has
to have a reasonably broad spectrum of authors to avoid looking like
a biased attack paper, i.e. like Soon and Balliunas.”
The reason for all the obstuction is clear.
“It is error only, and not truth, that shrinks from inquiry.”
— Thomas Paine
Briffa, Osborn and Melvin have a current grant from NERC of £231,441.00, to study dendro divergence, according to the CRU website.
Here is NERC’s policy on access to research outputs:
Well, that was drafted by a lawyer. There are more loopholes than in a crocheted rug.
“The phenomenon seems to baffle climate scientists.”
The phenomenon is intuitively obvious, I would think, even to Team scientists. You can however explain yourself out of this if you simply assume that any proxy which correlates well with the instrumental record will correlate *equally* well to temperature back in time. This assumption is apparently an axiom within climate science, with Mann’s sausage-machine as the ultimate manifestation.
Suggest you add Lonnie to your first Thompson reference above to help newcomers.
The logic is that you only want trees that respond to temperature. They never document that the trees ONLY respond to temperature, but in addition, they assume that if it responds to temperature now it will have done so in the past. Is this true? No. Case 1: a stand is currently well-drained soil and growing well, but 200 yrs ago a beaver pond made the site soggy and they were growing poorly irrespective of the weather. Case 2: currently the trees are widely spaced and growing well, but 200 yrs ago the stand was crowded until insects killed half of them. During the crowded period ring widths were narrow until the insect attack. Case 3: when young the trees were widely spaced and grew well for 100 yrs, the stand became crowded and they grew poorly for 200 yrs, then gradually trees died and they grew better. Case 4: (documented in a recent paper): the oldest trees in an area are genetically slower growing than average (because slower growing live longer) so a sample of trees will show faster recent growth than long ago growth because only the slow growing trees are the old ones. Case 6: trees also respond to precip (duh) and precip changes do not correlate with temperature changes over 1000 yrs.
In no case does the screening of trees to pick only those that correlate with 20th century temp prevent these problems.
Craig:
What is the mysterious Case #5 that you have purposefully hidden from us? 😉
The first rule about Case #5 is you don’t talk about Case #5
Your general point is taken Craig, without quibbling over some of the examples used (for example, it’s unlikely somebody would pick a microsite potentially subject to inundation or other large changes in water table).
The other big potential issues are:
(1) exactly how calibration is done, in conjunction with the detrending method used and
(2) the non-linear response issues you raise in your 2009 paper
Dendro work is like an inverted pyramid. A single assumption is made (that treemometers that correlate will with temp today always did do and that all other factors such as precip can be idngored) and then the pyramid is built resting on that point. this assumption has never been verified. In my 2007 paper I asked primarily: do tree rings agree with a composit of other proxies, and the answer was NO–which is wy I was labeled as such a bad guy, because I was undermining the entire enterprise, not just getting a different result. Doing an analysis based on an untested assumption can be interesting, but eventually you need to test such simplifying assumptions–especially after 30 yrs of high impact work. That this is unclear to dendros does not speak well for their reasoning abilities.
Craig,
I am a bit in the middle on this. While I agree that teasing temp from dendro is a rather weak methodology, I have been astonished at the fidelity with which some trees seem to track temperature. Of the problems you list above, most can be resolved, at least theoretically, by using age standardization curves. I would venture that the beaver pond scenario can be discerned from comparison to a standardized curve, and furthermore, if it is discerned, it should be thrown out from the studied population. Where I part ways with D’Arrigo is that I strongly believe that the paper should still describe the tree that was thrown out, and explain exactly why it was thrown out.
Please let’s not have another discussion on dendro.
That is not what’s at issue in this thread.
I was trying to ignore the pissing contest and focus on the much more interesting topic of data withholding. Your drug trial analogy is well wide of the mark. The main thrust of the counterargument was that the withheld data was “data without signal,” not merely data they chose to leave out of a paper. An analogy to a drug test would go like this: the hospital identifies 40 cancer patients willing to participate in a drug trial. The drug being tested is administered and the patients are tracked. At some point, the hospital informs the trial doctor that ten of the patients on the list do not actually have cancer. The paper is published on the remaining 30 patients with cancer, and you ask “but what happened to the cancer in the other ten?” That is what Jacoby and D’Arrigo are claiming.
Steve: All the data sets yield “chronologies”. The data sets were rejected not because they couldnt calculate a chronology, but because the resulting chronology wasn’t among the most “temperature sensitive”. Ex post screening.
I am missing the distinction. At least some dendros believe that some trees record temperature and that the data can be extracted in a meaningful way, while some skeptics such as Craig L. believe otherwise. Everyone on both sides of the argument believe that some trees do not record temperature accurately enough to be of use. Reasonable folks can disagree about how papers should address these trees. But about this we should all agree: the only reason to not archive and post the processed-and-rejected chronologies is fear that others will second guess you, and that type of behavior should be shunned by all those who wish to remain true to the scientific method.
The distinction is,
40 cancer patients are in the study, at some point its found that 10 do not have cancer. Paper published on basis of 30 cancer patients.
40 trees have growth rings, at some point its found 10 trees go not have growth rings. Paper published on basis of 30 trees with growth rings.
Post normal science at its best.
Re: Matt Skaggs (May 17 08:57),
Has anyone done research wherein they have set up accurately calibrated thermometers directly adjacent to a particular specie of tree now being used as climatological temperature proxy, and have then compared the actual temperatures being experienced by that particular specie of tree to its measured tree ring widths and tree ring densities — doing so for perhaps a decade or longer to ensure that reasonably sufficient observational data has been gathered over some representative period of time?
Has the immediately proximate contextual environment of these same trees also been characterized sufficiently over time so as to gain some insight into how that particular specie of tree might respond to local environmental conditions — sunlight, precipitation, soil conditions, the presence or absence of other trees in close proximity, etc. etc?
Without wishing to go off topic here (and mindful of Steve’s request), I think there was a recent report of a paper that did this in Norway – only the experiment was set up to investigate the effect of sheep grazing and temperature was logged as a controlling variable. Can anyone else remember this?
There’s no doubt that this test has been done
When the various FACE projects run
(Free Air Carbon Enhanced)
But it’s rare that they’ve chanced
To show that particular one
Since they measure CO2 (and raise)
And measure each variable’s phase
Temp, ozone, water vapor
Calibrating that caper
If published, would sure deserve praise
===|==============/ Keith DeHavelle
Scott,
The answer is “yes,” quite a lot has been done in that area, especially at timberline in the alps. This is a good place to start:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3047780/
Well, I do seem to recall a graph by Mr. McIntyre comparing a data-set used prominently in Mann’s studies that showed a similar trend to a local temperature gauge, but varied widely in detail…with an extreme of the warmest year in the 20th century showing as the 6th? coldest in 150 years.
Further, when it was brought forth that a lack of coherence between proxies and their known local temperatures didn’t match up well, the Team put forth the idea that these trees were instead responding to a Global Climate Signal. An interesting idea, but for which there is not a known mechanism for why a tree in Colorado ignores its local environment, but responds faithfully to conditions aligning with the average of some data-sets continents away.
I don’t recall the series of studies being too forthcoming about the lack of coherence between a used proxy and known measured temperatures.
“doing so for perhaps a decade or longer to ensure that reasonably sufficient observational data has been gathered over some representative period of time?”
That will only give you information on the relationships **over the measured time period**. Such information is helpful over short time scales, but we already know that most trees respond to temperature at short time scales–this is readily evident using the existing, archived tree ring data sets at the ITRDB. Even among sites suffering badly from divergence over recent decades this is sometimes true (though not sure on the frequency).
One of the biggest problems in tree ring analysis is getting the long term (aka “low frequency”) changes pegged correctly. And short term relationships with the environment, even *extremely* strong ones, will not necessarily do that. This gets into the whole detrending issue and it gets real tricky real fast.
It all comes down to the characterization and selection of “temperature sensitive” tree based on correlation of tree rings to the modern temperature record. It becomes circular logic.
Why are these “temperature sensitive” trees? Because tree rings match well to the temperature records.
Why are the tree rings good indicator of temperature? Because the tree are “temperature sensitive.”
There has to be a physical explanation for the selection of “temperature sensitive” trees. The explanations should be tested in a controlled experiment.
There are mountains of such evidence from controlled and observational experiments of all kinds. That secondary (radial) tree growth responds to the environment, including temperature, is beyond all dispute whatsoever. What is at issue are, rather, questions of the response of particular species in particular environments at particular time scales.
Evidence is there for response of trees in general to environmental factors of which temperature is one. If we are considering only temperature that is a whole lot of confounding variables.
The evidence is lacking for picking out a much smaller subset of trees as the “temperature sensitive” group and to using them as good indicators of temperature for the entire during of a chronology. This procedure of identifying “temperature sensitive” trees is as Steve says ex post screening.
What we need here is a good understand of physical properties that make up the “temperature sensitive” tree. Trees suffering from divergence are particularly helpful because they provide cases where the tree is either not sensitive to temperature or other environmental factors are overwhelming the temperature signal.
All these “non-temperature sensitive” trees provide wonderful opportunities to better understand the relationship between tree growth and temperature. It does no service to science to declare it bad data like Jacoby, not document tree selection criteria like Briffa, and ignore them during analysis.
The obvious solution here is to genetically engineer a species of tree that is a good thermometer. Maybe it could be set up with an external readout so no cutting would be necessary.
Often people mention the analogy of testing a drug on patients and then excluding those patients from the final report which did not respond as expected to the drug.
It is however important to understand where the dendros are coming from. If the purpose of the reconstructions was to establish whether trees are a reliable temperature proxy then indeed it would not be appropriate not to include all data. But remember, they assume temperature correlation a priori and then quickly move on to the reconstruction part – you see, you just have to filter out the bad proxies using expert judgement (or Mann’s sausage-machine). As far as I can see this is seriously how the thinking goes.
In any case, the drug test analogy is invalid – they are already prescribing it.
SteveM shows here a number of good examples of what is wrong with the proxy selection process as used in practice. I think those examples speak much louder than any personal, indirect as they are, exchanges between Gavin and Steve. Taking Gavin’s route of questioning motivation is a distraction from the real issue. At this point I can only think that the those defending the past selection processes do not have much to offer in counter to the criticism. I suppose some can hold out hope for this grand paper promised that will do right by the selection process. Based on the current reactions or lack of reaction to SteveM’s current and past sensitivity tests of the selection process I have strong doubts.
Besides what SteveM shows by example here it is informative to look at the individual proxies actually selected for use in published reconstructions as those proxies do not paint a very consistent picture of past or current temperatures – even though the picture delivered to the public comes across differently.
Exactly right, it is very hard to disprove when they wont disclose the data.
Study: WORLD IS GOING TO BURN UP, MAN TO BLAME!!!
Accusation: This looks a little off, can I see your data.
Response: No.
Accusation: Why won’t you let me take a look?
Response: {crickets chirping}
Accusation: Ok then, I will oficially request it.
Response: {crickets chirping}
Accusation: (years later after the FOI or EIR appeals finally release the data) You did this part wrong and hid the data
Response: That is an old study and doesn’t matter anymore. This new study (WORLD IS GOING TO BURN UP, MAN STILL TO BLAME!!!) shows the same results as the other one, so you are wrong.
Accusation: Well if the old one was wrong with new data, and the new study looks like the old one so it still looks a little off, can I see your data?
Response: No.
… repeat over and over again.
The Wegman report stated that the “paleo” community did not have any interaction with mainstream statisticians.
If they did Mann and co would realise that by selecting data to reflect a hypothesis is a classic way to generate a Type 1 statistical error, also known as a false positive (a result that indicates that a given condition is present when it actually is not present.
This is something we teach our first year undegraduates.
Of course Mann is enough of a mathematician to know this, but steadfastly refuses to admit that what he is doing is simply poor science as the rewards (Professorship/Tenure/grants/fame) are far too great.
Michael Mann knows exactly what he’s doing. And Gavin Schmidt knows exactly what Michael Mann is doing and aggressively defends him.
I used to show a movie to my students titled “Where did the Colorado go?” One person interviewed was Gordon Jacoby who was part of study to determine flow patterns from runoff using tree rings to determine precipitation patterns.
Stockton, C.W., and G.C. Jacoby. 1976. Long-term surface-water supply and streamflow trends in the Upper Colorado River basin based on tree-ring analyses. Lake Powell Research Project Bulletin 18: 1-70.
It was part of larger study on the Colorado River Compact to resolve disputes over water use by the Supreme Court.
http://wwa.colorado.edu/treeflow/lees/stockton.html
http://www.thefreelibrary.com/Water+wrongs:+why+can't+we+get+it+right+the+first+time?-a0115697385
I think it a little disengenuous to try to argue a rationale for hiding the decline by way of selection or application of some obscure ‘suitability’ criteria.
In order to have any confidence in the application of the technique in the past – one has to have SUPREME confidence of application of the technique in the modern. And if the entire population aren’t moving in concert with the modern record – there is a problem with the entire field of research.
If that is not the case, how would I know that I picked the 5 (or 10) special, magic representative trees that faithfully recorded the MWP? I cannot, and would of course be engaging in confirmation bias. They might call it ‘signal’ and try to extract it against some performance criteria – but how would they know?
It seems to me that this whole tree thing would only would work if it were the trend of the entire population – and the constraints on the growth are easily demonstrated to be something other than nutrients, water, sunlight, etc.
In deciding to exclude data, any reviewer would need to know why the data was excluded.
To evaluate that decision, he would need to see the data so that he could come to the same decision independently, or otherwise.
The claim then, that the data was not used, is completely spurious. It was used because it was considered and then rejected. The reason for rejection is open to review.
In Steve’s drug trial analogy:
“We didn’t use the other 26 because the participants died. We only used the survivors to prove that the drug is beneficial.”
Wow – the logic is breathtaking!
“If you want to make cherry pie, you have to pick cherries!”
This is hilarious. If a scientist actually uttered those words he/she should be drummed straight out of the Brownies.
It’s a bit like a company choosing the 10 best prices out of 36 months and using the results in their next prospectus: the only problem being that the missing 26 months show the share price wildly oscillating and then crashing. Dishonest doesn’t quite cover it.
Can’t we have these jokers disbarred from science altogether and start again or is the scientific method in climate science completely corrupted beyond all repair?
Re: Jim (May 17 13:31),
The D’Arrigo cherry pie incident was reported at CA ( here).
Thanks, Steve. I didn’t doubt it was said, I was simply incredulous that it had been. I wasn’t really aware that some of the research was quite so blatantly doctored as this. I’m not sure that the general public is really aware of it either. If they were, there’d be even more skepticism than there already is.
The real stunner, Jim, is that D’Arrigo would not have said that had she not thought it was methodologically sound.
Indeed. The more I think about it, it seems to me that this is true of the whole shebang: the hockey stick; climate models; “hide the decline”; etc. They are simply fitting the data to suit the theory. Anything inconvenient, like energy balance of Trenberth’s travesty email fame is swept under the carpet.
I have personal experience of the dangers of scientists who’ve assumed that they have arrived at the necessary conclusion of their research but, in fact, they were far from it. In this case it was medical scientists and their mistake could’ve killed me where it killed countless others. Scientists who actually believe they full understand something are fools and they are a menace to the general public. They should never stop checking that their understanding of things is correct and never, ever say they are settled.
Jim you will find ‘incredulous’ is very common reaction. The whole basis of Gavins RC-style argument is that a real ‘climate scientist’ can examine datasets, perform an ad hoc pre-screening process to pick a sub-group which have a ‘real’ temperature signal. Then treat the selected group by standard, and non-standard, statistical analysis.
Which reminds me again–and, I suppose, we should each copy this out and Post It note it on our computer monitors–of Feynman’s famous quote: “The first principle is that you must not fool yourself, and you are the easiest person to fool.” And of course the reason each of us is the easiest person to fool is that we have an emotional/egoistic–and probably unconscious–interest in the rightness of our own work or cause.
Being a relative newcomer to Climate Audit, I have just read “The Cherry Pie” story for the first time. Absolutely incredible. I think it might be worth re-running this as a head post for the many readers who may not be aware of it.
No. It would be like an economist picking the 50 stocks most correlated to the market, and try to reconstruct past market volatility from those stocks.
‘he’s seriously in tin foil country.’ no its just in full scale and bitingly ironic , denial .
When you start from the anti- science stance that all things that support you are good and all things that don’t are bad , as Schmidt does , the behavior starts to make sense. For like much of ‘the Team’ you need to spell science, r.e.l.i.g.i.o.n to understand why they make the claims they do .
Reblogged this on Climate Ponderings.
see the article on this: http://pjmedia.com/blog/the-death-of-the-hockey-stick/?singlepage=true
Some of these guys really aren’t very bright. They won’t show their cards, then get all upset because people don’t trust them. They need to find some other line of work, preferably one that does not require openness.
“I offered to jointly (with our coauthors) write a paper stating what we agreed on and what we disagreed on. He refused, saying that this would be “bad for his career”. To this day, I remain dismayed at this answer.”
I found this to be the most disturbing part of your article. It reads like the response of a scientist who’s actions and words are governed by fear. Did it seem like that to you at the time [or now, with hindsight]?
Is Gavin Schmidt just bad at sums?
The IPCC confirms that all the warming since 1850 is ~ 0.7°C and asserts that this warming is wholly due to Man-made CO2 emissions. A trivial check sum can be done by translating percentages of the ~33 °C Greenhouse Effect into °C for each active constituent.
The abstract of the NASA GISS paper http://pubs.giss.nasa.gov/docs/2010/2010_Schmidt_etal_1.pdf states:
“Attribution of the present‐day total greenhouse effect
……….. With a straightforward scheme for allocating overlaps, we find that water vapor is the dominant contributor (∼50% of the effect), followed by clouds (∼25%) and then CO2 with ∼20%. All other absorbers play only minor roles. In a doubled CO2 scenario, this allocation is essentially unchanged, even though the magnitude of the total greenhouse effect is significantly larger than the initial radiative forcing, underscoring the importance of feedbacks from water vapor and clouds to climate sensitivity.”
Gavin A. Schmidt, Reto A. Ruedy, Ron L. Miller, and Andy A. Lacis
Transposition of the above values to °C of greenhouse effect is as follows:
Water Vapour and Clouds ~75% ~24.75°C
Other Greenhouse Gases ~25% ~8.25°C
Other non H2O non CO2 GHGs gases (calculated according to CDIAC) ~1.2% ~0.41°C
Carbon Dioxide at 390 ppmv ~7.84°C
Natural CO2 280 ppmv (100% emissions since 1850) x 280/390 ~5.63°C
Man-made CO2 (full increase since 1850 Man-made 110 ppmv ) x 110/390 ~2.21°C
As the reported and acknowledged temperature increase since 1850 is known to be only ~0.7°C in total, how can this result be possible. Thus at 2.21 °C past Anthropogenic Global Warming is exaggerated to be more than three times the acknowledged temperature rise since 1850.
Clearly neither Gavin Schmidt nor his peer reviewing colleagues carried out this trivial check sum before publication. Had they done so, they would have seen that these give a gross exaggeration of Man-made influence on temperature even from past CO2 emissions.
All other published proportional data start out with water vapour and clouds accounting for ~95% of the greenhouse effect.
Nonetheless those promoting the alarmist “Cause” expect the Western world to revolutionise its economies based on this type of assertion and calculation. This is the type of trivial due diligence that seems never to be undertaken in the Alarmist Global warming camp. Instead radical and vastly expensive policies are formulated to address Catastrophic Man-made Global Warming. Inaccurate assertions of this nature have been widely accepted by governments.
These are the climate experts that World Governments via the UN IPCC depend upon and on which the Western world is basing its self-destructive and costly policy decisions.
Steve, my thoughts on this are self indulgent and too long so I posted it elsewhere. http://patrioticduo.blogspot.com/2012/05/one-reason-this-layman-doubts-some.html
Years ago I was advising someone on statistics. They were doing an allometric regression trying to predict leaf area from length. They had a pretty nice relationship but there were a cluster of outliers. He asked if he could just drop those “bad” points. I said only if there is something actually wrong. He went back to the data and found that these and only these points were leaves with insect damage where he had tried (and obviously failed) to estimate how much area was missing from each leaf. I said yes, drop those and say you only used undamaged leaves.
By this type of criterion, strip bark trees are obviously growing in an unusual way due to damage and should be dropped. The Yamal trees look like a case of conversion from shrub to upright growth form, and should likewise be dropped (esp since they are multi-sigma outliers). Instead they do the converse and use them repeatedly.
The fact that they will not publicly state how/why they dropped sites/trees is quite damning. It is either an untested assumption (per my comment above) or simply data snooping.
Craig: your comments have been very helpful to my understanding of the issue. Thank you.
This captures the real point of the flaw in Mann’s use of PCA. A priori, any PCA is as likely to find a cluster of anomalous behavior as it is the signal that is of interest. In fact, part of the reason for ensuring an actual physical meaning to the signal is to rule out factors that emerge simply because a sub set of the cases share some other feature. Mann’s PC1 should have been labeled Strip Bark BCPs to minimize any confusion or misinterpretation.
Lucia recreates the hockey stick, showing the methods behind dendrochronology. If you haven’t seen this, worth a read.
http://rankexploits.com/musings/2009/tricking-yourself-into-cherry-picking/
1. Steve did something similar and got similar results.
2. It seems to me that there is a step missing in most paleoclimatology studies. You start with a set of samples from trees, lake sediments, stalagmites etc. . You then identify a subset of samples which correlate well with observed temperature for the last 100 to 150 years. This is a perfectly reasonable first step but still does not prove that the samples remain valid indicators of earlier temperatures. I suggest that a further step is necessary. It should be shown, using the full period of data, that members of the ‘accepted’ group are from a different statistical population to those of the ‘rejected’ group.
Hmmm. No mention of the “CENSORED” folder? That’s a pretty clear example of studies being done, but being suppressed if they showed something ‘inconvenient’. The only wiggle room is whether Mann himself ran the unreported robustness checks or another member of his team.
Old, broken link to original discussion: https://climateaudit.org/?p=21
Steve McIntyre, posted on Dec 10, 2004 at 2:16 PM
Hmmm. No mention of the “C*NSORED” folder? That’s a pretty clear example of studies being done, but being suppressed if they showed something ‘inconvenient’. The only wiggle room is whether Mann himself ran the unreported robustness checks or another member of his team.
Old, broken link to original discussion: https://climateaudit.org/?p=21
Steve McIntyre, posted on Dec 10, 2004 at 2:16 PM
“…then he’s seriously in tin foil country.”
I am going long on aluminum foil.
Moderator/Steve, it seems that the filter is grabbing comments containing the word “CENS*RED”, which I think is a very relevant example.
Something I have often wondered about, is how much effort it would take to satisfy the average sort of request mentioned above? 10 mins and an email? If say Phil Jones had satisfied all the requests made of him, how much time in total would this have cost him? (And while we’re at it, lets not forget to compare this to how much time it would *save* climate science as a whole, esp those like Steve)
IOW, to what extent could time and money be a valid excuse for non-disclosure?
One of the big issues in management research is precisely these sorts of data-mining or cherry-picking issues. We tend to call the fallacy “sampling on the dependent variable” and it’s kind of a first-cut litmus test to see if people are serious scholars or not. For example, Jim Collins’s books have sold many, many copies and have a patina of science (using matched groups of firms, for example) but he resolutely will not stop sampling on the dependent variable (comparing successful and unsuccessful firms and looking for differences) so nobody smart pays much attention to his findings.
There’s a book called The Halo Effect by Phil Rosenzweig that tries to explain this (and related) problems to the broader public. I haven’t read it myself, but the reviews are pretty good.
I know what white noise is, but have never heard the term ‘red noise’ before. Can someone explain that?
It just found the answer. Red noise = Brownian motion.
Hi, Jim!
Although the standard definition of red noise is apparently the same as Brownian motion (see http://en.wikipedia.org/wiki/Colors_of_noise ), in climate discussions it is often used to just mean a series with heavy first order serial correlation. So don’t take it for granted what is meant when you see the term being used.
Here I get to show my possible ignorance: White noise, I believe, is that which may be generated by a series of random numbers, while red noise is that which can be generated by ADDING randomly generated numbers in sequence to the original random number. (I hope this makes sense. One may see how this “adding” effect equals Brownian motion, this being Hu McCulloch’s techy “heavy first order serial correlation”. White noise “starts all over again” with each number. Speaking of techy phrases, what do biologists call animals such as lions, tigers, pandas? Answer: “charismatic megafauna”. It’s true.)
If you just add random numbers, you get a “nonstationary” Brownian motion that has no tendency to revert to where it started. But if you multiply what you have by .9 before you add each new random number, the series will be very persistent, but will still tend to return toward zero if it gets very far from zero and therefore be “stationary.” In these climate discussions, the latter type of series is sometimes loosely referred to as “red noise,” though EEs might disapprove.
Because of this potential confusion, I think it’s best to avoid “red noise”, and just say “Brownian Motion” or “persistent AR(1) process”, depending on what you mean.
A nice, hot cup of tea?
I bet the Team was sorry to lose Peter Glieck as an ally. He is probably the only one in that coterie that has any hair…on top, that is.
Hair is overrated.
Such are the problems when the collection of data and the interpretation are in the hands of the same people.
Spurious Correlation + Confirmation Bias=The Whole Shebang
Hi all,
I’m looking for the reconstructed values for Cook et al 2004. Can’t seem to find them online – have any of you had access to them ?
Robert,
I have not been able to find them. If you do, I’d me most interested in your ourcome as there has been a reference to this work in a 2012 paper on Australasia. sherro1 at optusnet dot com dot au
Haven’t been able to find it… quite unfortunate
Hmmm. No mention of the “CENS*RED” folder? That’s a pretty clear example of studies being done, but being suppressed if they showed something ‘inconvenient’. The only wiggle room is whether Mann himself ran the unreported robustness checks or another member of his team.
Apparently CENS*RED gets caught in the filter even when quoting Mr. McIntyre.
If this was a prize fight they would stop it. But we can’t trust the judges to score it legit. Steve; you MUST score a KO……… it’s the only way.
Steve: I assume you have seen the article in National Review online written by Montford and Ambler, with a big old picture of you.
New article from National Review.
http://www.nationalreview.com/articles/300877/climategate-continues-andrew-montford
The truth about globull warming and record temperatures:
Indianapolis 500 records
Highest Race Temperatures
Races with air temperatures equaling or surpassing 90°F (32°C)
Year Degrees Race Winner Notes
°F °C
1937 92° 33° United States Wilbur Shaw
1953 91° 33° United States Bill Vukovich
With anecdotal, “unofficial” testimony placing air temperature at the track during
the race near or surpassing 100°F / 38°C, potentially the hottest race in history,
with at least one fatality, United States Carl Scarborough, due to heat exhaustion
1919 91° 33° United States Howdy Wilcox
1978 90° 32° United States Al Unser
1977 90° 32° United States A.J. Foyt
Note 96°F / 35°C, claimed for the start of the 2010 race, but subsequent data reviews indicate an inaccurate reporting
Coldest Temperature at Start of Race:
51°F / 11°C, 1992
References
^ National Weather Service archives for Indianapolis, up to 26 May 2012.
It seems to me the argument is that if data was considered and discarded, the discarded part of the data too becomes a part of the study. Therefore, that data too should be released. Others do not believe this.
It seems to me data that was considered is part of the study, and ought to be released, along with the criteria for discarding. Is that the crux of the current argument?
It is not fair to contrast what Schmidt claims against reality.
Gavin Schmidt and his cohort have absolutely poisoned the entire discussion on AGW. His attitude is thoroughly negative with almost everything dealing with this.
I hope the new Administration simply fire the entire lot of them.
6 Trackbacks
[…] Rosanne D’Arrigo […]
[…] haven’t updated the prospectus because as Steve McIntyre points out in his latest post on Schmidt’s “Conspiracy Theory”: In the mining exploration business, investors who trade mining stocks know that “late” results […]
[…] And a rebuttal to critics: Schmidt’s “Conspiracy Theory” […]
[…] produce key supporting pillars of the catastrophist position, such as the famous Hockey Stick. As recently as yesterday, the catastrophists were loudly publicly defending this work (and defending withholding the data […]
[…] https://climateaudit.org/2012/05/16/schmidts-conspiracy-theory/#more-16091 […]
[…] that the university depended on for future research funding. See a discussion here. Here is a set of similar stories of data and results being […]