Michael Kottek writes in the comment section:
The results of my FOI request to the University of Melbourne can be seen here:
I requested all correspondence between the authors and the journal regarding the paper. The referees reports were exempted as were documents relating to the resubmitted paper.
I also requested correspondence between the authors after the paper was accepted. Once again emails relating to the resubmitted paper were exempted, and personal material redacted.
I note that emails regarding the paper that were received by one author and not forwarded to the others would not have been covered by my request.
Despite the embarrassment of the withdrawn paper, the University is to be commended for their no nonsense approach to this request. As an alumunus, I am pleased that the response is far more sensible than the approach taken by the UEA and UVa.
Chronology (Steve: Oct 28, 9 pm Eastern)
Here is a more detailed commentary which raises questions about Karoly’s claim that they had “independently” discovered the screening error on June 5. [Note: times in the emails are in multiple time zones. In the analysis below, it is my understanding that in June 2012, relative to UTC, Melbourne time was +10, Switzerland +2, Eastern -4, CA blog time -5.]
May 31
As CA readers are aware, the issue of screening in Gergis et al 2012 was first raised in a CA post in a May 31 blog post, a discussion that directly quoted the following paragraph of Gergis et al:
Our temperature proxy network was drawn from a broader Australasian domain (90E–140W, 10N–80S) containing 62 monthly–annually resolved climate proxies from approximately 50 sites (see details provided in Neukom and Gergis, 2011)… Only records that were significantly (p<0.05) correlated with the detrended instrumental target over the 1921–1990 period were selected for analysis. This process identified 27 temperature-sensitive predictors for the SONDJF warm season (Figure 1 and Table 1) henceforth referred to as R27.
The CA discussion, and, in particular, the May 31 Name and Shame blog post, was referred to on numerous occasions in the internal emails among Gergis, Neukom and others over the next few days, commencing almost immediately with an email from Gergis to Neukom and other coauthors we follows:
We should all be aware that this is unfolding: https://climateaudit.org/2012/05/31/myles-allen-calls-for-name-and-shame
In my original May 31 post, I had presumed that Gergis et al had used correlation screening against trending series (which I termed the “Screening Fallacy”), a topic discussed on a number of occasions at critical climate blogs (see references in original post.) Against this, Jim Bouldin and others argued that Gergis et al had employed detrending screening, thereby avoiding the CA criticism.
June 6 (Australia Time)
The CA discussion quickly led to Jean S checking the correlations of the available series.
At 07:42 AM June 6 Melbourne time (blogtime- June 5 16:42), Jean S reported (CA comments here) that Gergis’ claim that they used detrended correlations for screening was false (asking me and others to check). Within an hour, Jean S’ comment attracted online notice from Hu McCulloch (blog 17:39; and Kenneth Fritsch (blog 18:19).;
About two hours after Jean S’ post – by now nearly 2 am in Switzerland (June 6 09:46 Melbourne; 01:46 Switzerland; blog 18:46 June 5), Neukom urgently notified his Australian associates of the same problem that Jean S had reported at CA a couple of hours earlier. Neukom had a skype discussion with Gergis, followed up by an email (2Gergis, page 77) to Gergis, Karoly and others. Neukom noted that the mistake was related to the proxy screening (then under discussion at Climate Audit) and thus a “delicate issue”:
As just discussed with joelle on skype, I found a mistake in our paper in journal of climate today. It is related to the proxy screening, so it is a delicate issue. In the paper we write that we do the correlation analysis for the screening based on detrended (instrumental and proxy) data, but in reality we did not use detrended data.
Meanwhile at CA (blog time June 5 20:11; Melbourne 11:11), CA reader HaroldW reported that he had confirmed Jean S’ results. I checked in at CA with a question to Jean S (blog time 20:49 June 5; Melbourne 11:49).
June 7 (Australia time)
The following morning (10 am June 6 blogtime; June 7 01:00 Melbourne), I reported that I had confirmed Jean S’ results, posting the discussion as a fresh post a little later (12:01 June 6 blog time; 03:01 June 7 Melbourne).
Five hours later (05:56AM June 7 Melbourne; 14:56 blog time), then late evening June 6 in Switzerland, Neukom wrote an assessment of situation to Karoly and expressed his desire to discuss matters with Karoly the following day. One hour later (06:48AM June 7 Melbourne; Switzerland June 6 22:48), Karoly wrote back to Neukom urging use of detrended data for their calculation (as stated in article):
I think that it is much better to use the detrended data for the selection of proxies, as you can then say that you have identified the proxies that are responding to the temperature variations on interannual time scales, ie temp-sensitive proxies , without any influence from the trend over the 20th century . This is very important to be able to rebut the criticism is that you only selected proxies that show a large increase over the 20th century ie a hockey stick.
The same argument applies for the Australasian proxy selection. If the selection is done on the proxies without detrending ie the full proxy records over the 20th century, then records with strong trends will be selected and that wi ll effectively force a hockey stick result. Then Stephen Mcintyre criticism is valid. I think that it is really important to use detrended proxy data for the selection, and then choose proxies that exceed a threshold for correlations over the calibration period for either interannual variability or decadal variability for detrended data.
Another hour later (08:03 June 7 Melbourne; 00:03 June 7 Switzerland), Gergis asked Neukom whether he was “250% certain” of the problem. The emails are then surprisingly quiet through the rest of June 7 (Australia).
June 8 (Australia)
Early in the Australian morning of June 8 (06:47AM Melbourne; 15:56 June 7 blog time)), Karoly emailed Neukom and others, referring them to the CA post of about 27 hours earlier (12:01 June 6 blog time; 03:01 June 7 Australia; 19:01 June 6 Switzerland):
Someone has now tried to reproduce the screening of the 27 selected proxies against the target Australasian temp series and is unable to reproduce the claimed results in the paper. https://climateaudit.org/2012/06/06/gergis-significance/. I suggest that you look at this Stephen Mcintyre post. Given that the error is now identified in the blogosphere, we need to notify the journal of the error and put the manuscript on hold
Although the CA post had cited Jean S’ results of June 5, Karoly disregarded these links back to the original provenance.
Within an hour (07:26 AM June 8 Melbourne), Gergis (2G:37, p 112; 2K:31) acknowledged Karoly’s email. At 08:24 AM Melbourne (00:24 Swiss), Gergis wrote to Neukom, but did not copy Karoly or other coauthors; she argued to Neukom that they had emails showing that they “became aware of the issue” prior to the “latest blogpost” because they had “contacted authors for permission to release their records”:
Hi Raphi, we have emails that predate this latest blogpost that indicate we became aware of the issue as we contacted authors for permission to release their records
CA readers will recognize that Gergis here is sliding over a couple of issues: the request for data had come before the detrended correlation issue had arisen. She had only asked authors to release records because of the May 31 CA blogpost in which screening had already been made an issue; Jean S’ results were reported in comments on June 5; my June 6 CA blogpost reported and linked to Jean S’ results reported the previous day.
Neukom immediately wrote back (08:26 June 8 Melbourne; 00:26 Swiss) warning his coauthors that caution needed to be taken with detrended correlations.
A little later (08:42AM June 8 Melbourne; 00:42 Swiss), Neukom sent Gergis a reconstruction with the (only) eight proxies that passed detrended correlation. Karoly quickly noted (08:54AM Melbourne; 00:54 Swiss) that some of the correlations were now flipped!
Throughout the rest of June 8, Karoly and Gergis started notifying others of the problem.
At 10:38 AM Melbourne (02:38 Swiss), Gergis (2K:34; page 73) notified coauthors Gallant and Phipps (cc Neukom) of the problem. In this first notice, Gergis said that Neukom had identified the problem on the morning of June 6 (presumably referring to Neukom’s email that had been received at 09:42 AM June 6 Melbourne (01:42 Switzerland):
Following on from my attempt to gain permission to release non publically available records released and submitted online with NOAA over the weekend, on Wednesday [June 6] morning Raphi discovered an error in the Aus2K temperature analysis….
Meanwhile, Stephen Mclntyre and co have located the error overnight (I was alerted through an intimidating email this morning): https://climateaudit.org/2012/06/06/gergis-significance . So instead of this being a unwanted but unfortunately normal part of science, we are likely to have an extremely negative online commentary about our work. Although it was an unfortunate data processing error, it does have implications for the results of the paper. We wish to alert you to this issue before the paper goes into final production.
Although Gergis refers here to a supposedly “intimidating email” (and uses the same phrase to Journal Climate later that day), no such email is included in the FOI emails. Nor did I send her any such email. Gergis also inaccurately notified her co-authors that “McIntyre and co have located the error overnight [June 8]”. In fact, the error had been identified at Climate Audit nearly two days earlier.
At 11:16 AM (Melbourne), Gergis sent Karoly a draft notice letter to the Journal of Climate. Karoly reverted at 11:47AM with his edits presumably those shown in the redlined version (2K:35, page 75). Karoly’s version stated that they had discovered the error as occurring on June 6:
While attempting to release non-publicly available records used in our study with NOAA over the weekend [June 2-3], our team discovered an error in our paper.. .
When we went to recheck this on Wednesday [June 6], we discovered that the records used in the final analysis were not detrended for proxy selection, making this statement incorrect.
Back-dating to June 5
Soon afterwards (12:35 Melbourne), Gergis sent a revised notice letter to Journal of Climate. In the revised letter, the discovery date was now said to be Tuesday, June 5 rather than Wednesday June 6, as stated in the draft letters. This back-dating ostensibly coopered up their claim to discovery priority indpendent of Climate Audit (but is not supported in their own correspondence). Gergis et al also inaccurately told Journal of Climate that Climate Audit had identified the error “overnight [June 8]”, more than 2.5 days after the actual time of the original report. Their letter stated:
While attempting to release non-publicly available records used in our study with NOAA this week, our team discovered an error in our paper….
When we went to recheck this on Tuesday [June 5], we discovered that the records used in the final analysis were not detrended for proxy selection, making this statement incorrect…
Meanwhile, independently of our team’s detection of this error, prominent climate change blogger Stephen Mclntyre has identified the issue overnight (I was alerted through an intimidating email this morning): http:l/climateaudit.org/2012/06/06/gergis-significance. So instead o(this being a unwanted but unfortunately normal part of science, we are likely to have an extremely negative online commentary about our work and possibly the journal.)
At at 14:19 Melbourne, Gergis sent a near identically worded notice to PAGES 2K again saying that they had discovered the error on June 5 (Tuesday), adding a warning to the PAGES 2K consortium that they might have to archive all their data”
In terms of the consortium paper, please run with the current version of the Aus2K temperature reconstruction but please note that it may change in coming weeks…
They are now demanded that the full network of records be made available. Over the past week I have been busy contacting authors of non publically available records that were not used in the final temperature reconstruction to attempt to release their data. Everyone managed to agree on just the C20th portions used for calibration be released, but some still no not want to make their full records available.
This issue has implications for other 2K groups: ANY mention of proxy ‘screening’ or selection criteria is likely to be heavily criticised . Although we attempted to be transparent about our methodology, this has backfired and caused a lot of trouble. I just thought you should be aware that it may not be enough that only the records used in the final analysis are already available. It is possible that every record from every region {those rejected from the analysis and those used in final reconstructions) will need to be made available once the consortium paper is published.elp benefit the broader group.
During the Melbourne afternoon of June 8, Karoly worked with University of Melbourne public relations staff on a statement, sending a draft to Gergis and others at 15:57 (2K:38); Gergis reverted at 16:17. This statement adopted the date of “Tuesday 5 June” as the date on which the error was discovered:
While the paper states that “both proxy climate and instrumental data were linearly detrended over the 1921-1990 period”, it was discovered on Tuesday 5 June that the records used in the final analysis were not detrended for proxy selection, making this statement incorrect. Although this is an unfortunate data processing issue, it is likely to have implications for the results reported in the study. The journal has been contacted and the publication of the study has been put on hold.
At 17:56 Melbourne (09:56 Swiss), Karoly sent Neukom a short and long version of their statement. In the long version, they neutrally said that the error was discovered on “Tuesday June 5” (without attribution); no date was mentioned in the short statement. Karoly said that they planned to send a statement to me containing the above paragraph. Neukom reverted immediately (18:18 Melbourne), suggesting that they include the date in the short statement: (2K, 168).
Maybe we can include the date when we discovered the error also in the short statement so that it is clear that we did not just do it as a reaction to the Mclntyre blog?…
And I will try to write down everything that happened in the correct chronological order to be sure l can recall this all correctly. Because I think it may be interesting for some people to see how the error and its discovery developed and when/how we (re-)acted.
Neukom also requested that his work be checked:
I think all the analysis needs to be replicated by someone else (maybe Ailie or Steven) to make sure all other errors I made can be identified and eliminated.
Karoly (18:36) reverted to Neukom that he would put the date in the email to me, but doubted that I would “accept that we didn’t find the issue without his help, but that doesn’t matter”. Karoly additionally asked Neukom to keep “good records” of what happened.
I am about to go home and have some dinner, then I’ll send this to McIntyre, so that he gets it Friday morning. Melbourne Uni wanted as little detail in the short statement as possible. l’ll put the date in my email to McIntyre, which he will likely post, as well as the short statement. I doubt that he will accept that we didn’t find the issue without his help, but that doesn’t matter…
Please keep good records of what happened when, and what you did. Also, keep any records of emails you receive from McIntyre or other bloggers. Joelle is being sent hate emails.
If the FOI release is complete, while there are some critical emails, none appear to me to be fairly classified as “hate mail” – the term “hate mail”, as used by climate scientists, appears to include anything that is merely critical. Karoly sat on the notice to me overnight and sent me an email the following morning Melbourne time with the same paragraph. Karoly additionally noted that participants at CA had “also” identified this “data processing issue”:
We would be grateful if you would post the notice below on your ClimateAudit web site. We would like to thank you and the participants at the ClimateAudit blog for your scrutiny of our study, which also identified this data processing issue.
I reported this at the time, with eyebrows more than somewhat quizzically raised at the Gavinesque coincidence that, after months of peer review and after acceptance of their paper, they had supposedly “independently” discovered the error in screening on June 5 – the very day that the precise error was spelled out at CA (though the issue of Gergis screening had already been discussed for a few days.)
The removal of the Gergis paper had been noted in a comment at RC (June 8 15:50 blog time; June 9 03:50 Melbourne). Another RC commenter pointed out to Schmidt that the problem had been discovered at CA:
Gavin – you ought also to mention that the problem was discovered at the Climate Audit blog
Mann appears to have contacted Karoly soon afterwards, as, within 10 minutes of sending this email to me, Karoly forwarded the email to Mann, with a covering note that the comment at RC about removal was correct. Even though Karoly had told Mann about the error, Mann reverted to Karoly that Mann and the other RC authors would falsely tell RC readers that they had “no further information” on the retraction of the paper from the journal website and that he would involve Schmidt and Steig in the plan:
We have simply noted at RC in the comments that the paper does appear to have been retracted from the AMS website, and we have no further information as to why. I will share this w/ Eric and Gavin so they know the status,
Mann also made defamatory remarks about me to Karoly:
Well I’m afraid Mclntyre has probably already leaked this anyway. I probably don’t have to tell you this, but don’t trust him to behave ethically or honestly here, and assume that anything you tell him will be cherry-picked in a way that maximally discredits the study and will be leaked as suits his purposes.
Karoly pointed out to Mann (2K:55 11:19 Melbourne) that there was discussion at CA of the announcement here. Karoly told Mann that they had a “fully-documented” record demonstrating their priority over CLimate Audit:
PS We do have a fully-documented record or who, when and how the data processing issue was identified by a member of the author team independent of, and before, any posts on this issue at CA or other web sites.
Needless to say, no such “fully-documented record” was disclosed to Michael Kotteck.
Climate Audit 
179 Comments
From ‘Part 2b Gergis.pdf’, email from Gergis to d’Arrigo 2 June 2012 (ignoring the strange references to a “smear campaign” supposedly started by McIntyre):
It would seem to me as they have jumped the gun on using some of this data for the Gergis et al publication. If it cannot be released because it’s still being developed, or some student is still working on a first publication about it, where is the quality control? Shouldn’t the raw data be subject to basic analysis before being included in a multiproxy study?
“The detrending of proxy records had been done in another paper on Southern Hemisphere temperature variations that we had been writing simultaneously, so we wrongly assumed the same thing had been done i the Australasian paper. REDACTED>>>>this was not picked up until now.”
Very curious what would have been redacted. From June 8 email Gergis to Chiang.
Indeed. I’m sure the words “climate” and “audit” are in that redaction somewhere.
Well, McIntyre and or Jean S not likely mentioned in redaction as the its noted below that the issue was (coincidentally) discovered at CA overnight.
They missed one of the redactions – within e-mail hand labelled 49 of Part 2b Gergis.pdf:
The detrending of proxy records had been done in another paper on Southern
Hemisphere temperature variations that we had been writing simultaneously, so we wrongly assumed the same thing had been done in the Australasian paper. The two lead authors on the paper were undergoing challenging personal circumstances at the time so this was not picked up until now.
That’s a redaction that I find myself agreeing with. Unnecessary personal information that shouldn’t be made public. In hindsight, it’s a shame that they missed it.
I think it’s sufficiently generic (and was sent by one of the co-authors to the journal, and presumably – see the other redaction – to the data providers) that it isn’t that big of a deal.
And it’s certainly not out of the realm of possibility that this is another way of saying “Woe are we. We were trying to determine how to respond to a request for data from that $#@! McIntyre” since that’s how they claim to have found the error in the first place.
The emails clearly show that CA was on their radar since Steve’s original request for data (May 30th). As noted earlier Neukom notified Gergis and Karoly about the error two hours after I pointed it out here. Anyone care to suggest chances for the astonishing coincidence that he found out the error on his own without seeing my comment on the very same thread that Gergis notified all authors about already on June 1st?
I think you meant “on their radar” ? Jean S: Yes, thanks. Corrected.
This is an amazing tale. Jean S. I believe you should be given credit for exposing this error. I don’t accept Raphael Neukom’s claim that he found this error independently.
Remember, this paper had already been peer-reviewed, accepted and published online. It was already being pimped by “Real” Climate and other media outlets. What reason would Neukom have to review the paper’s methods and calculations, if not for seeing the discussion on CA? None that I can think of.
Clearly, some co-authors of this paper were following the discussion on Climate Audit. I suspect Neukom saw your post, then verified your work during the two hour window, then made “his” revelation to Gergis the following morning. As far as I can tell from reading the e-mails, Neukom doesn’t explain to Gergis WHY he’d investigated into the paper’s trending / detrending math.
But on the small chance that this is really was just a coincidence, then I think it falls to Neukom to explain WHY he coincidentally was rerunning numbers from an accepted paper, late at night, within two hours of the Jean S post.
If this isn’t a coincidence, then Neukom has stolen credit for your work and claimed it as his own. This would be an act of academic and scientific fraud, and due to their credulity, this taint also touches Gergis and Karoly and the other co-authors. These folks cannot absolve themselves of the taint until they honestly discuss Jean S.’s and CA’s role in this issue, and it’s time frames.
not quite the timeline. Watch the Australia-Swiss times carefully.
Neukom was familiar with the issue and would have had a sinking-stomach feeling instantly on reading on Jean S post. He was worried, double checked and stayed up until 2 am to contact Gergis about the problem. If he had known about the problem earlier in the day, he would have contacted her earlier. The Australians regularly started emails at 6 or 7 in the morning (as shown by the dossier). If he had known about the problem prior to Jean S post and was as agitated as he must have been (to stay up so late), why wouldn’t Neukom have been in contact with the Australians before 7:42 a.m. June 6 Australian (when Jean S’ post went up). The evidence is circumstantial but convincing.
I think we’re saying the same thing. I’m aware of the exact time line. When I said “the following morning” I meant from Gergis’ point of reference. For Neukom, it was probably a very late night, and I suspect he probably didn’t get much sleep.
To be helpful to those who’ve heard of the timeline, but haven’t checked into it, I made this table with the UTC times and local offsets of the players for the two events. The table formatting depends on a non-proportional font, so if it looks mushed up in your browser, copy it to text editor or word processor that can display it im a non-proportional font.
Time Line (24 hour time format)
Jean S. posts on CA regarding faulty Gergis math,
including turnkey R Code and data
Climate Raphael Joelle
UTC Audit Jean S Neukom Gergis
UTC-5 UTC+3 UTC+2 UTC+10
—— —— —— —— ——
21:42 16:42 00:42 23:42 07:42
6-5-12 6-5-12 6-6-12 6-5-12 6-6-12
Two hours and 4 minutes later, Raphael Neukom’s e-mail,
in which he claims to have just discovered the same
exact, Jean S. error, arrives in Joelle Gergis’ inbox.
Climate Raphael Joelle
UTC Audit Jean S Neukom Gergis
UTC-5 UTC+3 UTC+2 UTC+10
—— —— —— —— ——
23:46 18:46 02:46 01:46 09:46
6-5-12 6-5-12 6-6-12 6-6-12 6-6-12
Nope, sorry, the browser upload feature removes duplicate spaces, so this won’t display properly if copied and pasted. Feel free to delete the post. If there’s interest in a comma delimited version for pasting into a spreadsheet, I’ll make and post one.
My favorite realclimater (Gavin Schmidt), May 31st:
🙂
Gosh, it seems that that expectation ought to clue them in that McIntyre ought to be among the first referees to examine any future papers, since he is so expert in finding potential problems, Save them a lot of trouble that way!
I couldn’t agree more. If these guys were really smart, they’d try to engage McIntyre rather than view him as Public Enemy #1. The fact that their first reaction is almost always an ad hominem attack makes me incredibly skeptical of their behavior and results. I gotta admit, they actually might be able to change my opinion of CAGW if they were obtain Steve’s validation of their methods and results. Otherwise, with regard to McIntyre, I believe they all doth protest too much.
Nice contrast (first part from Redcords post on other thread)
David Karoly, 7th June 2012 to Gergis and Neukom:
“The same argument applies for the Australasian proxy selection. If the selection is done on the proxies without detrending ie the full proxy records over the 20th century, then records with strong trends will be selected and that will effectively force a hockey stick result. Then Stephen McIntyre criticism is valid. I think that it is really important to use detrended proxy data for the selection, and then choose proxies that exceed a threshold for correlations over the calibration period for either interannual variability or decadal variability for detrended data. I would be happy for the proxy selection to be based on decadal correlations, rather than interannual correlations, but it needs to be with detrended data, in my opinion. The criticism that the selection process forces a hockey stick result will be valid if the trend is not excluded in the proxy selection step.”
Raphael Neukom, same day to Karoly:
“I agree, we don’t have enough strong proxy data with significant correlations after detrending to get a reasonable reconstruction.”
-Now, from Part 2a Journal Correspondence file, On June 14, Gergis back to editors:
Over recent days we have been in discussion with colleagues here in Australia and internationally about the use of detrended or non detrended data for proxy selection as both methods are published in the literature.
People have argued that detrending proxy records when reconstructing temperature is in fact undesirable (see two papers attached provided courtesy of Professor Michael Mann) .
While anthropogenic trends may inflate correlation coefficients, this can be dealt with by allowing for autocorrelation when assessing significance. If any linear trends ARE removed when validating individual proxies, t hen the validation exercise will essentially only confirm the ability of t he proxies to reconstruct interannual variations. However, in an exercise of this nature we are also intrinsically interested in reconstructing longer-term trends. It therefore appears to be preferable to retain trends in the data, so that we are also assessing
the ability of the proxies to reconstruct this information.
University of Melbourne’s actions in response to a potentially embarrassing FOI request stands in stark contrast to those of University of East Anglia.
UEA continued to fight tooth and nail to avoid releasing an email, I had FOI’ed, despite a legal Tribunal decision that they should do so.
I currently have posts about this at the “Bishop Hill” blog “UEA Footdragging”
Compare and contrast…
Eric Steig, June 9th 2012:
Just classic!
Karoly to Neukom & Gergis (June 7th) (my emphasis):
So if it were not identified in the “blogosphere”, Karoly would have simply ignored it?
JeanS
I think you should be careful about overinterpreting Karoly’s mail. Its fine to take a little time to work out a response to the journal which would have included a reliable quantitative estimate of how much things would change. This may well have been what Karoly meant. I doubt he would have left the record uncorrected if this wasn’t in the blogosphere and he knew there was a mistake in the paper.
Jean S: Sure, you are right.
About the only part I’m dubious about is honesty on Karoly and Gergis’s part about who really discovered this error. Boning up to mistakes is an important part of the process, but so is admitting to the origination of the discovery of the error. Claiming that you independently found the mistake yourself, when this does not seem supported by your own emails, ultimately undermines trust in anything else you have to say.
Re: Carrick (Oct 28 12:34),
in their defense it has to be acknowledged that Neukom claimed having discovered the error on his own.
Jean: these people have been known to mislead eachother. And after Climategate and recent FOI successes, it is likely they are writing emails in such a way as to present a narrative that is flattering to them.
But you knew that . . .
Jean S, does the email predate your observation of it on Climate Audit?
I admit if I were in their position (and I vaguely have been), I have shared discovery with other people who discovered something contemporaneously, simply because doing otherwise makes it look rather odd.
Jean S: The e-mail was sent about two hours after I made my observation, see here.
Thanks Jean S, that was the impression I had.
In my opinion, unless new evidence comes forward (which will look rather suspicious at this point given the FOIA response has been made without such evidence included), Karoly and crew have no case to make for independent discovery at this point, and certainly no claim for priority, and should say as much.
Jean S, you didn’t find an error in their paper, you could just not reproduce a step in their paper after following their laid out methodology.
Neukom, no doubt spurred by the fact you could not reproduce such early steps in the analysis, then checked their code and found that they did not perform the detrending step. Thus, he really could genuinely claim to find the error. Much like Gavin with the Steig data errors though, Neukom’s “finding of the error” was no doubt a direct result of analysis done at CA. To give credit, as with Gavin, would be an enormous loss of face.
There is a quick but accurate spreadsheet to switch between times in early June 2012, with UTC and 4 locations. All of the dates and times I have checked in the FOI material have proven accurate, but I have not done all of them
http://www.geoffstuff.com/TimelineAB.xls
The missing piece of information is evidence of a Skype communication between Raphael Neukom and Joelle Gergis around midnight to 2 am on 5 th June, Bern time. If Dr Gergis wished to enhance the reputations of herself and her University (of which one son is a graduate), she would volunteer this information. If one relies on emails alone, then the error being discussed is more plausibly the discovery of Jean S and Steve Mc.
Steve: here is my chronology (rough) http://www.climateaudit.info/correspondence/foi/gergis/chronology.xls
Its nice to see M. Mann and Myles Allen making cameo roles…
On the topic of Mann’s post (arguing against detrending) has there been a CA post specifically addressing the supposed advantages of not detrending ? I found the discussion about random noise failing validation tests interesting. Has there been a discussion of that here ?
Note – the above questions reflect curiosity and are not demands for “room service”.
Agreed on the Mann cameo. It is as if he has transmogrified into a caricature of himself – as if, when confronted with any issue confronting climate science, Mike Mann asks himself:
“What would Mike Mann do?”
It seems the answer is to use the word “orchestrated” more than any other English speaking person ever has before.
Bob, there has been much discussion on the problem of spurious correlation – and nonsense correlation – which has a rich history in the statistical literature, starting from yule 1926:
http://www.math.mcgill.ca/dstephens/OldCourses/204-2008/Handouts/Yule1926.pdf
The detrending issue falls into this area. Essentially two time series, both with trends, will produce a strong correlation score even if the trends are unrelated. In this type of analysis, this causes problems and some mechanism is needed to account for it.
Mann goes to great lengths to avoid accounting for it since spurious correlations underpin most of his results (e.g. reaction wood from BCPs or Finnish civil engineering projects correlating with NH temperature).
Detrending is just one aspect, but a relevant one since detrending is one mechanism through which you can prevent spurious correlation from influencing results. Although discussion of spurious correlation goes back to 2005 at CA (e.g. here), the most interesting detrending story was probably in the von Storch vs. Mann discussion between 2004 and 2006, which Steve has commented on e.g. here – also chiming in on Wahl et al 2006.
In Mann’s post, he admits to mis-informing RC readers (i.e. he clearly knows why the paper was retracted at this point). And bringing other Team members in on it.
“We have simply noted at RC in the comments that the paper does appear to have
been retracted from the AMS website, and we have no further information as to
why.
I will share this w/ Eric and Gavin so they know the status,
mike “
While ignorant deniers might call it a consp*racy, those on the Team call it “orchestrated”, a much more elegant term. The use of proper terms is so important. And it isn’t a “lie” it is “a fully documented record”.
oh boy.
Steig also asks the authors something along the lines of “Privately, do you think it will affect the conclusion?” while at the same time posting on RC (a few comments below the one Steve linked in the post update about “found by CA”) that his initial impression was that the results would change little, if at all.
Interesting once again to compare their public vs private statements.
I have 2 observations on these email exchanges: First the clubbiness that characterizes a number of the exchanges would appear in my mind to preclude any tough judgments on the work of club members by other members and shows that a thorough review of these works requires an outside and independent analyses. Even the one member (was it Myles Allen) who temporarily stepped over the line with a comment that could have been perceived as criticism was confronted with seemingly a request to explain away the criticism. I thought that was particularly weak. Secondly, I hope that the several other Gergis paper problems are not overlooked in this kerfuffle about the detrending versus not detrending. I think that overlooking what was actually done with regards to detrending was a strong indication of the pressure the authors were under to publish for the AR5 review deadline. Results that confirm previously held views on matters are not always given the same scrutiny as those that do not. I can see where the reviewers of this paper would not come forward.
Ken: In the correspondence with the journal, did you notice that the paper had been submitted the previous November and more than half a year to appear online. Doesn’t this suggest that peer review had required significant revision, especially with deadlines for AR5 looming?
Frank: Are you implying that the accepted/withdrawn Gergis paper that we at these blogs have been analyzing had more problems than we have found. Perhaps the reviewers can take some solace in their accepting a paper with the remaining problems simply because they had so many to weed out. Should we be considering that it is worse than we thought.
We need to be aware that the originally accepted paper had the admitted error of not detrending and the problems of a) going outside the paper’s defined boundaries for Australasia to obtain 7 proxies with 5 having exaggerated modern warming, b) the chronology method used for TRW proxies was not that referenced in the paper and thus we have no way of replicating the paper’s handling of TRWs, c) clustering of the TRW proxies in New Zealand and Tasmania, d) the reconstruction including no proxies from mainland Australia which is the largest land mass in the defined Australasia region, e) all but one coral proxy (and that one proxy was from outside the Australasia and pieced together) were from a relatively recent time period and finally f) the paper performed a rather unorthodox reconstruction considering what has transpired in reconstruction methodology in recent times with an ensemble ordinary least squares regression Principal Component Reconstruction (PCR) analysis.
I think most of these problems stem directly from the false notion of selecting proxies based on how well the proxies track the recent instrumental warming trend and the extent to which that search had to go even with the given that the authors and evidently the reviewers except such a false notion of proxy selection (and rejection) and the wide latitudes that that notion allows for finding proxies.
Ken: Above you wrote:
“First the clubbiness that characterizes a number of the exchanges would appear in my mind to preclude any tough judgments on the work of club members by other members and shows that a thorough review of these works requires an outside and independent analyses.”
I am suggesting that the time between submission and publication suggests that this paper may have received a thorough review that demanded substantial revision(s) before it appeared. The peer review it received obviously did not include the type of auditing done here at ClimateAudit, but this isn’t expected by journals or journal readers. (Co-authors should be auditing each other’s work.)
There are reasons why proxy selection could improve a reconstruction (I made a suggestion below) or bias it. The minimum standard should be to publish a reconstruction that shows ALL of the available proxies and separate reconstructions that show the effect of each selection step. If a Hockey Stick isn’t vaguely apparent in all of the data, but is appears after a selection step, the authors will face the impossible task of explaining why removing proxies that are assumed to contain only noise should cause this outcome. If the selection step merely reduces the confidence interval around the reconstructed temperature, then I see no problem. Authors should also be required to create pseudo-proxy data with noise appropriate for their situation and provide a quantitate estimate of potential bias introduced by their selection process.
If I randomly draw two sets of data (“intervention” and “control”) from a single population of data, they will be judged to be “statistically different” 5% of the time. If I am allowed to repeat this process 20 times, I have a reasonable chance of finding one experiment that demonstrates that my intervention worked. With the ability to select an infinite number of different subsets of proxies by changing selection criteria, confidence intervals lose there meaning.
Frank: The point has already been made by SteveM on this thread that once a physically and/or reasonably based selection criteria is imposed the proper statistical approach calls for using all proxies that meet that criteria. I am well aware of temperature proxies that meander all over the place with ending trends that can be upward or dopwnward or flat. That might well be telling us that these proxies as a group do not generally respond fully to temperature. If that is the case then selecting from these non responding proxies becomes a travesty and not simply just a one off error.
I do not know your background, Frank, but as a layperson in this matter with some experience reading climate science papers, I say it takes little effort to find the problems with the original Gergis paper. It gives me little confidence in the science and peer review process if for whatever reasons these problems were not addressed in review.
That reviewers insist on a selection process without reasonable priors is also troubling. And here I am not talking about a prior that merely tests each proxy for fit to modern instrumental temperatures – because that is completely wrong headed. Unfortunately, I am familiar with some, otherwise very intelligent, people who make this same error in other fields. Scientist I think are prone to this error in thinking because of preliminary testing they are sometimes required to do in testing a theory. There is also the problem, that someone on this thread made, where an individual assumes that a proxy signal exists a prior and thus one only needes to search for confirming proxies. All of these problems become clearer when you realize that little of no work has been done to produce reasonable proxy criterion for selection, making selections based on that criterion and finally looking at and comparing the time series that result.
Thanks for your input. I have a better idea of some of the other problems with this paper. I still think the time it took this paper to get published suggests that the paper was subject to peer, not pal, review and I don’t think peer review is intended to catch the mistake about de-trending, but you have demonstrated that the peer review was inadequate. I think the authors of any reconstruction should be required to show an all proxy reconstruction and calculate the potential bias introduced during any selection process as well as the robustness of their conclusions to minor modification of the selection procedure.
I don’t understand what was wrong with the reconstruction methodology. I understand that Von Storch proved that Mann’s methodology produced reconstructions with significantly reduced dynamic range (making straighter sticks and overlaying the historical record misleading). I know subsequent publications have methodology that better retains dynamic range, including something new in the past few months. Unfortunately, the names and acronyms of these methods mean little to me. Did peer review of Gergis miss the use of methodology that has been proven to be inferior?
Karoly’s “official” statement that he asked Steve to publish on CA refers to discovering the error on “Tuesday, June 5.” But at several points in the emails the discovery is said to have been made on Wednesday or Wednesday morning: p76 from the pdf 2b Karoly; p. 77 from 2b Gergis; p. 117 from 2b Gergis.
Too bad for Karoly the emails became available.
Re: Lance Wallace (Oct 28 15:57),
yes, Steve noticed the same here. Karoly and Gergis were not aware of the fact until Wednesday morning (Melbourne time).
I’ve now done a detailed chronology of these new emails. In the earlier versions, Gergis said that they discovered the error on Wednesday June 6:
but this was changed to Tuesday June 5 in their official statements.
Karoly also told Mann (the realclimate authors quickly involving themselves in the dispute) that they:
Unfortunately this “fully-documented record” was not included in the release, the documents of which show that Neukom reported the problem to Australia several hours after Jean S’ comment. And apparently with some sense of urgency since it was then nearly 2 a.m. in Switzerland.
Steve McIntyre:
Which really leaves them in a pickle I’d think. If they have such a “fully-documented record”, it should have been part of the FOIA release.
Now they have to explain why this “fully documented record” wasn’t part of the FOIA release. I speculate their US colleagues are used to deceptive responses and this won’t be seen as big deal.
Re: Steve McIntyre (Oct 28 17:52),
Their “fully-documented record was probably just a timeline comparing Neukom’s first email to the time of this CA post, not realizing the relevant time/point was in a comment by Jean S on another thread.
Diligence is not something I would be expecting from these [insert mildly derogatory plural noun].
Re: Steve McIntyre (Oct 28 17:52),
And it appears that Neukom withheld from his coauthors the actual inspiration for his discovery.
One more.
Channeling Mosher:
Emphasis mine.
Could the famed word parsers of UWA be at work here? A post is not a comment. So technically the above is correct.
Wednesday in Australia, Tuesday in Switzerland, perhaps?
c th m,
I post comments all the time in the comment section. Sure it’s a comment but it’s also a post.
It could be argued that a post isn’t a comment. I’m not sure I would agree with that but the argument could nevertheless be made.
On page 7 of 2b Gallant they discuss
Selecting just proxies with temp signals “strong”
How justified are they, and why should we trust
If their notion (and those local temps) is just wrong?
===|==============/ Keith DeHavelle
On page 24 of doc Gergis 2b
She notes a “barrage” that she got by CC:
But in all her emails, it’s not there to see
That certainly seems quite a conflict to me
It also now strikes me that “needless to say”
She has to explain her behavior away
She would have cooperated in every way
Except Steve asked first what archive rules held sway.
And so, since he hadn’t asked J. Gergis first
She naturally feels quite affronted. Lips pursed,
She now can “reply” to him (just shy of “cursed”)
That she will not help.
I’m assuming the worst.
===|==============/ Keith DeHavelle
1) Isn’t it so embarrasing, that 14 years after the first Hockey Stick paper, “experts” in the field still don’t have a clue how to process data ?
They are basically discussing issues from year 1 of Steve McIntyre’s criticism and fundamental issues remain unresolved.
2) Did the Swiss public or media take notice of the increasing number of Swiss climate scientists with disputed science, behaviour or attitudes ?
The (germasn speaking) Swiss media “inform” like PBS and NY Times (even though there is quite a number of papers, but all with the same PC-content).
And there is a non-conforming weekly, called “Weltwoche”.
The Nobel price winners are still heros, who prove that Liliput has an important role to play in the international community.
If Mark Steyn would write about ahr Raphi, and a translation is sent to “WW” to publish for free, some people might read about it.
Perhaps she considered him a misogynist? (Sorry…)
“Despite the embarrassment of the withdrawn paper, the University is to be commended for their no nonsense approach to this request.”
My question for Michael is did you really need an FOI? Or did you try just asking?
The latter would be commendable.
FOI’s real intention is uncover corruption from people that don’t wont to release all the information.
Either way is commendable, as is the response of the university, which shows that they don’t consider a FOI request the slur you say it is.
In the context of past correspondence and hostility, an FOI approach would seem to be the default. Did you just tune in?
There is nothing wrong with making a request under FOI. It does not imply wrongdoing and it should not be interpreted as a hostile act.
This is OT but I don’t know where else to put it and I don’t know if it’s been mentioned here but Anthony at wattsup has a new post on a paper that appears very troublesome for Dr. Mann especially since it is coauthored by one Keith Briffa.
I’ve just added a lengthy update, setting out a detailed chronology relating to Karoly’s claim that, on June 5, they had supposedly discovered the error “independently” of CLimate Audit. Readers will be entitled to raise their eyebrows quizzically.
Never let it be said this blog doesn’t allow its readers to come to their own conclusions, based on the facts presented – and even their own facial expressions.
It was always absurd that they didn’t credit CA and Jean S but this time the chronology is out in the open very early, by climate standards. Eyebrows at the ready.
The reason why they probably won’t consider giving credit to Jean S. and Steve, is because it would show competence and diligence on CA’s part and might draw many far more important issues into examination. Heck, Steve and Ross just might be…gasp…correct about the hockey stick?! And many other things. The MSM might even start paying attention to what they say. That would never do. The “team” has the media convinced that CA are incompetent oil shills. They intend on keeping it that way.
One hypothesis is that Neukom (or someone else) saw Jean S’ post before any of the Gergis authors recognized his mistake. The problem with this hypothesis is that Neukom had only two hours to notice Jean’s comment, review his work, call Gergis, and then write his email. If Neukom spent all of his spare time reading skeptical climate blogs, this might be possible, but it isn’t an attractive hypothesis. Of course, many people are looking at ClimateAudit and one of them could have alerted Neukom. However, it seems more likely an outside party would have alerted Gergis or Karoly, who were located where it was daytime.
An alternative hypothesis is that the discussion at ClimateAudit prompted Neukom to review his work and identify the problem. Then he chose to wait a few days to see if his mistake would be discovered before deciding what to tell his colleagues. The paper had been accepted, published on the web, and used in the FOD of AR5. The damage, if acknowledged or discovered, was already done. He might not have recognized the damage that would follow if the problem were publicly identified at ClimateAudit before his colleagues could act or that his email would be subject to FOI and shown to have been written 2 hours after Jean’s comment. (Although it doesn’t seem likely, he might have put something in writing to establish a record.) This hypothesis would explain why Neukom might have been reading ClimateAudit at midnight in Switzerland and able to alert his colleagues to the problem in less than two hours.
In any case, Neukom wouldn’t have discovered a mistake in his paper and emailed his colleagues at 1:45 am if ClimateAudit hadn’t subjected his paper to intense scrutiny and the mistake could have remained undiscovered without Jean S.
Re: Frank (Oct 29 03:26),
It takes less than five minutes for him to check the correlations even without downloading my turnkey code. He is the second author, and presumably has all the code and data.
Why not? As Steve notes, Gergis had allerted a few days earlier all authors about the very post under which I reported the error.
Very unlikely. It is not even clear that it was “his mistake”. Anyhow, just for psychological (and ethical if you will) reasons, I’d expect him to have notified at least Gergis (the first author) immediately. There is no indication that Gergis knew anything about it prior to Neukom’s skype call just before the email.
Jean:
I agree with you that Neukom (or someone else) could easily have done everything you say in the two hours after READING your post. I’m focused on the amount of time it would take for him to notice your post if he had no special reason to look for it. Those who aren’t familiar with your contributions to ClimateAudit need some time to separate the wheat from the chaff in the comments section. (My view of the comments is threaded and therefore hard to monitor.)
If Gergis et al really cared about what was happening at ClimateAudit, we should have seen an email assigning someone or volunteering to monitor what was happening at ClimateAudit.
You are correct about the ethical responsibilities of the person who discovered the mistake, but that person may not have acted immediately. He may have wanted to discuss in person his responsibilities with a mentor or he may have been unable to immediately lay his hands on all of the information needed to have complete confidence needed to report a mistake. You may have provided that. At ClimateAudit, you have the ability to post a surprising result and ask others to confirm your findings. Neukom would need to be 100% sure of his position before alerting his colleagues. (Gergis had an ethical responsibility to notify the journal when she became aware of the mistake, but she didn’t write for several days.)
Finally, I’m merely presenting an alternative hypothesis that IMO doesn’t diminish the scientific importance of the hard work presented at ClimateAudit. If I gave you that impression, I owe you an apology.
Frank:
In early June, as I observed, the email dossier shows numerous emails directly referring both to Climate Audit and to the thread where jean S’ comments are posted, commencing with Gergis’ email to David John Karoly; Raphael Neukom, Ailie Jane Eyre Gallant; s.phipps, saying:
Plus many more.
If the emails contained no references to Climate AUdit, then maybe you could argue that they weren’t paying attention to CA. But that’s contradicted by the emails. And not only the Neukom-Gergis coauthors. Within a couple of hours of my first post on Gergis et al, Gavin Schmidt contacted Gergis; Mann contacted Karoly and Steig contacted AIlie Gallant.
It’s trivial, easy, to set up a link
To get all these comments sent right to your place
So this is not quite as far-fetched as you think
They can’t stand to credit CA, or lose face
===|==============/ Keith DeHavelle
Frank, even if Neukum independently discovered it, he doesn’t have priority for the discovery, unless he has documentable evidence.
Jean S has priority on this, which Gergis et al are failing to relent for obviously political reasons.
Since Neukom had been alerted to the post (by Gergis) he could have easily set up an RSS feed (see links under Meta on the right) to monitor comments as and when they came in.
Neukom wouldn’t have had to “spent all of his spare time reading skeptical climate blogs” to find the comment, his RSS feed would have told him, within a short period of time, about the new comment and possibly even shown him the contents.
So Neukom may be the only one that really knows (?). Was it a massively improbable coincidence or did he *really* find the problem independently during his 2-hr window of improbability??? Needless to say, it sounds like the rest of the authors hold that he did find out the problem independently.
This appears to be the case; if he shared the paper trail with the other authors, it would have turned up in the FOI request. Neukom is not at Melbourne Uni, and I have no idea what FOI procedures apply in Switzerland.
I would think that in all the circumstances, a voluntary disclosure of the paper trail would be most welcome.
Well, they can always say there was not ‘record’ of the actual finding preceding the Skype call to show the discovery.
“Nor did I send her any such email.” – this issue, however, can be directly challenged with a very public inquiry for Gergis to bring forward the threatening email.
Michael – I believe you mentioned on the other thread that you intended to inform the ‘authors’ of your impending disclosure of the FOIA emails. If you don’t mind my asking; did you get any feedback from them upon your notification?
From Gergis: “Although we attempted to be transparent about our methodology, this has backfired and caused a lot of trouble.”
I do hope that the authors do not interpret this contretemps as a negative effect of transparency. The paper was fairly good in its description of [intended] methodology, although (from memory) it omitted mention of using Spearman correlation and two-tailed p-values, and provided no justification for the inclusion of several candidate proxies from outside their defined Australasia region. It is preferable to include code (in whatever computer language) in the supplemental information to ensure reproducibility, and I would urge the authors to do so with the resubmission. [“A module is worth a thousand words.”] My experience with preparing code for review (or archival), is that it prompts a re-inspection from a different perspective, and has often resulted in improving the code and occasionally in finding a bug. Had they taken that step, perhaps Neukom would have discovered the oversight in a timely fashion.
Perhaps this is the “intimidating e-mail’? From Part 2b Gergis: Steve quotes “Mr Wolf” in “Pulp Fiction”…
“Subj ect : RE: Gergis et al 2012
Dear Sir and Madame,
Gergis et al 2012 states:…….
……..Pretty please with sugar on it,
Steve Mcintyre”
Just a general whine…
Karoly admits freely that sorting to trended data will force a hockey stick result:
” If the selection is done on the proxies without detrending ie the full proxy records over the 20th century, then records with strong trends will be selected and that wi ll effectively force a hockey stick result. ”
Nobody serious seems to refute this fact anymore, yet none of the old CPS and MV garbage has been retracted. It is implausible to me that scientists can simulteneously grasp that sorting trended data causes a hockey stick, yet miss that detrended data also can create a FAKE high correlation result from data that is vastly noise. Considering that so many series are pre-sorted before use, I don’t know how much function the detrending really provides anyway. The problem is a difficult one but ignoring it won’t make it go away. It is as though they refuse to employ critical thought on the matter.
Slope or not, the sorting methods are complete garbage science on the face of it, and whether they identified a nuance of one bad method over another or not, they are simply highly paid mathematical scribblers.
Jeff, I corresponded recent;y with Fredrik Ljungqvist and was critical of their ex post correlation screening. Ljungqvist replied:
The follies of Team paleoclimate seem without limit.
This has always bothered me as well. The argument seems to be: “sometimes these records are good proxies and sometimes they aren’t.” What seems to be missing is some physical argument, hypothesis, or justification for explaining why some of these records do not make good proxies. (There may be one, and I just haven’t been paying close enough attention. However, I would be very surprised if there was a testable hypothesis, let alone one that has already been tested.)
Steve, it seems to me, from reading these emails, that your comment was very prescient, when you said they will probably find some way to resubmit the paper by reverting to a method (non detrended data) that the previous version of Gergis, et al., eschewed. One can already see the arguments forming in these emails, and various authors slowly coming around.
It’s more insidious than someone overtly saying (or thinking), “Hey – this method produces the answer we want, so let’s do it that way.” They really do seem to be convincing themselves that this is the better method, and losing sight of the fact that it is “better” only because it – rather than the original method – produces the expected result. This is far and away more of a problem in climate science, and science in general, than the insignificant number of truly bad apples who would knowingly lie about their results.
And it is doubly a problem because those who are unconsciously swayed towards the expected results are then more likely to accuse their critics of the same failing. But there is hope, I think. It seems like the overreach among ‘consensus’ scientists is, more often these days, unraveling, and fault lines are more readily exposed. One person in the emails – I think it was Karoly, but I may be misremembering – said of Climate Audit [paraphrasing]: “Just ignore the rude comments and insults, but don’t ignore the science.” Perhaps they won’t resubmit after all, and perhaps one or two folks inside this clique won’t actually allow for substitution of a method that was previously disavowed. Here’s hoping.
A very high acceptance threshold with trended series is basically the only way to get valid results because low frequencies are actually those that are especially interesting. But there is one big issue with low frequencies, they are quite bad in reference series (T). We don’t get any credible result until this issue is settled.
One way to do this is to check and possibly adjust the reference series by proxies of very good high frequency correlation in detrended mode.
The only way to do this is no acceptance threshold which is not justified by known physical reasons. Drought, disease, methods etc..
Average the rest and the signal, if there is one, will out.
“But many reviewers have been extremely critical against not screening the proxies. In fact, some reviewers suggest rejection of a paper based on that the proxies have not been screened by correlation to instrumental temperature prior to being used in a reconstruction.”
So present the results with and without detrending steps. Discuss the difference in the results pointing out that there is no way you could be sure correlation with temperature over the callibration period for a given proxy is not due to noise or some other artifact, and likewise noway of knowing that the earlier part of a match was or was not likewise affected. Who can argue with that ? The referees looking for a hockey stick get it, with a discussion of why it’s not valid (or more simply, why you’re guaranteed to get it), and those looking for an honest analysis of the proxies without “post analysis” proxy selection get it too.
Ljungqvist – “But many reviewers have been extremely critical against not screening the proxies. In fact, some reviewers suggest rejection of a paper based on that the proxies have not been screened by correlation to instrumental temperature prior to being used in a reconstruction.”
That is a bit disappointing with so much of the ‘modern’ science focused on reducing the variance loss problem. From the little I know of him, an argument of popularity in review is beneath Ljungqvist and a little surprising. I must be a little overconfident in the paleo-climate scientists facing up to this fact. It isn’t without basis though, because there are several papers and emails discussing the problem in an open manner.
Steve and Jeff:
Is the situation really this simple? Doesn’t the inclusion of “bad” proxy data (with no temperature signal) interfere with the ability to abstract a useful temperature signal (with narrow confidence intervals) from the good proxies (those with some correlation to temperature)? If so, a method that can eliminate some “bad” proxies would be valuable – if it didn’t introduce significant bias into the reconstruction or the introduced bias could be quantified.
Some of these emails said that if one wanted to do a reconstruction of climate change, one needed proxy records proven to correlation with long term temperature change. This approach guarantees that you will select bad proxies that show long-term positive correlation to the temperature record by chance and reject the “balancing” bad proxies showing negative correlation.
A better approach appears to be look for proxies with high correlation to annual temperature changes or higher-frequency temperature change. A tree ring probably responds mostly to the climate during the season it is grown, and probably only a little the climate in previous years (which may effect the health of the tree). The deuterium content of a layer in an ice core depend on the temperature the year the water condensed and evaporated. Some proxies may integrate the temperature signal over several years and show poor correlation on a yearly basis, but I’m not aware that such proxies are common. Detrending before looking for correlation should help because it eliminates the lowest frequency signal, placing more weight on the correlation with higher frequency signals that are less likely to bias a climate reconstruction.
Steve: clearly a proxy needs to have a relationship to temperature. However, in my opinion, once you take the position that high-altitude larch chronologies or delD or whatever is a proxy, then you need to take the entire population and not pick and choose ex post. Whatever criterion one uses has to be ex ante and and not ex post.
Correlation to temp series should be just one of many screening tools to look for temp sensitivity. With tree rings, there should also be a robust correlation matrix with both latitude and age-standardized growth. Trees closer to treeline should show a stronger correlation, and trees with the best fit in age standardization and uniform growth should be preferred. After a full double blind matrix is developed, a single tree will have the highest score in terms of correlation to local temp, highest latitude, and most perfect growth pattern. Then if you show me the full matrix, including all the trees that were rejected from the final study, and I see a pattern that suggests that your results conform with all pertinent aspects of dendro theory (higher lat is more temp sensitive, etc.), I just might believe that you have a usuful paleo study.
Steve: this is not at all how I would express things. for present purposes, we are talking about the narrow issue of ex post screening and not trying to figure out all the problems in the field.
Matt,
You have oversimplified the problem. If you choose a certain group of trees as superior based on physical rationale, you can use the entire group, weight the group uniformly and compare to temp after processing. If you simply correlate by screening and find a trend in selection which matches your screening results, you have potentially identified a real physical effect. What your screening does though is correlate (signal + noise). Both negative noise and weak signal are rejected in preference to strong signal and positive noise. Your “good” noise therefore is amplified in the test just as much as good signal. The existence of a pattern in the screening (I would be interested in seeing an example of this) would simply mean that the noise hasn’t completely covered up the temp signal.
Whether screening reveals some pattern of conformance to dendro theory is therefore completely moot to the validity of the reconstruction.
Steve. Would the work of Esper et al qualify ex ante in your opinion?
Steve: it’s high latitude tree ring data. Esper ex post excluded some measurement data and has refused to provide the screened out data, diminishing the value of the study IMO. Nonetheless, the study has interesting aspects and made me stop and think about some constructive points that I hadn’t thought about.
SM: “…diminishing the value of the study IMO”. With “IMO” I believe that you are being modest to a fault. By refusing to provide the screened-out data, even scientific dodos (possibly including myself) can tell that the value of the study is certainly diminished; it is beyond a matter of opinion.
Steve: Your arguments are very sound: Once one admits that a proxy might not work in the documented past, how can we trust that proxy to reconstruct the undocumented past? I’m tempted to say that the signal-to-noise ratio of “valid” proxies can vary with LOCATION (not time) and one might want to get rid of those proxies with demonstrably low signal to noise ratio. Before doing so, I would be ethically obligated to show the reader a reconstruction made from all of the proxies, so the reader is aware of how selection (and possibly selection bias) shaped a final reconstruction. And I should be required to estimate selection bias using appropriate pseudo-proxy data. And I probably should be required to demonstrate the result is robust to modest changes in selection stringency. Would you tolerate this process?
Deuterium isotopes in ice cores can be valid proxies in locations where the source of the precipitation (where it evaporated) remains relatively constant, and much noisier proxies where this isn’t the situation. TRW near the tree line may possess a higher signal-to-noise ratio for temperature than elsewhere and give inverted results in situations where water limits growth. (Carrying this line of thought to absurdity, I don’t have to use data from all of the thermometers in my lab, if I have calibrated some of my thermometers against a standard. Calibration introduces whatever biases the standard possesses.)
Has anyone simulated the degradation that takes place when too many proxies with very low signal-to-noise are used in a reconstruction?
Frank,
” Doesn’t the inclusion of “bad” proxy data (with no temperature signal) interfere with the ability to abstract a useful temperature signal (with narrow confidence intervals) from the good proxies (those with some correlation to temperature)? If so, a method that can eliminate some “bad” proxies would be valuable – if it didn’t introduce significant bias into the reconstruction or the introduced bias could be quantified. ”
First, of course bad data does cause problems. This proxy data is noisy though so the question becomes what is good and what is bad. Are we really sure that any of these proxies do or don’t contain a reasonable temp signal? In math, we know correlation is an crude yet unbiased method to compare the patterns between two series. The assumption that a threshold of good vs bad correlation doesn’t apply only to the signal in the data, it applies to signal plus noise.
What you get as a result is two categories of data which are NOT comprised of good temperature sensitive proxies and bad ones. That is a complete misnomer as the good doctor points out in his statement about the automated creation of a hockey stick shape. i.e. big blade, flat handle.
What you get as a result of this sorting is rather, one set with the best correlated (signal + noise) and a second set with (signal + less helpful noise). The signal may or may not be stronger in either case as this is not the question correlation screening asks. So in neither dataset have you proven the proxy was or was not temperature sensitive. The process does guarantee good correlation to temp though, including overtight CI’s. In the end it provides literally ZERO information about historic temperature but plenty of room for silly discussions about the MWP.
—
The same processes occur in multivariate regression methods as well. In those cases, the series are weighted according to how much they look like temp. Often in these methods, the series are even non-phsically inverted if the resulting pattern has a tighter fit to the target curve. The reality is that the problem of BAD DATA cannot be solved by math. It can only be solved by better data.
Steve:
If it had been my paper, I would have wanted to assign co-authors to stand watches 24/7, but they would have other work to do. Once ClimateAudit get its hands on the data, the half-life of the scientific credibility of most papers showing a big hockey stick now seems to be about a day or two.
Taking credit for someone else’s discovery is despicable. Sitting on a mistake until someone else finds and publicizes it isn’t much better. Unfortunately, placing too much confidence in the former hypothesis when other explanations are possible could be counterproductive. Everyone not blinded by the political considerations should understand that Neukom’s email disclosing the mistake was almost certainly written because of Jean’s comment two hours earlier and that without ClimateAudit the paper wouldn’t have been withdrawn.
I don’t wish to risk further offending you, Jean or your other readers by stubbornly continuing to defend my hypothesis (unless you see some value in my doing so).
Reply to Jeff’s Oct 29, 2012 at 1:56 PM and Steve’s reply in my comment posted Oct 29, 2012 at 2:21 AM:
Jeff wrote: “Are we really sure that any of these proxies do or don’t contain a reasonable temp signal?”
Let’s consider deuterium isotope proxies. If I understand correctly, there is a well-established physical mechanism that explains how much precipitation will be enriched when water vapor condenses to rain or snow at various temperatures. This produces an unambiguous temperature signal. If we are reconstructing surface temperature, noise is introduced by changes in the altitude at which precipitation occurs. (Temperature drops about 6.5 degC for each kilometer of altitude on average, but radiosondes show variation). Noise is also introduced by the variable amount of deuterium in the water vapor, which depends on the temperature of the water from which the water vapor evaporated. (Globally, SST’s vary by about 30 degC.) The deuterium isotope content also changes as part of the water vapor condenses on the way to the proxy site. The idea that deuterium isotope proxies from different locations are all equally “good” or “bad” (equal signal to noise, never showing an inverse relationship) seems unreasonable under these circumstances. I suspect the same applies to all proxies. These problems aren’t likely to be solved by new or better data.
Any reconstruction should initially be calculated using all of the proxy data. Can that reconstruction be improved by selecting proxies that are better correlated with the de-trended local temperature record? I wish I had the technical ability to answer this question. You and other bloggers have unambiguously illustrated the dangers inherent in any selection process. It’s essential to quantify the potential bias introduced by any selection process and the robustness (sensitivity to modest changes in selection criteria). Given the easily demonstrable dangers inherent in selection processes, the peer review process should prevent the publication of any paper that doesn’t quantify potential bias and robustness specifically for their reconstruction. Once these factors have been fairly discussed, it isn’t clear that a second reconstruction made from selected proxies will be judged to be an improvement over the all-proxy reconstruction.
From a practical point of view, a strict ban on ex ante selection may be a more practical solution than expecting the paleoclimate community to properly police the selection process.
Frank, I think many of us here would agree that oxygen and hydrogen isotope ratios could potentially be good temperature proxies as the physics/chemistry is rather straight forward.
That these proxies can get the gross temperature changes from ice age to ice age right is documented. The problem comes when we attempt to look for smaller changes as represented by the current warming period. I have seen a number of ice core isotope proxies taken in close proximity that would appear to yield very different temperature trends. This may well be the result of sample handling and storage or even instrument problems. Obviously as already noted not knowing the exact origin of the water vapor condensed at a site and changes in the origin can effect the interpretation of these proxies.
I also see a tendency in those doing these reconstructions to merely gather the data and publish and not worry that much on how the sample were obtained, stored or tested or under what conditions. I have seen only a few papers that have attempted to analyze what can go wrong with these samples.
Frank,
Your comment is reasonable yet there are some issues:
“The idea that deuterium isotope proxies from different locations are all equally “good” or “bad” (equal signal to noise, never showing an inverse relationship) seems unreasonable under these circumstances. I suspect the same applies to all proxies.”
Vegetation can show an inverse relationship to temperature – some on this thread have even published on the matter.
“Can that reconstruction be improved by selecting proxies that are better correlated with the de-trended local temperature record? I wish I had the technical ability to answer this question. ”
The question is quite simple to answer.
” It’s essential to quantify the potential bias introduced by any selection process and the robustness (sensitivity to modest changes in selection criteria).”
Actually, if you have a proper method, there is minimal selection and zero bias from the method. Ex Ante (before the process) selection based on physical properties has a greatly reduced chance of bias. To give points to your statement, one can imagine that there are situations where pre-selection of data based on seemingly unrelated criteria could create bias which would then require discussion of potential bias. Once the data is selected, the choice of data which matches the assumption of result is …. insane.
” Once these factors have been fairly discussed, it isn’t clear that a second reconstruction made from selected proxies will be judged to be an improvement over the all-proxy reconstruction. ”
The various methods being published are not based on selection of one good data series over another less good one. They are specifically selection of one set of data+noise over another set of data+noise. The result is universally to select the best data and more importantly, the best noise, in the calibration period.
Since the noise is also preferentially selected, the fit of the expected result to temperature from selected noise+signal proxies is guaranteed to improve and the variance of the historic result becomes moot.
Completely useless math.
In noisy data with no metadata, it is very hard to beat a simple average.
Lungqvist himself has published papers based on this concept. So has Craig Loehle.
Reply to Jeff’s comment Posted Oct 30, 2012 at 6:20 PM:
You wrote: “The various methods being published are not based on selection of one good data series over another less good one. They are specifically selection of one set of data+noise over another set of data+noise. The result is universally to select the best data and more importantly, the best noise, in the calibration period.”
However, the biased selection of noise and the distortion it produces only applies to the selection period. Actually this isn’t correct, sense the selection period is often the calibration period and selection bias feeds into calibration (dynamic range). However, year-to-year temperature changes are large compared with long-term trends. They provide an opportunity to select temperature proxies with high signal to noise from low S/N in a manner that doesn’t appear to force large biases into the reconstruction. I’m under the impression that detrending is a crude way select mostly for the annual changes and minimize biasing the longer “climate change” signal. (A filter might work better.) Has anyone in the blogosphere or literature proven that correlation with detrended is still ridiculously bad?
You wrote: “In noisy data with no metadata, it is very hard to beat a simple average.”
I’d say that, in this situation, the authors should be required to prove using pseudo-proxy data (with appropriately red noise) that their selection process produces a result which is superior to a simple average (which they must show) and robust to minor changes in the process.
In my 2007 paper I did not pre-screen and did not weight but used all proxies with equal weight. Reviewers (including bloggers) had a fit that I should have screened.
The problem is that the scientists believe there is a signal, and fiddle with the data until the “best” or “strongest” signal appears, with no control for spurious results and no understanding of the problem that if some of your proxies (tree rings) go down after 1960 how do you know others were reliable in the distant past? How one could get past intro to stats without understanding the data mining fallacy excapes me.
Craig,
Our comments crossed in cyberspace. Fortunately mine appears above yours, since I certainly consider your 2007 paper to be a useful contribution!
Jeff C.: “Slope or not, the sorting methods are complete garbage science on the face of it, and whether they identified a nuance of one bad method over another or not, they are simply highly paid mathematical scribblers.” Hey, Jeff, don’t SUGARCOAT it like that!–come right out and tell it like it is! (With apologies to “Butch Cassidy…”)
William,
Tis my style.
The reality is that the problem exists and it creates a false answer. What is worse is that only the solution is the one which Dr. Loehle used where all proxy data is averaged can correct the problem. One BIG problem with even that approach is that much of the non-correlating data has simply been deemed non-sensitive to temp and are difficult to even see because they don’t get published. This pre-selection is essentially the same correlation-sorting algorithm used to create hockeysticks except that the processing happens in human-space rather than in the code.
I see that Mann was kind enough to send the Wahl et.al. comment from Science in 2006 defending Mann’s use of non-detrended data. I hope Dr. Karoly was smart enough to check the reply from Prof. von Storch (Science, 2006) who said “The calibration and validation of any statistical method using nondetrended data are dangerous,because the nonclimatic trends are interpreted as a climate signal”.
http://www.sciencemag.org/content/312/5773/529.2.abstract?sid=6683961e-db31-493c-8efb-d595ee8bb6a2 and http://www.sciencemag.org/content/312/5773/529.3.abstract if you have access to Science.
Well we can now all agree on one thing, this is definitely not “called research”.
Research is carried out by fallible human beings who make honest mistakes, particularly ones that support their preconceptions. This work will remain research if the authors deal with the mistake in an appropriate manner. They appear to have passed up their best chance to do so, by correcting and revising their paper to show the reconstruction with detrended data (too few proxies), non-detrended data (potential bias introduced), and all data; followed by a candid discussion of the merits of each approach.
Gergis: “This list allows any researcher who wants to access non publically available records to follow the appropriate protocol of contacting the original authors to obtain the necessary permission to use the record, take the time needed to process the data into a format suitable for data analysis etc, just as we have done. This is commonly referred to as ‘research’.
We will not be entertaining any further correspondence on the matter.”
Ill mannered and arrogant in equal measure. I’m assuming she’s been promised a mention in IPCC AR5 and this led to her hubris.
You are correct: This wasn’t acceptable. We’ve talking about publicly-funded research published in a journal that requires authors to assist others seeking to replicate or building on their work. However, it is a normal part of human fallibility. Who wants to assist the opposition?
Wasn’t this problem partially or totally corrected when brought to the attention of others.
Mind you, I missed this at first reading:
So someone finds a mistake with your work quickly and early, and the “problem” is giving them enough information to do that?
So it would have been less of a problem if the mistake had never been found…?
At least Neukom does seem to understand the screening fallacy, and uses clear, strong words to endorse having some mechanism to prevent it; but he is not lead author, and the discussions between Gergis and Mann show no such care. I wonder who caved in and what the final paper looks like?
But many reviewers have been extremely critical against not screening the proxies. In fact, some reviewers suggest rejection of a paper based on that the proxies have not been screened by correlation to instrumental temperature prior to being used in a reconstruction
Clear indication of the breakdown in the peer review process. This should be the final nail in it’s coffin but it won’t be.
There is much gold among the dross in this FOI, which Michael let me have some days ago. Some has already been picked up by sharp eyes, but from time to time there are passages that could make articles in themselves.
Here is an example of clear-cut decision making for the proper handling of data. Really, only one way is optimum and it might not even be included the selection below.
It read s like what it probable is , an arse covering exercise from those that know they dropped one and don’t have the guts to admit it in public .
Gergis did after all make a dam fool of herself and has true follower of Mann , you expect her to take no other path than denial and smear as ‘the master ‘ does .
A minor addition to the timeline on June 8th.
The disappearance of the paper from the J Climate website was I think first noted by me at CA at 9.02am CA time, following comments from Jean at BH that references to the paper on author websites had been mysteriously vanishing.
Shortly after that, I posted a similar comment at RC (without an exclamation mark). Gavin Schmidt deleted my comment and inserted his own, comment 34 on the thread you link to here, misleadingly saying that the link was broken.
The relevant BH thread is here. The first comment from Jean about the Gergis et al paper disappearing from web pages was 9.48am UK time.
I think it is significant that by the 1st June these relatively junior scientists (compared to the Team) were informed that Gavin Schmidt thought:
Whatever the context I think that quote would stick out in the minds of people who were trying to gain elevation into, and the respect of, the Team. Take that together with Schmidt’s dire warnings of providing a “talking point” (the horror!), and add the apparent weight they seem to attach to the CA piece on Myles Allen (further PR horror!), then maybe it seems a reasonable assumption that anyone who already held a niggling doubt about the work realised the dreadful risk of providing something far worse than a mere “talking point”. I think it is inevitable that at least one of the authors was monitoring CA pretty damn closely. 😉
Steve: The CA piece “on Myles Allen” was mostly on Gergis et al. I used a quote from Myles Allen as a text for criticizing Gergis et al.
it is remarkable that three RC authors (Schmidt, Mann and Steig) each contact Gergis coauthors within hours of CA posts.
There are your fixers, moshe; The Three Stogies, exploding ones.
===============
That had me racing to Google. Stooges?
“…there is no chance whatsoever that [Steve McIntyre et al] will examine your work and find no faults…”
Wow. Every time I submitted a paper and the reviewers found faults the result was a better paper.
On the issue of replication and falsification of scientific protocols, one of the most damning statements from Climategate was Phil Jones’ comment that he wouldn’t share anything with Steve McIntyre, because he (Steve) would “only try to find something wrong with it.” This comment showed so plainly his (and by association, “the team’s”) open contempt for basic scientific fundamentals.
Replication of results should be possible for ANY scientific protocol. When replication of results has not been possible, the alarmist side of the debate defaults to “circle the wagons” attitudes, imperious rejections of criticisms from the other side. Then there are the issues of pal-review style peer reviews and circular peer citations to artificially boost credibility of papers. When data is requested, or methods need to be clarified, the authors in question behave as if such requests are tantamount to harassment. This must stop, if climate science is ever to regain it’s lost credibility among critics.
There was a good thing exposed in the e-mails. I give credit to Ms. Gergis for her attempts to get the data Steve asked for. Following her e-mail exchange asking Gavin Schmidt for his thoughts (although he was highly insulting toward Steve), he ultimately gave her some good advice. “Just give it to him.” Maybe these two scientists can be a vanguard of change, and instead of reacting negatively, set up a scheme where in the future, replication is anticipated, and criticism is seen as a necessary and good process that will preempt this kind of fiasco.
I respectfully offer these suggestions to publicly funded climate scientists:
ARCHIVE ALL THE DATA FIRST! After designing a scientific protocol, before analysis, treat the data like it will be reused by others, and use it that way yourself. Act as if you understand you’re only one of possibly many people who could take an interest. In the Gergis et. al. case, this would mean EVERY proxy candidate would be archived, not just the selected candidates. The rejection of proxies was part of the protocol, so the rejected proxies may have a lot to say about the credibility of the paper’s conclusions, ergo, they too, need to be publicly available.
Which leads to subsequent problem for which I offer this excellent heuristic:
REJECT PRIVATE OR UNSHARABLE DATA! If there are ownership issues, and data cannot be shared, don’t use that data in your protocol. If no future replication can be done because of the privacy issues, what good will your paper’s secretive arguments be? They cannot possibly be challenged or repeated. Much published climate science is already doomed to be ignored in the future for just this reason, IMO. Don’t add to this store of waste.
DO YOUR OWN REPLICATION TEST PRIOR TO PUBLICATION before peer review begins. Assign a non-author graduate student a clean, blank computer, the completed but unpublished paper, and a pointer to the data archive, and ask them to validate the paper. If he/she cannot, you’ve just saved yourself a ton of time. Rewrite and repeat until a non-author can validate the math and get the same results.
Steve: this is more or less what I’ve been urging for years. BTW the original Phil Jones comment was in respect of Warwick Hughes in early 2005.
“Following her e-mail exchange asking Gavin Schmidt for his thoughts (although he was highly insulting toward Steve), he ultimately gave her some good advice. “Just give it to him.” Maybe these two scientists can be a vanguard of change, and instead of reacting negatively, set up a scheme where in the future, replication is anticipated, and criticism is seen as a necessary and good process that will preempt this kind of fiasco.”
Mickey: I always looked at what appears to be motivations behind these comments, and in this case I see it as one of damage control and not of what should the scientist’s instinctual reaction of having their work critically reviewed and duplicated by others. In much of climate science there seems to be the worries of policy advocacy hanging over the science and that advocacy does not seem ready for inviting others to find errors in their work.
I think I have posted this link before, but it seems more relevant than ever. It is the text of a lecture by David Hendry, whom CA readers may know from Steve’s link to his LSE inaugural lecture about spurious correlation (cumulated rainfall “explains” the UK price level better than cumulated money supply), to senior students in the University of Western Australia. It distinguishd two types of academics, those committed to getting ahead and those committed to the community of scholars. t becomes easy to place some climate scientists! Even more interesting is his description of how he behaved as an editor. Alas very few people seem to follow his example, but if they had paleodendrochronology would surely be in a better place.
See
http://ideas.repec.org/p/uwa/wpaper/10-01.html
A bit of local politics here – unlike the University of Western Australia (host to the parasite Lewandowsky), UM is the top ranked Australian university in most of the international indices. They have a reputation to protect – but as to how they choose to protect it, we don’t know. One thing we do know is that august institutions don’t generally hang their dirty washing out for everyone to see.
As for the fortutitous finding of the catastrophic error – it must have been tele-whatever Mann said, or serendipity, or coincidence. One of those Great Moments in Science. I feel privileged to have been around to witness it.
I feel privileged to be able to read some of the wittiest math/science humor ever to be published. “tele-connections” indeed.
I second what Craig said. Thanks for the smile johanna!
Why, thank you. For my favourite lighthearted take on the Hokey Stick, Minnesotans for Global Warming’s “Hide the Decline” is hard to beat. It’s on youtube – won’t give a link here because it has been deleted and changed a few times. But, check it out – it’s a hoot – J
In the post, it is assume that, in June 2012, relative to UTC, Melbourne time was +10, Switzerland +2, Eastern -4, CA blog time -5.
So Steve, JeanS et al working for free can see the faults in a paper in a couple of hours. The paid buffoons who wrote the paper cannot see it after months of work and then discover it all by themselves 2 hours after Steve, JeanS et al do.
Of course one of them appeared in THAT video bragging about ‘peer review’…oh the irony!
Remarkable!
Oh and I’m sure Steyn would love copies of Mann’s emails, it seems he doles it out but cannot take it.
That’s not exactly fair. When you are deep in a piece of work you are very often the last person to find a simple oversight in the data processing chain.
It’s entirely plausible that the attention given the paper on 31st drove the guys to double check their work.
The timing suggests an alternative scenario for the actual discovery. We will never know. The kid knows of course.
Steve I take a different, harsher, view than you. It is entirely plausible as you say for them to make a simple oversight in the data processing chain. But I don’t buy it, because they have a duty to the people paying for their research, i.e. the ordinary Australian taxpayer to whom Aus$300,000 would be a fortune, to do take every reasonable step to ensure they haven’t.
It would be have been simple for them to consult someone in the University with statistical skills to check out their work before peer review. I’m sure they would have found a willing statistician to help them, else they could have called on Steve Mc who no doubt would be delighted to help them get their statistics right. As it is people with statistical skills took less than 48 hours to see the paper was fatally flawed.
It appears to me that the problem with climate science is that climate scientists are masquerading as statisticians and simply don’t have the necessary skills.
Cui bono.
We are not dealing with nincompoops here.
They know the methodology and, after all the years of discussions both public and private, they CHOSE to present the data to review, expecting it to pass with minor modifications.
The reason for the quick “turn-around time” and the round-robin of consultations, is simply passing along word of being caught out.
Andy – You say “The paid buffoons who wrote the paper cannot see it after months of work and then discover it all by themselves 2 hours after Steve, JeanS et al do.”
But it’s worse than that. From the article on the affair we learn that:
A PIONEERING paper on climate change has been put on hold after a mix-up in its methodology was identified.
The study, published online last month by the US-based Journal of Climate, was led by a University of Melbourne scientist Joelle Gergis at the head of a 30-strong international team.
30-strong international team? To write that paper?
Andy, sometimes a fresh set of eyes can see things quickly that the authors themselves fail to see. It’s more due to authors lapsing into tunnel vision than stupidity. It becomes stupidity when the authors adopt a reactionary attitude and treat criticism as the enemy rather than as an opportunity for improvement.
There is an active and vibrant community of amateur astronomers who hunt comets and supernovae. There is an established protocol for recognizing who the first observer was along with credit and naming rights. The timing of discoveries, all based on UTC, can get down to just several minutes. The 2-hour gap alone is enough to make the credibility of Gergis, Neukom and Karoly go supernova.
The script for “Karoly’s Choice(s)” by The Independant Scientist Alliance.
(from Prof. Bunyip: http://bunyipitude.blogspot.ca/2012/10/karolygate-ii.html#links)
Hi Raphi and Joelle,
Following some email discussions with Mike Mann and helpful discussions with you both last week, there appear to be several different approach es that we can take with revising the Australasian temp recon paper. I am going to go through some of them briefly, and then raise some suggestions for further data analysis that might be needed.
1. Amend the manuscript so that it states the actual way that the proxy selection was done, based on correls that included trends and were significant at the 5% level. The calibration was also done using the full data variations, incuding trends, over the calibration period. As Mike Mann says below and in the attached papers, this is a common approach. Don’t seriously address the proxy selection for detrended data
2. Revise the manuscript to present results for reconstructions based on both proxy selections for full correls and proxy selections for detrended correls. Expand the paper to show both sets of results and explain why the full correls are better.
3. Re-do the analysis for proxy selection based on what the manuscript says, proxy selection based on detrended carrels, which gives only about 9 selected proxies and only one prior to 1400. No reliable reconstruction prior to 1400.
4 . Redo the analysis based on proxy correlations with local/regional temps at interannual and decadal timescales, not the Australasian area average; select proxies that have strong local temperature signals, then average the proxies to get the area average temperature. This approach is like what Raphi is doing for the SH paper, I think.
My preference is now for 1. or 2. above, and not for 3…..
They look to Michael Mann for advice? Amazing.
Karoly earlier favored #3 when he wrote (6 June, emphasis mine):
Karoly is entitled to change his mind, of course, but it seems a rationalization — it’s “a common approach” not to detrend — rather than a change of conviction.
Oops, the bolding didn’t come out as intended. Karoly initially describes detrending as “much better” and “really important”, and agrees that a hockey stick shape is forced by not detrending. His later writing, suggesting detrending is not necessary, does not address why he is no longer concerned by this tendency.
I rather enjoy channeling Feynman, so I will again. If he were visiting here he would again reiterate that “the first principle is that you must not fool yourself, and you are the easiest person to fool”. Then he would state that a scientist must be interested only in the truth and not one’s own status in the scientific community. So, were he Gergis, et al., he would WELCOME all criticism of his work (and in fact he would have postulated as much of such criticism as he could think of in his original paper) and would give 100 PERCENT of the credit to CA/Jean S. even if he had, mirabile dictu, coincidentally discovered this fundamental error at the same time. And, in fact, I believe that Steve McIntyre would have done exactly the same, mutatis mutandis.
Agreed. That’s why I said above:
It just could have been mirabile dictu ie remarkable to relate and in that case it should have been related. In other words credit should have been given to CA and Jean S.
Does Jean S want credit? If so, why not notify the authors directly with his/her concerns? If not, why make an issue out of attribution?
Was this some type of trap set by JeanS? To prove what exactly?
Steve: the obligation to properly credit is part of most academic codes of conduct. Instead, Gergis and her coauthors chose to taunt Climate Audit and made untrue and defamatory statements to editors of Journal of Climate and others.
Think of it like a Twelve-Step Program. It wouldn’t be the end, or even the beginning of the end. But it would be a start.
I must heartily agree with Bob K.
I cannot recall the number of times that I’ve:
Taken code to a “layman”.
Begun an explication of the problem and what the code was supposed to do.
And then mumbled “Never mind,” stumbling off to fix the glaringly obvious error(s).
When it comes to prose, I cannot reliably proof my work until I have forgotten what I thought I wrote. Otherwise, I see what I meant to write, not what’s on the page. (Hope there’s no typos in this)
You know, I could believe they discovered the error without reading Jean S’s comment about it if they would just come out and say that’s what happened. If one of the authors made a blog post somewhere that said:
I’d be willing to believe it. It’d seem weird, but I wouldn’t assume someone was lying just because they happened to discover something at about the same time someone else did.
But if they won’t say it, how can we possibly believe it?
Steve: even if Neukom discovered the problem at say 22:42 June 5 Swiss time – an hour before Jean S’ post – it is absurd for Karoly to suggest that he was examining screening correlations “independently” of the topic having been raised at CA. They would have acknowledged realclimate without blinking. Prior to the email dossier, the public position of Karoly and Gergis was that, by coincidence, they had discovered the problem on June 5. This position is very much closed in by the dossier which shows that Gergis and Karoly did not learn of the problem until June 6 and that they altered their announcement to an earlier date.
Steve, I agree. That’s why I specifically included a line about discussion at Climate Audit causing the authors to review their methodologies. It is completely impossible to deny credit is deserved for at least that much. The fact they won’t do so discredits everything else they say.
Coincidences do happen, but nobody should believe something is merely a coincidence if the people saying it is one are dishonest about the situation. If they are not up front and honest about one thing, it is reasonable to assume they are not up front and honest about other things.
It would be easy to say: scrunity at CA led us to another chk of our data and methods. We found an error which was also identified by jean s
Yup. So would various other acknowledgements that were never made.
i dont see why the team doesnt have a fixer on board
The lead author (Gergis), was unaware of the error.
Neukom may have found the error around they same time, however, there is no evidence.
The only person to have found and DOCUMENTED the error “first”, was Jean S.
As this error is a significant discovery, credit must be given where credit is due. The required acknowledgment would have been a simply statement for Jean, and there is still time in my opinion for that to happen. (but not long.)
Without it, it’s plagiarism. One) They produce the required documentation, Two) acknowledge Jean, Three) Melbourne Uni takes action.
Melbourne Uni has zero tolerance for plagiarism. I suggest a letter to them.
In sports, my kids learned that you shake hands with the other team at the end of the game. In pick-up games of basketball that I played in college, it was always a matter of honor to call a foul on oneself before anyone else could call it. In writing a paper, one is supposed to cite people who have done relevant work whether you like them or not. If you back-date an invention to get a patent, you can get in big trouble. there is an acknowledgements section at the end of a paper so you can thank people who helped but were not coauthors. This denial of credit and anger that someone found an error in your paper is just so unseemly, so unsportsmanlike, so ungentlemanly. Who are these people? Would they deny that they fouled you in a game? I am guessing they would.
There may be a third theoretical possibility not yet considered.
That Neukom had discovered the error earlier, but did not communicate for whatever reason until it was in the public domain anyways.
But, similar to intellectual property, the issue is not about who pretends to have had an idea first, but who was the first to make it public.
Exactly, If you have an answer on an exam which equals 6676, and the next 3 people have put the incorrect answer as 6766, they will be busted for plagiarism.
“But I honestly got the wrong answer!” Yeah, sure you all did.
The onus of proof is on Gergis….
This is arguably even worse. Let’s flesh this out a little. Let’s suppose that Neukom realized that he’d goofed when the screening topic was first raised at CA (say June 1). He was aware of the issue and might have realized the problem almost immediately.
It’s entirely possible that, in such circumstances, he would hope that nobody else would notice and let sleeping dogs lie. But then Jean S spotted the problem. Neukom and the Gergis coauthors were monitoring the CA thread. Neukom now had to fess up to his coauthors and stayed up to 2 am to do so.
The irony I’m sure you’ve noted is that if in fact Neukomm noted it before Jean S published, then he was hiding it. One or t’other, damned either way.
===========
re:’Mann also made defamatory remarks about me to Karoly:
Well I’m afraid Mclntyre has probably already leaked this … but don’t trust him to behave ethically or honestly here, …’
OH THE IRONY!!!
I was wondering how this was reported and found this, seems to imply that to me that Manfred is on to something.
From the Australian
A PIONEERING paper on climate change has been put on hold after a mix-up in its methodology was identified.
The study, published online last month by the US-based Journal of Climate, was led by a University of Melbourne scientist Joelle Gergis at the head of a 30-strong international team.
It was reported as the first large-scale reconstruction of Australasian climate and confirmation that the period since 1951 has been the warmest in 1000 years, an outcome consistent with an increase in greenhouse gases.
The results are to be the Australasian region’s contribution to an Intergovernmental Panel on Climate Change report on past climate.
But print publication has been put on hold after one of the authors discovered that the paper wrongly described how the data had been processed, according to team member David Karoly, professor of meteorology at Melbourne.
Professor Karoly said the team would carry out a fresh analysis of the data, which could produce different results, and had done “the scientifically ethical and correct thing” by alerting the Journal of Climate editors.
He hoped the paper would be ready to go back to the journal for peer review by late July or early August. “Each time (there is an error in a study) you curse yourself and hope that it doesn’t happen in the future,” he said.
“It’s better that we admit our mistakes — and it’s not even clear that it’s a mistake.”
He said a key step in the study was to establish the relationship between temperature variation and the response of natural systems, such as tree rings and ice-cores, by looking at the period (1920-1990) when yearly temperature records were available.
Once these natural responses had been “calibrated” they could be used to estimate temperature variation far back into the past.
There was a choice of two methods for the data analysis; one using only the year-to-year temperature variations, the other using this data as well as the long-term trend for the 70 year period.
Although there were respectable scientific arguments for including this long-term trend, Professor Karoly said the Gergis team had intended to use just the year-to-year variations.
“We felt that by including that longer term trend you’re including part of the answer that you’re looking for,” he said.
The McIntyre website, Climate Audit, says “the irony in the Gergis situation is that they tried to avoid an erroneous statistical procedure … which is not merely condoned, but embraced, by the climate science community”.
The website suggests that the way the Gergis study inadvertantly analysed the data is circular and distorts the results.
Professor Karoly said the data would be reanalysed using the year-to-year variations only. A switch in the computer code was wrongly set to include the long-term trend and this went unnoticed.
http://www.theaustralian.com.au/higher-education/climate-paper-flawed/story-e6frgcjx-1226393519781
Nice to see that someone in the MSM is paying attention.
I thought emails were adjusted to local time?
And yes they are, and the reason I mention that is they are not adjusted 100% all the time, but probably are.
I suppose the FOI we really need is Ralphi.
He is a Raphael, not a Radulf.
God+Healer and not Counsel+Wolf.
Hebrew and not Skandinavian.
1″Amend the manuscript so that it states the actual way that the proxy selection
was done, based on carrels that included trends and were significant at the
5°/o level. The calibration was also done using the full data variations,
including trends, over the calibration period. As Mike Mann says below and in
the attached papers, this is a common approach. Don’t seriously address the
proxy selection for detrended data.”
2″Revise the manuscript to present results for reconstructions based on both
proxy selections for full carrels and proxy selections for detrended carrels.
Expand the paper to show both sets of results and explain why the full carrels
are better.”
“That’s enough for now. I am coming around to the idea that the current
analysis is fine, but we need to explain why it is ok to use proxy selection
based on teh full temp record, rather than the detrended data.”
Wouldn’t 2 be more scientific? Karoly is only interested in getting a certain result, shame Melbourne Uni, shame.
As I commented in mid-June:
1) “we didn’t cherry-pick our data and the results show it’s worse than we thought”
Bright neon lights, “meeja” sound and fury, full Bollywood treatment, AR5 tick
2) “well, err, actually we did cherry-pick … but, so what ?”
No bright neon lights, no Bollywood “meeja” treatment, AR5 tick remains indelible
That is how hearts and minds are won. Which is the whole point of such exercises – propaganda. And it works
Indeed science by press release and great claims in public which later fall down under review , seem to be one of the standard ways of working in climate science. Ironically its the very political nature of this area of study which encourages this and with a largely willing and unquestioning press you can why its done. But it does not make for ‘science ‘ worth a dam.
In the light of this correspondence, there are some interesting comments by Karoly in June 15th edition of The Australian as preserved by Skiphil here. The full article is paywalled, so if anyone can locate a free version I’d appreciate that.
[Steve: http://www.theaustralian.com.au/higher-education/bloggers-scientists-claim-high-moral-ground-on-climate-data-error/story-e6frgcjx-1226396038196%5D
Google a full sentence from it and click through from the search results to see the whole thing – many paywalled newspapers have this agreement with google.
Thanks so much for keeping an eye on these logic-choppers for us all Jean and Steve. It’s comedy gold if nothing else.
You can easily find a full version with a bit of googling, for example google “Then Karoly stepped in. In an email to McIntyre” and click first link seems to work.
It’s a good article, by the same writer two days later than the one quoted by Short Backward Square. For example this is one of the most accurate descriptions of CA I can recall seeing in the MSM:
And Jean gets a mention:
Thanks, FergalR and Paul. In fact I did that already before but could not access the article. Now with little experimenting, it seems that a plugin I’m using prevented the access. Disabling that I’m able to access the full article through Google.
IMO Bernard Lane should have another ring to Karoly. Here’s a little icebreaker:
Interesting, both content and tech. The Australian giving way to Google is part of a major global climbdown by Rupert Murdoch, as was laid out in a stimulating article by John C Abell of Reuters a month ago. And I confess I’d not seen the Electronic Freedom Foundation’s efforts to piggyback https on top of the normal sloppy encrypted/unencrypted mix. The tradeoffs abound, as ever.
It is amazing the degree to which climate scientists feel free to make untrue statements. Karoly’s assertions are flatly contradicted by the emails. I’ve sent an email about this to Bernard Lane, author of the article, which seems to have tried to give a reasonably balanced view.
Re: Steve McIntyre (Oct 30 07:46),
it’s funny what you find once you start googling (caveat: I know nothing about Alan Jones or his show beyond what is said in Wikipedia): Shock jock Jones told to get ‘factual accuracy’ training.
Maybe they still have room for Karoly in the “factual accuracy” training Jones is attending.
Re: Steve McIntyre (Oct 30 07:46),
did you hear anything back from Lane?
Naive question: does the blog host obtain/record a list of accessing IP addresses?
Steve: CA has been hosted on wordpress since Dec 2009. I don’t know whether the information is available through wordpress; in any event, I don’t know how to access it if it is.
The correspondence also gives new light on the question when the paper was withdrawn. After lengthy email exchanges (and IMO very professional approach by Journal of Climate editors Chiang and Broccoli) the editors finally granted them a hard deadline after which paper would be considered withdrawn/rejected.
Since we know they did not meet the deadline and we know for sure that the paper was officially withdrawn by September 9th, I think it is safe to assume that the paper was officially withdrawn by the end of July. However, when the University of Melbourne updated their information page before August 2nd to reflect the change in the resubmission date, they left the text “Print publication of scientific study on hold” untouched. Misleading to say the least.
That’s what they do best, mislead. Anything for a hockey stick.
These FOI papers have sopme serious principles under attack. Did you know for example that about 70% of the way through the Karoly emails, the paper was actually canned by the Editor?The Editor then suggested a complete revision, with about a month to go before the submission date for AR5 of the IPCC.
A suspicious person might think that pressure had been put on the Editor to loosen the stance. Anyone want to do another small FOI? I think it preposterous that there is a chance that this paper or a derivative will find its way into AR5.
There is at leat one author who is writing of a refusal to part with data outside the period of decades that are subject to discussion about detrending or not. All authors should note that the original request was for a list of papers rejected as well as accepted. They should also note that it is possible that the paper could be improved by cooperation.
Re: Geoff Sherrington (Oct 30 06:34),
let’s say so that I have a gut feeling that Steve is going to make a post later today about the handling of the paper 😉 So you may want to save discussion on these matters to that thread. Stay tuned.
Geoff –
The idea of publishing data of rejected proxies only for the screening period was apparently a compromise suggestion by van Ommen. It is true that this would have enabled full replication of the published results, in particular the screening phase. However, it would not necessarily have alleviated doubts about “data peeking.” In earlier emails, it is revealed that at least one dataset was still pre-publication, which is a restriction with which I can sympathize. [If I recall correctly, FoI regulations have an exemption for research data which has not yet been published.]
The clubbiness disappoints me — why data is provided in confidence to some but not to others. I understand an occasional need for confidence [e.g. pending publications], but not for the exclusivity. Scientists don’t have to like each other or agree with each other, but there are certain standards of professional courtesy which should be adhered to. As you say, cooperation can produce improvements.
Michael Mann’s comment that ‘Mclntyre has probably already leaked this anyway’ only makes sense to me if Mann misread David Karoly’s email to which it was a reply.
Karoly wrote to Mann (9 June 2012, p179 of the Karoly pdf):
My guess is that Mann assumed Karoly had asked Steve McIntyre to hold off posting anything until Monday. I can’t make sense of his reference to ‘leaking’ otherwise.
Re: DR_UK (Nov 2 18:57),
Mann’s typical ranting projection makes no sense, really, but you are probably right that he imagined that Steve had something to “leak” from the Karoly saga. There was no “leak” involved, simply critical discussion of publicly available materials (more limited than they should have been due to intransigence and non-archiving by climate scientists). Mann’s advocacy of unethical stances and Real Climate posting deceitful statements during the very period that Mann is ranting about Steve M’s ethics provides particularly delicious irony in this sad saga.
SI: “it is remarkable that three RC authors (Schmidt, Mann and Steig) each contact Gergis coauthors within hours of CA posts.”
Don’t underestimate your influence Steve. You taught these guys probably before they knew it that non-detrended data naturally creates hockey sticks. So their focus is obviously on how to create more of them.
This story here smacks not of conspiracy but of one or two with the reins bringing along a bandwagon. When reality sets in, it morphs into trying to coax the horses up a steep hill that they balked on. I am fascinated with this. I can’t wait to see that discussion of why detrending the data for proxy selection is a bad thing. It must be a detailed blow by blow, step by step description of the reeducation curriculum that Karoly went through over few days. I am dying to see it!
“– the term “hate mail”, as used by climate scientists, appears to include anything that is merely critical.”
Stephen, that is standard practice for many neo-Marxists/post-modernists, which the majority of climate alarmists are.
One reason is that their underlying ideology treats words as though they create reality, instead of describe it.
But broadly, it is a con artist’s technique to try to put the monkey on your back. Criticize someone for bad behaviour, such as dangerous driving, and they may respond with “If you can’t say anything nice don’t say anything.” (perhaps “polyanna” applies to that one), or try to put words in your mouth or claim you did something – both lies. They’ll go with lines popular in the news – such as claiming your are bullying them with your verbal criticism, and adopt other’s lines (such as Iran using expressions popular with neo-Marxists in the US, and tyrannies in SE Asia claiming denigration of females is their “culture”).
A very mild way to describe some of those tactics is “smoke screen”, another is “trying to get some mud stuck on you” (to discredit you of course). Wriggling and minimizing is common, not as serious on a scale of behaviour.
I remember you and others like Anthony Watts being the target of such scummy tactics before. It has been used against me.
The neo-Marxists/post-modernists/climate alarmist types are hypocrites – complain about hate speech but they scream at people they don’t like, making wild accusations.
(Others do to, but IMO much less often. We did recently see religious conservative writers invoke a case of sexual abuse in describing Michael Mann’s academic/political behaviour, that was stupid authoring – such writers are loose cannons.)
PS: People like you, Anthony, and Roger Pielke Jr. have observed that climate alarmist “scientists” have a tendency to support their like-minded peers when they’ve been called out for making false statements or unethical behaviour (the cases of Joe Romm and [theif who took documents from the Heartland Institute] for example).
That is common with neo-Marxists/post-modernists – who must support the collective, and with garden variety citizens I describe earlier (whose spouse/friend etc. will blindly support them).
Keep in mind these people are not fully rational.
3 Trackbacks
[…] Audit reports on the discovery of the error here, and the battle with the Journal of Climate […]
[…] https://climateaudit.org/2012/10/28/g…orrespondence/ Pretty interesting e-mails on the Gergis paper that was discussed here months ago. The paper has since been withdrawn as it had a major error but the interesting point of this post at CA is just how petty these "climate scientists" are. Climate audit starts its discussion on the paper. E-mails between the scientists in question show that they are following that discussion at CA. CA then finds the error. Then the authors of the paper who are following the discussion claim to the journal that they had independently discovered the same error. They lie to the journal about when they discovered the error and also about when CA discovered the error so that they can claim that they had discovered the error independently and first. Now the paper is withdrawn never to be published and that is the best thing as the paper was the usual hockey stick crap. But he sheer pettiness in these e-mails is repulsive. You cant give the skeptics one ounce of credit less you lose the appeal to authority war. Mens Sana in Corpore Sano Reply With Quote […]
[…] Being assumed climate specialists does not require any verification apparently, a paper on these subjects can be “Pal Reviewed”, whenever needed by their mates to give it the thumbs up before being presented to the wider scientific community, it does not have to be verifiable apparently, or so Gergis thought, pershaw!!. However, it does still have to pass inspection in the wider community and be able to be reconstructed and tested by anyone who questions the paper, it’s data or assumptions, that is how it is supposed to work. Enter Climate Audit maths guru Steve McIntyre. […]