In today’s post, I’m going to present a narrative of Richard Tol’s efforts to obtain rater ID and timestamps from the University of Queensland. See pdf here. While there have been a number of references to these efforts, few readers and commenters will (or can reasonably) spend the time to pull the threads together.
Tol requested data including rater IDs. In his initial responses to Tol, Cook undertook to provide the requested data, but asked for a delay due to his busy conference schedule. Cook initially promised to provide the data quite soon, but failed to deliver. Tol then moved his requests upstairs at both the University of Queensland and the journal Environmental Research Letters: Ove Hoegh-Guldberg and then DVC Max Lu at U of Queenslandl and Dan Kammen and then Paul Hardaker at ERL. In all cases, Tol was assured that Cook would provide the requested data, but needed time to properly anonymize the data, with the parties requesting an allowance for Cook’s busy conference schedule.
In mid-August, Cook delivered a file to Tol, which, instead of “meticulous anonymizing” of rater names, Cook withheld rater ID and timestamp/datestamp information by simply deleting the columns, something that could have been done in less than a minute. One can’t tell from the present record whether this was Cook’s plan all along, or whether he embarked on this course only after Hoegh-Guldberg “realized” who Tol was and fostered or condoned an environment in which hostility to Tol was sanctioned.
DVC Max Lu appears to have particularly misunderstood the situation. After Cook’s delivery of data withheld rater IDs, Lu “explained” to Tol that Cook could not deliver this data without breaking conditions of his ethics approval. Lu even told Tol that providing him with rater IDs would be a violation of Australian law. We now know that there was no ethics approval for the SKS ratings program and that the native datafiles were already anonymized to rater IDs.
Although University officials repeatedly provided false information to Tol, there’s no reason to believe that they did so intentionally. However, their communications to Tol were copied to Cook, who had an opportunity to correct the record on many occasions, but failed to do so. Nor did the University officials apologize to Tol, upon learning that they had misrepresented the situation. Instead, they made rude remarks about him and condoned such attitudes on Cook’s part.
But rather than try to summarize or editorialize further, here’s a narrative pdf. I’m not sure why I bothered, other than I had collated some of the correspondence to get an idea without planning to spend much time on it. Then I figured that I’d finish pulling it together for other readers and wasted too much time on it. Andrew Gelman http://andrewgelman.com/2013/09/30/using-the-aggregate-of-the-outcome-variable-as-a-group-level-predictor-in-a-hierarchical-model/ has an apt description for this sort of situation, that he used to describe his attitude towards Structural Equations Models (Lewandowsky’s technique). He recalled an assignment at primary school where a classmate was invited to write an essay. The classmate wrote: “I got locked in a pay toilet and couldn’t get out.” At first, Gelman thought that his classmate ought to have written more, but on reflection, he realized that there was nothing more to say.
Climate Audit 
37 Comments
SM,
I salute your patience chronicling this sad saga!
I salute Tol’s patience and persistence.
I am disgusted by the behaviour of Cook.
I am saddened by the ineffectual behaviour of the University hierarchy.
Wow, I just read all 18 pages of a long pdf file. Rather than an ambiguous opinion based on not having the required data, I’m convinced from this that Cook has much to hide. There are just too many similarities in reply from past history. In addition to being a purely political piece, the paper is an unsupported Mannian Crockery. Imagine what would happen if that data were actually released!
Steve: the data is now available thanks to Brandon and, as I said before, I don’t think that there’s any smoking gun in it. It reminds me of Mann’s excel spreadsheet thing which was a pointless lie and a lie that was sure to be discovered.
Thanks Steve.
Here I sit all broken hearted.
=================
Tried to scrib and only tarted.
==================
..and is the S.I. unit of lies a “thin tissue” or a “Bayeux Tapestry”?
No relation to the subject of the post.
But on page 9 of the PDF at the bottom, should the second “expurgated” be corrected to say “unexpurgated”?
Cook’s response was partly expurgated, but in the expurgated portion Cook said:
On page 17, should it say “Tol’s” instead of “Cook’s”?
citing as supposed supporting evidence Cook’s efforts to obtain research data
which it characterized as “harassing” and “threatening”:
Cook was so busy he couldn’t send a file to a fellow scientist but had time to rate 1200 articles playing the volunteer citizen in his own Consensus project. Priorities at the UoQ are different.
Just wanted to express support for Anthony Watts having read his post that is the first trackback below. Comments are disabled, because discussion isn’t needed, but I recommend everyone read it.
Richard, I tried to find the offending post on Dr. Seitz’s web site last night, without success. Perhaps someone gave him the word that he was over the top, and he pulled it down. One would hope so. Ridiculing someone’s physical disabilities is beneath contempt.
Steve: no such reasonableness from the contemptible Seitz. See http://vvattsupwiththat.blogspot.ca/2014/04/the-deaf-of-global-warming.html and here
Said Russel Seitz systemically denigrates science, specializing in ad hominem attacks.
Why should the university release the data when all you want to do is find something wrong with it?
As Physics Nobel Laureate Richard Feynman explained scientific integrity to Caltech grads in 1974:
Perhaps an interesting side-note that relates to the above oft-used quote…
With apologies to Dr. Tol … Isn’t Cook technically correct that his study methodology/results can be replicated with only the data he provides?
(Perhaps the issue then is something other than being able to replicate inferior methods to achieve questionable results). How can that be solved though, if the standard out there is simply that the authors provide just the ability to arrive at the same conclusions at which their work arrives..?
No. The rater IDs are necessary to establish whether there are rater biases. You could replicate his explicit results presumably, but that is a simplistic effort and is pretty much the universal reason for not disclosing data and methods properly. It is precisely what Phil Jones (was it PJ?) was explicitly avoiding in the “… all you want to do is find something wrong” quote. The IDs tell you whether there were any systematic effects in the rating and if they correlate to specific IDs.
No one wants to have someone tell you your methods are so rank they could be tracked by smell by a blindfolded undergrad through the Cairo dump. On the other hand, if you suspect your methods really are THAT BAD, or worse know they are, you have only a few viable choices: retract the work (and re-enter school to learn a trade you can really succeed in – possibly dish washing), hide, or get aggressively defensive and try to redirect the debate. Science tends to attract the latter personality more than the others.
To do science you have to be confident your ideas are better than the next guy’s, or if the next guy’s are good, be ready to steal them. One of the dirty little secrets in science is that type of personality is why so many practicing scientists don’t actually publish that much. They are driven by curiosity, not the goal of enlightening the world. Climate science, especially “the team” is apparently occupied by fairly marginal types that are not that confident to begin with, but still have a lot of ego.
On page 17:
citing as supposed supporting evidence Cook’s efforts to obtain research data
which it characterized as “harassing” and “threatening
I think that should be “Tol’s” not “Cook’s”.
‘I got locked in a pay toilet and couldn’t get out’
has ben rated at 4a, the author is clearly writing in support of the concensus on anthropogenic global warming
Cook and UQ plainly did not learn the watergate/Nixon or Lewinsky/Clinton lessons. The coverup is always worse than the crime.
One can only hope there might still be further repercussions Down Under.
We all understand about John Cook, so why waste the effort in chronicling more of his egregious behavior?
So that:
People generally don’t dealing with the messy, low-quality world of climate literature so they take solace in the clean world of climate physics and the like. Depressingly however, this is what you have got, out there – a ‘paper’ by climate activists, promoted by a US presidential political campaigner serving as journal editor, tweeted by the president himself.
Ah, so the hand puppet has put the need upon science to cleanse itself.
There’s a legend that Ernest Hemingway was challenged to write an entire novel in 6 words, and produced:
“For Sale, baby shoes, never worn.”
What does rater ID and timestamps mean? The rest of the article I understood but still have no clue what the aforementioned are and why they are important even after reading the PDF.
John,
Rater ID refers to the identity of the person who rated a given abstract (there were 11944 abstracts). About 12 people rated most of the abstracts. So, you could say, rater #1 rated abstracts 1, 2, 3, 5 and 6 as 4, 4, 3, 3 and 2 respectively.
Rater IDs are needed to assess inter-rater reliability, and thus validity of the rating process.
Timestamps tell you when a rating was performed. When you have a limited number of persons rating/evaluating a large mass of data, following a set protocol, fatigue and stereotypy become significant factors in determining ratings rather than any purported content of the target material. If you rush people, they’re likely to click, click, click – checking off the same categories. You can check for this with time-stamps.
I don’t know much about databases but I would guess they could record a timestamp for record creation. Cook has however denied.
This is one of the most poorly written reports of this kind I have ever seen. I could not extract facts from it so overwhelming is the ego of the writer.
For your information the report describes what happened in the order it happened. The technical term being “in chronological order”.
Reblogged this on I Didn't Ask To Be a Blog.
David L. Hagen gives me too much credit.
The bizarrely selective view of the climate science literature expressed here very much recalls what Richard Feynmann had to say about ‘nuclear winter’.
Steve: I try to write accurately and if you believe that I’ve made errors anywhere, I would appreciate it if you would draw them to my attention. In addition, if I’ve expressed opinions on the larger questions that you believe to be incorrect, again please draw them to my attention. You should also understand that one of the major “communications” problems for alarmists is that so many of their spokesmen are unlikeable, unpleasant or, as is the case in your contributions, contemptible.
Feynman had lots to say about the proper pursuit of science. You(Seitz)obviously weren’t paying attention.
What Feynmann told me about the serial selection of parametric outliers by Sagan’s cohort in order to to force their model to extraordinary exteremes in the name of the Precautionary Principle applies symmetrically to the anecdotal selectivity and serial susceptibllity of climate contrarians to crank hypotheses. My congratulations on recusing yourself from many of Watt’s more bizarre– and short lived – lines of argument, but he’s always coming up with new distractions, and if you really wish to avoid further ridicule on that account you really ought to prevail upon WUWT to cease being so patently ridiculous.
Steve: You made the following accusation “The bizarrely selective view of the climate science literature expressed here” [referring to Climate Audit]. I asked you to support your accusation and instead of providing any examples from Climate Audit, you referred to WUWT. Again, I ask you to provide support for your allegation that I have in any way endorsed “crank hypotheses”. If you have none, I can only conclude that you made your accusation without any basis and will assess other statements by you accordingly.
I have criticized poor practices in paleoclimate, such as use of Finnish lake sediments contaminated by agriculture or strip bark ring widths provide magic thermometers for past climate. Or the deletion of adverse data (“hide the decline”). Etc etc. These criticisms seem to me to be proper statistical and scientific practice and, to date, I am unaware of any coherent or rational response. It further baffles me why IPCC supporters continue to endorse this sort of nonsense. Such continued endorsement exposes IPCC to the “further ridicule” that does their cause a disservice.
Even by the low standards of many websites, I was taken aback by the juvenile and coarse standards at your website. Over the years, it has seemed to me that many more people who begin with an open mind become “skeptics” by being offput by such coarseness by climate scientists than through reading WUWT.
In addition, I found your satire of Anthony Watts’ deafness disability to be particularly offensive. That you have not abjectly apologized on this point – regardless of what you think of WUWT – is contemptible.
Seitz, you originally claimed “bizarrely selective view of the climate science literature expressed here”, and were asked to be specific.
Now it’s about Steve needing to prevail upon WUWT.
Seitz, cometh thee with empty holster?
Steve on Russell Seitz (10:20 PM):
That’s surely true.
That needed to be said.
All that need be said about Watts’ website is that the thing speaks for itself.
Steve: you made an accusation about Climate Audit. In a previous exchange, I asked you to back it up, in particular asking that you provide evidence for your allegation that I have in any way endorsed “crank hypotheses”. I observed that I had criticized poor statistical practices by climate scientists, practices that people in the field, including yourself, ought to have criticized long before me. Instead of providing evidence from Climate Audit, you unreponsively re-iterated claims about WUWT while once again failing to provide any evidence that I have in any way endorsed “crank hypotheses”. If you purport to be a “scientist”, you should concede that you had no support for your allegation about Climate Audit and withdraw your allegation with an apology. You are contemptible.
Come on, Russell. Do tell explicitly what it was about Climate Audit.
We don’t want to make the equation; Russell Seitz = The Empty Holster.
“All that need be said about Watts’ website is that the thing speaks for itself.”
Odd behaviour that you invest yourself in speaking about it, then, isn’t it?
One Trackback
[…] Steve McIntyre writes in detail about the sorts of gyrations that went on behind the scenes with Ove Hugh-Goldberg and John Cook, […]