For over a year, John Cook and the University of Queensland have repeatedly refused Richard Tol’s requests for information on rater ID and timestamps for the SKS ratings for Cook et al 2013. Recently there have been two events that shed new light on the dispute. First, in mid-May Brandon Shollenberger located the requested information online without password protection, which he placed online a few days ago. The new information shows that the majority of ratings were done by coauthors and nearly all ratings were done by coauthors and named acknowledgees, rather than by anonymous volunteers. Second, Simon Turnill received an FOI response from the University, that showed that the University did not make ANY confidentiality agreements with SKS raters. More surprisingly, Cook had done the SKS ratings program without submitting an ethics application for this program or obtaining ethics approval. Previously, both Cook and the University of Queensland had made public statements referring to “ethical approval” and confidentiality agreements. Each of these statements is, at best, misleading, especially when parsed in the light of this new information, as Brandon has done.
I’ve re-drafted this post to better reflect the lede, now beginning with the new information and moving to parsing of the statements, rather then the opposite.
Majority of Ratings Done by Coauthors
As many readers are aware, Brandon Shollenberger recently located the SKS ratings data that Cook had placed online (at the aptly named website http://www.welloiledcatherd.org) without password protection on the ratings data. A few days ago, Brandon uploaded this data to an online mirror. Brandon also preserved the online images as they appeared to him at archive.org: for the TCP Results page here and the ratings data here (to demonstrate that the information was not password protected in case the University tried to so argue, as SKS had done with their Nazi images).
The long withheld information shows that majority of ratings (54%) were done by coauthors, including Cook himself, with an additional 34% done by acknowledgees named in the acknowledgements to the paper, as shown in the pie chart below.
Figure 1. Pie Chart of SKS Ratings by Rater
Seven raters (Cook, Nuccitelli, Green, Richardson, Winckler, Painting and Skuce) are named as coauthors, while 7 more raters (Jokimaki, Reitano, Honeycutt, Scadden, Tamblyn, Morrison and Coulter) were named in the Acknowledgements to the paper, where they were thanked for “rating abstracts”.
|
|
|
2783 of 11944 papers had more than two raters. In 83% of the cases, the final ratings were given by one of the authors. Of the 9161 papers that were only rated twice (agreement in Final), in 82% of the cases, at least one author rated the paper. In other words, only 14% of the papers were entirely rated by non-authors.
In my opinion, it is “of scientific value” (a term that will be discussed later) to know that coauthors were also raters and, indeed, had done the majority of ratings and this information should have been reported in the original paper and disclosed to Tol at the time of his original request.
No Ethics Approval for SKS Ratings Program
Recently, under Queensland FOI, Simon Turnill of Australian Climate Madness requested copies of any confidentiality agreements, agreement on intellectual property and ethics applications and approvals regarding Cook et al 2013.
In response, the University produced NO confidentiality agreements, NO agreements with third parties on intellectual property and NO ethics application or approval for the SKS ratings program. Here are the FOI documents.
They only include an ethics application for the author self-rating program, but, this application refers to the SKS ratings (for ~12000 papers) as already having been carried out by parties described as “Team members”. Nothing for the SKS ratings.
The only alternatives are that (1) the University withheld responsive documents i.e. the ethics application for the SKS ratings program and confidentiality agreements with SKS raters; or (2) there are no such documents. The latter seems far more likely.
Parsing University Statements
Over the past year, both Cook and the University have made a variety of statements in which they’ve tried to connect their withholding of SKS ratings to obligations arising from ethics approval, while disguising the non-existence of ethics approval for the SKS ratings program. As too often, one has to watch the pea very closely. Brandon Shollenberger has done so and, while I do not necessarily agree with him on all points, the following exegesis reflects his comments.
In this post, I’ve not gone back to the University’s correspondence with Tol. This is an interesting topic on which I have work in hand that I’ll try to write up. Today, I’ll deal with the most recent statements by the University.
UQ Legal Threats
On May 15, shortly after Brandon announced that he was in possession of the withheld data, Jane Malloch, counsel to the University of Queensland, wrote a legal letter to Brandon, which, among other assertions, stated that the SKS data was property of the University of Queensland which had “contractual obligations to third parties” in connection with this property:
The intellectual property in the data set (the “IP”) you have in your possession is owned by The University of Queensland. The University of Queensland has contractual obligations to third parties regarding the IP. Any publication of the IP will expose the University to civil actions from third parties.
Indeed, it was this letter that prompted Simon Turnill’s FOI request. However, according to the documents produced under FOI, there were no confidentiality agreements between the University and third parties nor any agreements between the University and third parties (SKS raters) under which the University acquired the intellectual property. These claims by the University in the above paragraph appear to be completely without foundation.
Response to Tol by Cook, Lewandowsky and others
In their recent response to Tol’s published Comment (published online by the University of Queensland, Cook, Lewandowsky and others stated:
The release of privacy-protected identifying data discussed in T14 [Tol 2014] is unnecessary to replicate the C13 [Cook et al 2013] survey, and the data was withheld to protect the privacy of raters who were guaranteed anonymity.
Timestamps for the ratings were not collected, and the information would be irrelevant. Two timestamps would be needed for each rating: rating-started and rating-ended. Moreover, the time to complete an abstract rating is dependent upon several factors such as the length of the abstract, technical level of the abstract language, and interruptions occurring during the rating. Hence T14 is incorrect to state that this information (which does not exist) would shed further light on C13.
All data relating to C13 of any scientific value was published at http://sks.to/data in
2013… The only data withheld was information that might be used to identify the individual research participants. This protocol was in accordance with University ethical approval specifying that the identity of participants should remain confidential and was approved by the publisher.
First, datestamps are included in the data that Brandon located. If authors are going to publish statements that deny the existence of timestamp information without disclosing the existence of datestamp information, readers are equally entitled to have little confidence in anything that they say without consulting a Philadeplhia lawyer. Further, in a letter to unidentified associate on July 30, 2013, Cook said:
ERL said I didn’t have to include time stamp info but I’m probably going to anyway, just to show Tol’s fatigue theory is all rubbish.
It seems odd that the system that Cook used to collect datestamp information would not also have collected timestamp information (all ratings data were in chronological order, including many ratings from the same day. In August, Cook had been instructed by the UQ ethics officer to preerve all data pertaining to Cook et al.
Second, as Brandon observes, the discussion of the release of ratings information is in two different paragraphs, separated by the discussion of timestamps, and the vocabulary in the two paragraphs is different.
In the earlier paragraph about SKS raters, there is no explicit reference to “ethics approval”, only an assertion that “data was withheld to protect the privacy of raters who were guaranteed anonymity.” Precisely what form (if any) those “guarantees” took remains unknown. Nor is it known who made the guarantees or on what basis. It’s hard to understand how a University could “guarantee” anonymity to coauthors: the idea is absurd.
According to Brandon’s exegesis, Cook took the position that rater ID information on SKS raters was “of no scientific value”, whereas the rater ID information on author self-ratings was “of scientific value” but withheld under different reasoning: because of the ethics approval relating to the author self-rating program.
All data relating to C13 of any scientific value was published at http://sks.to/data in 2013… The only data withheld was information that might be used to identify the individual research participants. This protocol was in accordance with University ethical approval specifying that the identity of participants should remain confidential and was approved by the publisher.
Brandon (not justifying but trying to get inside the mind of Cook and Lewandowsky) argues that one is left with a dispute over what is “of scientific value” – the sort of dispute that goes on all the time – but that the statements are not untrue on their face when narrowly parsed, even if the overall effect is misleading.
In today’s note, I won’t review the prior correspondence with Tol. However, it seems to me that University administrators did not recognize the difference between the ethics application situation with the author self rating program (where there was one) and the SKS ratings program (where there wasn’t) and that Cook allowed the University officials to persist in this misunderstanding. When SKS rater IDs were discussed, the ethics application for author self ratings would be pointed to, tricking the unwary.
But Cook is walking a tightrope here and it’s hard to keep everything straight. In the above text, obvious questions arise about who guaranteed anonymity to the SKS raters and on what authority. Problems also arise when University officials, not fully cognizant of the trick, make public statements, as I’ll discuss next.
The UQ Press Release
In May 2014, the University of Queenland issued a press release with the following language:
All data relating to the “Quantifying the Consensus on Anthropogenic Global Warming in the Scientific Literature” paper that are of any scientific value were published on the website Skepticalscience.com in 2013. Only information that might be used to identify the individual research participants was withheld. This was in accordance with University ethical approval specifying that the identity of participants should remain confidential.
This language tracks the second paragraph of the statement by Cook and Lewandowsky discussing the author self-rating program, but omits any mention or disclosure that “data was withheld to protect the privacy of raters who were guaranteed anonymity”. Clearly the University press officer didn’t realize that Cook and Lewandowsky were walking a tightrope here, but the net result is that this language is untrue in respect to the SKS raters.
Conclusion
The larger issue is, of course, the contradiction not faced by “climate communications” theorists e.g. Dan Kahan who are blind to the corrosiveness of misleading/deceptive statements by climate scientists and supporters on matters that can be verified (as in FOI disputes) on their expectations to be trusted on larger issues.
Nor is it easy to understand the purpose of some of these machinations. As I’ve said before, I took zero interest in Cook’s study (or in “skeptic” protests against it) as it seems evident to me that there is a “consensus” of climate scientists on many points. I believe that the strength of the “consensus” varies by proposition and that too often climate promoters will bait-and-switch from consensus on something relatively uncontroversial (e.g. GHG having some impact) to green solution fantasies, but that is a different story.
Nor do I think that there is some smoking gun in the rater ID data. So it’s hard to understand why Cook made such an issue of it. But we’ve seen very odd conduct from climate scientists: think of Cook and Lewandowsky on the SKS link, Jones on non-existent confidentiality agreements on data, Mann on excel spreadsheets, etc etc. On matters which can be understood and verified by non-clmate scientists, we’ve seen bizarre behaviour by prominent people in the field.
In drafting this post, I chatted briefly with Lucia about this seeming blindness. Lucia wrote (in her usual forceful style):
Yep. I don’t see how people can’t see that if UQ lies and climate scientists just seem to think that’s ok, then the public will see the climate scientists as likely to be lying on other things. We are seeing tons and tons and tons of “how to communicate” documents, but none seem to point out the obvious: We need to stop being caught lying. Oh… here’s a strategy to stop being caught: Don’t lie in the first place!
Both Cook and Lewandowsky were, of course, involved in a previous incident also involving lying: see here, a conclusion which Tom Curtis of SKS also reached in respect to Lewandowsky (see here) but not Cook, though, in my opinion, the evidence against Cook is overwhelming.
Climate Audit 
104 Comments
I don’t agree with the portrayal given for the first two quotes of this piece. In both cases, important context is left out. I’ll repeat the first quote with additional context:
If you only look at the second paragraph here, the answer is unquestionably false. Data which could not possibly be used (on its own) to identify individual research participants, such as datestamps, was not released.
However, there is nothing to indicate that paragraph should be read in a vacuum. The paragraph before it specifically limits the data being discussed to only that which is “of any scientific value.” Cook et al have said timestamps would be of no value. I disagree, but it is their stated position. Given that context, I’d have no reason to assume the second paragraph covered datestamps. Similarly, the second quote has additional context:
Again, we see the context indicates only data considered to be of scientific value is being discussed. Another aspect is repeated though. In both sources, we’re told the only data withheld was that which could be used to identify “research participants.” The research participants listed in John Cook’s FOI request are only authors contacted to perform self-ratings. Skeptical Science raters were not called this.
Based upon those two points, the simplest interpretation is these quotes did not refer to Rater ID information at all. They, like date/timestamps, were considered to be of no scientific value. Based upon that, they were excluded from the discussion in context not presented in this post. Under that interpretation, the quotes referring to withheld data only refer to six self-rating entries filtered out by John Cook. They don’t refer to rater IDs, date/timestamps or other things like them.
As I see it, the quotes are accurate representations of the position adopted by John Cook and the University of Queensland regarding what data is of value. I suspect they intentionally wrote these things to be difficult to decipher, but I think they can be interpreted in a sensible manner that doesn’t require assuming they ever referred to Rater ID information.
Steve: I’ve expanded the quotes to include the sentence you consider relevant. I don’t think that it affects the meaning but am happy to let readers decide.
”
The paragraph before it specifically limits the data being discussed to only that which is “of any scientific value.”
”
Any data (such as rater ID’s that you requested) which might affect the analysis has “scientific value”, hence I don’t see how anyone can reasonably make the claim they did.
It would be an interesting parallel to state that, according to my survey, <1% of respondents didn't feel they were racially or sexually discriminated against at work, without revealing that all respondents were white males, for example.
Kneel, I agree their position is wrong. It is, however, an unsurprising position. There are plenty of past parallels. For instance, think about how many times someone involved in paleoclimatology filtered out some data. How many times did we hear them say the data not used was of no value?
They were wrong. John Cook and the University of Queensland were wrong to say date/timestamps were of no value. They were also wrong to hold Rater ID data as having no value (assuming, of course, you agree with my claim that was their view). All of this unused data has value, and they are wrong to say otherwise.
All I’m trying to point out is even though they’re wrong on this point, their view is consistent. They didn’t think Rater ID or datestamps were of value. Because of that, when they discussed what data of scientific value was withheld, they weren’t considering Rater ID or datestamp information.
Steve: This leaves a couple of questions, They conspicuously omitted stating/mentioning that they had withheld rater ID and timestamp information – both of which had been at issue with Tol. Misrepresentations can occur by omission as well. While I think that their statements contained actual misrepresentations, I think that your argument implies that their misrepresentation occurred through (intentional) omission rather than overt misrepresentation. I’ll try to parse this some more and get back on this.
I don’t think most of this was intentional deception, by omission or commission. The way I read things, the people other than John Cook were just people with a poor grasp of the subject trying to give honest answers. I see the same sorts of problems in discussions all the time. People think they understand a subject better than they actually do, and the exchange winds up all muddled.
To be honest, I don’t think most of the people understood things well enough to be deceptive. The only person who seemed to grasp the subject well enough to manage it was John Cook, and I think he knew it well enough to play all the angles. I don’t think he ever lied or omitted a key detail, but I do think he phrased things in ways that made them less clear.
That’s a common tactic I think most readers here are familiar with. You don’t say anything wrong, and you do provide information; you just do it in a way people are unlikely to accurately interpret. Then, if someone goes a bridge too far and calls you a liar, you have cover and you can paint them as unreasonable.
Basically, I think Cook made things unclear, people tried to muddle their way through it, and if we read too much into this, Cook will yell, “Gotcha!,” and I have no idea if it was intentional.
Brandon,
I tend to think Cook and UQ were intentionally deceptive. When crafting his discussion about released data, he knew other people disagreed about whether time stamps were relevant and he certainly knew after further discussion. But he persisted in hiding behind his unshared diagnosis that time stamp data is not relevant. Obviously, when Tol was specifically asking Cook for time stamp information, which in fact had been collected, recorded and not released a rebuff with an similar to “all relevant data have been released amounts to a lie. Unless Cook forgot he had the time stamp data, it’s an intentional lie. The inclusion of the word “relevant” doesn’t save this from being a lie because someone specifically asked for time stamp data.
If someone wants to shift the argument to whether or not time stamp data are relevant, the honest thing to do is explain those data exist but the author thinks they are not relevant and prefers not to release them.
It’s all well and good to try to claim what Cook wrote is somehow not a lie. But in that case, we are starting to define “Clintonian” constructions that sound like this “”It depends upon what the meaning of the word ‘is’ is.”
lucia, I have a comment in moderation which expresses my disagreement. To restate it simply, being unhelpful is not the same as being dishonest. Stonewalling is not the same as lying.
Side note, we don’t know if timestamps were recorded. All we know for sure is datestamps were.
Brandon, I don’t know about this database in particular, but in general, these things use 32-bit Unix time, in which time and date are all one number. It doesn’t seem likely that they have a system with dates that doesn’t also contain time information.
Brandon, there is one more sentence in the paragraph – bolded below:
The “privacy-protected identifying data discussed in T14” pertains to the SKS survey. They say that this information on SKS raters was “withheld to protect the privacy of raters who were guaranteed anonymity.”
Even though they have tried to write obscurely, the bolded statement links unambiguously to SKS rater information which they say was “withheld” not because it had no scientific value, but to “protect the privacy of raters who were guaranteed anonymity.”
A couple of points.
We more or less now know for certain that there were no confidentiality agreements between the University and the SKS raters, since none were produced in response to an FOI request. Nor is it plausible that the University would “guarantee” that they would preserve anonymity of coauthors or even “Team members”. Properly informed, the University might well have taken the position that the number of ratings performed by coauthors ought to be disclosed and that rater IDs linking to coauthors were disclosable. I get no sense that the University was properly informed. This is because there was no ethics application. Had there been such a document, then, presuming that it looked like the ethics application for author self rating, many of these issues would have been addressed and there would be no speculation.
It also seems likely to me that the University officials didn’t understand that the rater program did not have an ethics application. Nearly all of Cook’s direct emails were expurgated from the FOI documents so there is much needless speculation. But it seems evident to me that, in discussing requests for SKS rater information, Cook pointed to the obligations of the different ethics application for author self rating, thereby tricking the unwary university officials, who were unprepared for such sleight of hand.
I don’t agree we can read those two paragraphs as referring to the same things. There is an entire paragraph in-between them, discussing an issue different from the Rater ID topic. That paragraph even has a clear conclusion at the end of it. That produces a logical break, suggesting the next paragraph will be about a new issue. The break in the section’s logic can be readily seen if you imagine a header between the paragraphs we’re quoting.
If you think of the section as broken up like that, we can see an evident difference in nouns used. The first paragraph we quote refers to “the privacy of raters who were guaranteed anonymity.” The last paragraph we quote refers to an:
If you’re careful when reading this section, you can read two different subjects being discussed. At first they discuss the Skeptical Science group’s rating, saying raters “were guaranteed anonymity.” They provide no information about the nature of this guarantee. Later, they discuss the confidentiality of “research participants.” In this case, they state the nature of the confidentiality as a legal one.
A careful reading allows us to read Skeptical Science raters were guaranteed anonymity in some unspecified (informal) manner, but the authors contacted to do self-ratings were guaranteed confidentiality in a formal, legal manner. This reading would mean there are two are distinct topics placed in the same section so it is incredibly difficult for readers to interpret correctly.
In other words, what they say is technically correct but intentionally designed to be misleading. Cook et al want people to believe both sets of raters had the same guarantee of anonymity, but they can’t explicitly say so without being liars. Instead, they write something technically correct but so difficult to parse no reader could be expected to understand it.
As for the idea University of Queensland representatives were fooled by a similar trick, I could believe that. It’s easy to be fooled by writing which is technically correct but intentionally misleading. I’ve always felt those representatives didn’t grasp the nuances of the topic. That is a plausible explanation for why they wouldn’t have.
I don’t agree with this post’s argument reflected in comments like:
The paper had nine authors. There is no way a person reading just the paper could know exactly seven of them were raters, much less which seven. Similarly, there are 12 people mentioned in the acknowledgments for “collecting email addresses and rating abstracts.” There is no way a reader of the paper could know seven of those twelve had done ratings, much less know which seven they were.
Not only is this logically evident, it’s evident in that people reading the paper interpreted authors and people listed in the acknowledgment sectoin as having done ratings even when they hadn’t done any. That shows people did not know which names listed in the paper performed ratings. I don’t agree the University of Queensland claimed to have done this:
But even if it had, it is clear merely listing the names of raters as authors and people to be thanked did not tell people who did ratings. It gave some idea of the identities of some of the raters, but that’s all.
And there’s always the possibility those people whose names were listed were asked prior to being listed. Some raters openly stated their activity in public forums. Clearly, they could waive any confidentiality. Why couldn’t some have done so for the paper itself?
Steve: hmmm. my point was a little different. The overring point is that there was no ethical approval requiring confidentiality of SKS raters. I take your point that the coauthors were identified as raters merely by being listed, but they were raters nonetheless. I’ll try to re-express this more clearly.
I’ve edited the post to reflect Brandon’s comments. I agree that being listed as a coauthor doesn’t per se identify a coauthor as a rater and in that sense such a lsiting would not have violated a hypothetical confidentiality agreement between the Univrsity and the coauthor. However, there were no such agreements so the entire discussion (which I started) doesn’t make any sense. While I agree that Brandon has a technical point that one cannot identify authors as raters merely from being listed (hence my editing of the text), the point is more technical than real. If the authors were raters, that ought to have been disclosed. If I’d been a rater who wanted to keep my involvement confidential, I would not have wanted to appear as an author. Once someone appears as an author, there’s no plausible way that the University could undertake confidentiality. But the whole thing gets a bit convoluted because it starts from the fundamental misrepresentation that there were confidentiality agreements in respect to SKS raters.
Brandon believes that there is some sort of “truthy” Gavinesque way in which individual statements can be parsed as true, even though, taken as a whole, a reader will be tricked. It is very difficult to pull off such trickiness and in most cases people don’t succeed. But even if they do, this is not what should be happening in climate science if authors expect to be trusted.
The issue to me is that the raters were the authors themselves, and I find nothing in their published documentation that even hints at this. So while I agree that Brandon’s points are true, they are wrapped in a lie by omission. Did anyone guess from in front that the lead authors were also doing the rating? Because that sure wasn’t evident to me in any form.
w.
Willis Eschenbach, the acknowledgements section hints at it by thanking people for collecting e-mails and rating papers then saying, “to name just those who are not listed as (co-)authors to this paper.” I think that’s it though.
Personally, I don’t understand the logic of having some raters be authors. It’s like having the people analyzing a poll participate in that poll. It seems wrong.
Brandon, thanking a bunch of people for either collecting emails OR rating papers, and to say “to name just those who are not listed as (co-)authors to this paper”, certainly didn’t read to me as “the authors did the rating on more than half the papers” … nor does it seem ethical in the slightest to do that. I mean, it seems that “conflict of interest” rings a bell for you, and I know it does for me, but it sure didn’t for the authors.
Thanks for the response,
w.
Agreed. It’s incredible how many things they managed to do wrong.
Of course, it’s kind of bad they did. With so many problems, it can be easy to get distracted from the biggest ones.
Some more details on author involvement in the ratings.
2783 of 11944 papers had more than two raters. In 83% of the cases, the final ratings were given by one of the authors.
Of the 9161 papers that were only rated twice (agreement in Final), in 82% of the cases, at least one author rated the paper.
In other words, only 14% of the papers were entirely rated by non-authors.
Brandon writes: “Personally, I don’t understand the logic of having some raters be authors.”
Understatement of the week. This “wood” gets a bit lost for your additional “trees” here now, however important you think they are.
having authors as raters whether disclosed or not. is a problem.
agree? or disagree?
Steve Mc: Let me take this. I can’t think of a good reason why one would prefer unknown and anonymous raters to named authors. Even if one felt that the named authors were biased, at least one would know who one was dealing with. But I don’t see how the authors can reasonably expect not to identify their own work. Maybe they don’t want to be criticized, but then don’t be authors. They should have ordered big boy pants from Lucia.
” Let me take this. I can’t think of a good reason why one would prefer unknown and anonymous raters to named authors.”
The raters would not know what you were trying to prove in the paper.
I just use my own example of having to rate incoming freshman composition placement essays.
If I gave an essay a 1 or 2, then they were in remedial ( I dont have to teach yeah!)
If they got a 5 they passed out.. if I gave them a 3 or 4 they were likely going to take a class from me. so you gave a score of 2 to somebody who deserved a 3 and a 4 to somebody who deserved a 5. Fishing for good students.
Ideally, you train the raters and they know nothing about the goal of the study
They do control for this with looking at author ratings..
I’m with Mosh on this one. The main reason is that if the raters don’t know the purpose of the study, then even if they have an axe to grind they won’t know which way to grind it. It could be a study either in favor of or opposed to the skeptical viewpoint, and if they don’t know which, they will be unbiased.
If the authors do the rating, who’s going to trust them? I mean if I, a known skeptic, were to rate a bunch of papers, people would justifiably accuse me of quite possibly having my thumb on the scales. Wouldn’t mean I did … but it would certainly mean I could have.
It’s the ancient idea of the blind or double-blind study.
w.
These are not random raters in a random field. Who will believe that these “consensus” raters did all this work just for the heck of it without thinking of or planning a publication about it later on? Give me a break.
This is neither blind or double blind but one eye open.
The readers of “Quantifying the consensus on anthropogenic global warming in the scientific literature” are the only ones kept in the dark till now, as even Willis admitted.
A quote from Cook et al 2013/ Methodology: “This letter was conceived as a ‘citizen science’ project by volunteers”. The authors are citizens yes, they were also volunteers yes but is this honest Scientific reporting no.
It is hard to do a statistical analysis of subjective data created by people who know what the outcome is supposed to be. As most people here probably agree, I found the whole premise/conclusion of the paper wholly uninteresting and a clear political piece, but generally right in its conclusions that most climate papers reviewed mention global warming in some way that acknowledges its existence.
I am still shocked to find out that the authors were the ones who made the data, and have no doubt that several of these particular people would uprate the perceived “data” at every turn. Perhaps others approached for rating found it just as interesting as me and left the authors with all of the work. The few suckers who jumped in for “the cause” made the acknowledgements.
Jeff, the analysis is pretty straightforward here. Given that we know from the forums that they consulted with each on the “right answers”, this ranking data has no scientific value.
Thus we see that when John Cook and UQ said they had released all data of scientific value, they did not lie. In fact, they could have released none of it and still not been lying.
In line with Brandon’s comment: “All data relating to C13 of any scientific value was published.” Most likely true. I don’t see any scientific value at all in Mr. Cook’s work.
I’ll try offering a simplified version of why I think the additional context I referred to in this comment matters. After that, I’ll stop bringing it up save to respond to others. Repeating myself over and over would waste everyone’s time.
Suppose a person creating a temperature reconstruction collected 20 proxy series. They then applied a correlation filter, requiring series have a correlation of .2 or better with temperatures of their region. The result was 15 passed. The 15 were combined with whatever methodology, and a temperature reconstruction was published. When people asked for the data used, the author responded by publishing 14 proxy series, saying he published all the data that had any value, withholding only one series because he didn’t have the right to redistribute it.
This position would be clear. Nobody would be confused about what data was released or why. The author would be wrong in saying only 15 proxy series had value, but his explanation of what was being released would still be clear. The same is true for Cook et al.
Cook et al decided datestamp, Rater ID and other material would have no value (like the 5 filtered proxy series). When discussing what data of value (the 15 proxy series) was being released, they had to anonymize some data (a portion of those 15 proxy series). Eventually, they concluded some data had to be withheld because it could not be anonymized (the one proxy series which couldn’t be redistributed).
If someone told me that, I wouldn’t say they were being deceptive. I’d say they were wrong. I’d have an no problem understanding what they were telling me even though I didn’t agree with it.
Steve: You’ve definitely spotted an analogy that close to home 🙂 and this helps me understand your argument a bit better. I must confess that your position up till now has seemed very bizarre to me. On the other hand, each analogy brings up issues of comparisons. I’ll think about this and follow up.
Brandon, no analogy is exact. As another analogy, suppose that someone calculated a variety of verification statistics, finding out that he got “good” values of one statistic and “bad” values of another statistic, but then reported only the statistics that had “good” values. When later pressed, that person argued that he had reported all the statistics that had “scientific value”. But on other occasions, he reported both values when it was to his advantage. Doubtless we can envisage such a possibility 🙂
Would one say that he had properly disclosed the results? That he was merely “wrong” in his assessment of what had scientific value? The answer depends to a considerable degree on the author’s reasons for not being clear.
In the case at hand, Cook et al failed to disclose explicitly that they had withheld rater ID information. Nor is this implied by their language.
Steve McIntyre, I offered the analogy because I thought it might be clearer. I’m glad to hear it was to some extent. As for the analogy you suggested, I obviously don’t think that person would merely be wrong in assessing what had value. There is a clear indication of dishonesty in that case because we know the person examined that data, used it and found it valuable. There’s no way to square that with him later claiming it has no value.
The Cook et al case is different. As far I know, John Cook has never tried to glean any information by comparing how ratings differed by rater. He also seems unconcerned by the fact some people rated 200+ papers in a single day. All I’ve seen from him regarding this data is apathy. That is easy to square with him saying the data has no value.
That said, there was a little discussion of comparing results by rater in the Skeptical Science forum. I believe Cook indicated he thought it’d be worthwhile. I wouldn’t rule out the possibility he knows what the Rater ID data shows and is trying to hide it. If so, I bet he’s acting on his own and the other people involved haven’t considered the issue (and they would probably try not to).
Cook apathetic?
This 97% consensus was up on Wikipedia before the ink was dry on the printers and has been used extensively by tweeting Obama’s.
Cook was not slow and not apathetic
Brandon, thanks for your responses. However, I’m puzzled by this example you give:
For me, that is data snooping and is invalid on the face of it. As a result, whether the person felt the discarded series were were of no value is immaterial. The entire procedure is of no value, it’s post-hoc selection.
This problem is rampant in climate science. Modelers routinely discard runs that don’t fit their preconceptions, and their claim would certainly be that the runs are of no value. But of course, they are perhaps the worst people in the world to make that distinction. The world would be far better off if we could see and evaluate their “no value” results as well as the “high value” results that happen to fit e.g the historical record …
So I simply don’t buy the argument that the author of the study gets to hide results at his discretion simply by declaring that they are of “no scientific value”. That way lies scientific madness.
w.
Willis Eschenbach, I agree about that being data snooping and wrong. This disagreement has been about whether or not Cook (and others) said the data would be hidden. My understanding is our host believes people said the data would be released then later changed their position and said it would be hidden. I say those people said the data would be hidden from the start.
But regardless of what they did say, I’m sure we would both agree the data should not have been hidden.
Steve: “My understanding is our host believes people said the data would be released then later changed their position and said it would be hidden. ” I do, but didn’t discuss that issue in this post, but do intend to cover it.
Thanks for the clarification, Brandon.
w.
Brandon
Sure. But if someone specifically asked “Can you provide the data you tested with the correlation filter including any that failed” and the answer was “All relevant data have been released.” That answer would amount to a lie. Possibly, you think you are managing to be tricky by hiding behind “relevant”, but you know you have been asked for data that did not pass the correlation filter. You know the other person believes that information relevant. And you know that data exist and could be released even if your opinion is it is “not relevant”.
The fact of the matter is many, many, many people call this sort of response “a lie” and recognize it as such if they are on the receiving end of this sort of thing.
lucia, I don’t agree what you describe is lying, nor do I agree most people would view such an answer as lying. There is no deception involved in the answer. It’s just non-responsive. It’s the same sort of stonewalling answer people get from authority figures all the time. I got it from my parents growing up, from teachers in school and cops when they’re on the job. Journalists see it on a daily basis. I could turn on nearly any briefing from the White House Press Secretary and find examples of the same thing.
That an answer is unhelpful does not make it dishonest.
Brandon
Calling it stonewalling isn’t any defense to the notion it’s lying. People often perceive certain types of stonewalling dishonest. Stonewalling with “Sorry. I won’t answer that.” is not dishonest. Quite a bit of other types of stonewalling is dishonest — and in fact amounts to lying.
Sorry, but the fact that your parents, teachers or cops did something doesn't make that action "not a lie".
People lie to journalists on a daily basis.
Sure. And many people recognize that these people sometimes lie.
I didn’t say the fact it was unhelpful made it dishonest. I said it was dishonest.
lucia:
Right. You just didn’t explain why it should be considered dishonest. Since you didn’t provide any reasoning, the best I can do is guess at your logic and/or just talk about whatever I feel like talking about.
So that’s what I did. It’s true what I said didn’t rebut any argument you made, but that’s only because you didn’t make any argument.
“That an answer is unhelpful does not make it dishonest.”
“It’s the same sort of stonewalling answer people get from authority figures all the time.”
No, “Stonewalling” is refusing to answer a question. That’s not the same as answering in way that’s calculated to be misunderstood. This is the way that lawyers, car salesmen, and politicians lie, while maintaining plausible deniability. In like manner, “an unhelpful answer” is perfectly okay, when it’s obvious it’s an unhelpful answer — that is, it is not intended to deceive. (“What time is it?” “Buy a watch and then you’d know!”). Even a mystical, zen-like statement, which confuses the hearer is not a lie. That’s not what we’re talking about.
Your “argument” is mostly an appeal to authority, “most people”, “from authority figures all the time”, etc. What it really boils down to is that you believe that Cook took multiple steps — conducting his project, writing up a paper, publicizing the paper, and then answering follow-on questions — in an unfortunate and careless way that by chance was fundamentally flawed at each step, and only happens to look purposeful and deceitful. (Don’t attribute to malice what can be attributed to incompetence, and all that.)
You’d almost have a point if all of the various research missteps and the misleading answers were random and disconnected. Sure, Cook could have said, “I’ve released all relevant information” in a vacuum and meant that he’d released everything that he considered to be relevant to his paper, and he wouldn’t be bothered to do more. (Perhaps you’d call this “stonewalling”.) But he didn’t say it in a vacuum.
Until several people dug, and pushed, and dug, all of these actions and statements had one appearance. Now they have completely the opposite appearance. If it’s all a cosmic comedy, a series of missteps by an ill-fated scientist-wannabe, your point is very sensible. But it’s stacking a series of “just happened to be close to reality but seriously misleading, totally by accident”events and statements together in a way that strikes me as highly unlikely.
Wayne, you say:
But you offer no basis for this claim. If you’re going to write four paragraphs, surely you can devote at least one sentence to justifying your position.
No, it is not. lucia made a claim. I said that claim was wrong. Recognizing she, like you, had done nothing to justify her claim, I discussed why her claim doesn’t mesh with what I’ve seen in my life. There was no appeal to anything other than my experience in life.
This isn’t even close to true. I’ve repeatedly said things to the exact opposite effect. Not long after the paper came out, I started saying John Cook was laundering lies through the media to exaggerate his results. A few weeks ago I flat-out said Cook lied about his results. Practically everything I’ve ever said about this paper contradicts what you claim my beliefs are.
I’d appreciate it if you’d refrain from making bold claims about my beliefs that have no basis in reality.
I’m with you on this one Lucia. To me its a crystal clear case of lying by omission.
mis·lead
tr.v. mis·led, mis·lead·ing, mis·leads
1. To lead in the wrong direction.
2. To lead into error of thought or action, especially by intentionally deceiving. See Synonyms at deceive.
“Logicman” was a rater?
How could you possibly argue with someone named logicman? ☺
Steve, this statement of theirs is extremely problematic: “Only information that might be used to identify the individual research participants was withheld.”
Researchers are not participants. I say again: Researchers are NOT participants. The distinction between researchers and participants is as fundamental as any distinction in scientific research, or the ethical review of scientific studies.
It is extremely hard to imagine anyone from an IRB or university setting confusing researchers for participants. Moreover, researchers who serve as raters in a subjective rating study are not special, and there is no custom of calling them “participants” anywhere in the world, to my knowledge.
The fact that they called them participants is a big issue. Something is very wrong. They are not going to be able to defend any action by calling researchers “participants”. Note that this study by its nature was not a conventional study driven by participants. The “participants” in the first phase were the abstracts, and in cases where they committed fraud and broke protocol, the entire climate paper would have been the participant. Only in the second phase were there any conscious participants – the scientists who responded to the survey about their abstracts. (Side note: If the scientists knew who they were dealing with, knew that it was the partisans at SS that had sent them the survey, knew what SS was, that may have influenced who responded — i.e. a selection effect — but it’s unclear in what direction it would skew, if at all.)
If the latter case proves true, it would expose another ethics issue. The threat letter signed by Jane Malloch opened: “I am a solicitor acting for The University of Queensland.” As such, the Australian Solicitors Conduct Rules (ASCR) should apply to authoring the letter.
The Senior Ethics Solicitor or the Queensland Law Society authored an article, ASCR 2012, to familiarize the members with ASCR (emphasis mine):
It will be interesting to see if the letter holds to the standard of being “honest and courteous in all dealings.”
Ms Malloch, the lying lady lawyer. Somebody should lay a complaint alleging professional misconduct.
‘We are seeing tons and tons and tons of “how to communicate” documents, but none seem to point out the obvious: We need to stop being caught lying.’
“After you have convinced people that you fervently believe your cause to be more important than telling the truth, you’ve lost the power to convince them of anything else.” One of my favorite quotes.
http://www.theatlantic.com/business/archive/2012/02/peter-gleick-confesses-to-obtaining-heartland-documents-under-false-pretenses/253395/
Lewandowsky, Cook’s mentor, supported Gleick’s forgery and fraud as follows:
Lewandowsky, like AGU and the rest of the climate community, turned a blind eye to Gleick’s forgery.
I was fascinated by an extended discussion somewhere on julesandjames where James Annan repeated a number of times that the simple logical conclusion is clearly that Gleick himself forged the document. Several other commenters repeatedly refused to accept that, and kept coming up with very far-fetched explanations for other conclusions.
Lewandowsky is probably the same: he _knows_ that Heartland is the villain, that Gleick is a good guy, and therefore it just cannot be accepted that he made up the document that fits his preconceived notions so well.
That makes them a little less dishonest: they are condoning mail fraud, not forgery – but it also makes them stupid. Who would trust people on the issue of climate science, who clearly can’t use their brains properly when it comes to this issue?
Steve: Lewandowsky’s Hoax paper contained a lie about SKS. Initially it might have been passed off as an error, but Lewandowsky doubled down even after given an opportunity to gracefully correct. Tom Curtis, who is committed but forthright, condemned Lewandowsky’s deception. In which Cook aided him and made his own lies.
There is the fun quote where Cook and a co-author were chatting in the leaked SkS forum
where said thetwere rating some abstracts whilst cross-training. And the other said whilst playing the guitar
LOL … can just imagine them x-training in their leotards 😉
How many did it in the nude? Surely there’s scientific relevance there.
==================
A program for guaranteed weight loss – envisioning John Cook reviewing abstracts wearing nothing but his SS cap.
“All data relating to C13 of any scientific value was published…”
That’s not for them to decide. They don’t get to decide what has scientific value. Something terrible is happening here. Steve, have you contacted them again? Who can we contact?
Steve: please don’t contact people. If an approach is made, there should be one well-briefed approach to a responsible authority.
Agreed. If they collect the data in the first place, then it is because they presume it possibly to have scientific value. If subsequently they decide that it does not have scientific value, then they have committed a form of the ages-old “ex post screening” no-no. This ought to be a no-brainer. If you collect it, you HAVE to publish/archive it, all of it. There have been all kinds of examples in the scientific literature where some bits of data seemed “irrelevant” to the researchers, but these same data-bits turned out to be game-changers to other eyes. (The “polywater” to-do is one such example.)
I believe the correct url for the sssssekret site is http://welloiledcatherd.org/ . If you try it you’ll get an .htaccess block.
Steve: I fixed the reference. Brandon preserved the online images as they appeared to him at archive.org: for the TCP Results page here and the ratings data here (to demonstrate that the information was not password protected in case the University tried to so argue, as SKS had done with their Nazi images).
Elephants dancing on the heads of pins.
Richard, I get the opposite impression with SkS folks – pinheads dancing like elephants.
The phrase came to mind and I just wrote it down without context. Naughty me. It wasn’t in reply to Lucia but to the whole thread – that mistake because I’ve been experimenting with ‘Reply’, at the JavaScript and HTML level, in various WordPress blogs, with and without threading.
If how many angels could dance on a the head of a pin was an obsession of yesteryear, incomprehensible to us today, the weight of multiple elephants putting paid to the experiment, however dainty they looked in those frilly skirts, was the primary image here. Blindingly obvious, in other words. As Steve said of the PDF he prepared, what a gigantic waste of time. (Paraphrasing a little.) Why does one have to delve so deep in ridiculousness in combating the non-science of climate? Even the elephant in the room had got fed up in my imaginings.
ScienceRocks, Cook refused to release data on grounds that it would violate requirements stemming from the ethics approval, on grounds that his volunteers were study participants. In return Tol wrote in his open letter to University of Queensland president and Vice-Chancellor:
Later, the same Cook and co-authors in their reply to Tol in Energy Policy, wrote:
and
This is opportunistic posturing and flexibility with the facts, as the convenience of the situation at hand demands. The objective was to not release volunteer data, so characterizing them as research participants was useful. Critiquing Tol at a later point was attractive and they performed it using Tol’s own points and against their own previous stance!
Secondly if Cook’s consistent refusal to release volunteer data citing ethics considerations were true, either (a) the later characterization in their reply to Tol was wrong and should be retracted, or, (b) the Cook et al project was conducted without needed (as per Cook) institutional oversight as the now-released ethics review shows, and therefore should be retracted.
I’ve re-drafted this post to better reflect the lede and comments, now beginning with the new information and moving to parsing of the statements, rather then the opposite. I haven’t tried to mark deletions, since the article is thoroughly re-organized. I think that the flow is much clearer as re-drafted.
I encourage readers interested in the topic to read the new draft.
There are some changes in emphasis. I’ve incorporated much of Brandon’s exegesis of UQ statements and moved the discussion of University statements down in the article so that the new facts are introduced first.
If anyone is interested in the earlier draft, I can make it available.
The paper would have been much better having used additional raters that were outside
the SKS team. I would have listed the name of each rater, the number and name of the
papers they rated, the scoring system that was used, their scores for each paper, etc.
Full documentation, full transparency. I see only 14 raters for about 12,000 papers, that works out to about 857 papers per rater. I can’t imagine reading and rating nearly 900 scientific papers in a short timespan.
Steve: they rated the abstracts. Cook did 30 while exercising for half an hour on an exercise bicycle. Given that many of the articles had nothing to do with estimating climate sensitivity, this isn’t as ludicrous as it sounds.
Steve, your final sentence in the OP appears to represent me as having agreed that John Cook had lied about posting a link to the Lewandowsky survey on SkS. This is the second time you have misrepresented me in this manner. John Cook made the incorrect claim, but as an honest error – a point I have always explicitly maintained. I expect a retraction of your claim, and an apology for misrepresenting me again on a point on which you have already had to correct yourself.
Steve: I don’t think that the language said what you interpreted. I wasn’t trying to overstate the position as the evidence is compelling enough without overstatement. In any event, I’ve re-stated it to clarify your position on Cook as opposed to Lewandowsky, though, in my opinion, the evidence of Cook baldfacedly lying to Chambers is overwhelming, but I understand you disagree on that and wish to accurately reflect your views.
Lewandowsky ‘lied’ to me personally (or was he just utterly incompetent) , the email exchange has been published, he claimed to have the url for the survey at Skeptical Science, but could not find it… Wayback machine and Tom Curtis have shown it never existed.
Tom, you say:
When I was younger, I noticed that if I made a mistake once, I was prone to making the same mistake again. In squash as in life. So I try to avoid making the same mistake twice, but not always successfully.
To do so, I try to carefully examine mistakes. Could you assist me in this program by giving me a link or further particulars to the earlier incident in which I supposedly misrepresented you on this point. I examined the threads at https://climateaudit.org/2013/03/28/lewandowsky-doubles-down and https://climateaudit.org/2013/04/03/tom-curtis-writes and, even in hindsight, my remarks seem measured and accurate. Indeed, I went an extra mile to remind readers that your concession on the Lewandowsky issue did not mean that you had abandoned any of your concern about global warming or that you were not sharply criticizing me elsewhere on other matters.
In connection with the dispute at the time, you conceded that Cook’s statement was wrong, but felt that it could have arisen through honest error. I note that Cook has had ample opportunity since that exchange to correct the error (as you and I both try to do) but hasn’t done so.
You opened your side of the exchange by sharply criticizing me as follows:
But you later apologized and explained that it was late and you were tired when you wrote earlier exchanges (possibly including the above).
For ease in consulting the previous exchange, here are some links to relevant comments by yourself and me:
March 28
https://climateaudit.org/2013/03/28/lewandowsky-doubles-down/#comment-407867 SM 17:52 discussing Curtis
https://climateaudit.org/2013/03/28/lewandowsky-doubles-down/#comment-408058 TC 21:31
https://climateaudit.org/2013/03/28/lewandowsky-doubles-down/#comment-408044 3/29 20:50
March 29
https://climateaudit.org/2013/03/28/lewandowsky-doubles-down/#comment-408051 TC 3/29 9:15
https://climateaudit.org/2013/03/28/lewandowsky-doubles-down/#comment-408076 SM 3/29 10:19
https://climateaudit.org/2013/03/28/lewandowsky-doubles-down/#comment-408079 10:24 3/29 Mosher
https://climateaudit.org/2013/03/28/lewandowsky-doubles-down/#comment-408100 11:29 3/29 Curtis
https://climateaudit.org/2013/03/28/lewandowsky-doubles-down/#comment-408181 17:12 TC
https://climateaudit.org/2013/03/28/lewandowsky-doubles-down/#comment-408197 19:07 TC
https://climateaudit.org/2013/03/28/lewandowsky-doubles-down/#comment-408187 DGH 18:13
April 3
https://climateaudit.org/2013/04/03/tom-curtis-writes/ 8:55 10:55 Eastern
https://climateaudit.org/2013/04/03/tom-curtis-writes/#comment-409502 4/3 9:37 TC
https://climateaudit.org/2013/04/03/tom-curtis-writes/#comment-409524 10:35 AM
https://climateaudit.org/2013/04/03/tom-curtis-writes/#comment-409527 10:39 TC
https://climateaudit.org/2013/04/03/tom-curtis-writes/#comment-409539 11:21
John Cook’s only “evidence” that he posted the link to SkS (which has never been found, even in searches of the Wayback archives) is that he told Lewandowsky he had posted it. Yet (so far as I know) the respondents to the survey all came from other sites but not SkS. So it does not seem possible that the link was ever posted on SkS, which has a much larger traffic than some of the smaller blogs which did provide 100+ respondents.
Note: what Cook claims as “forensic” evidence is not forensic evidence in the modern sense, but merely his (false, or at least unsubstantiated) statement to Lewandowsky. Forensic evidence in this case would be something like digital proof that the link was actually posted, or a large number of respondents from SkS readers to the survey demonstrating that the link had indeed been posted at SkS.
[in the ancient Roman world “forensic” could mean anything presented in the forum, but in the modern usage it is evidence satisfying rigorous legal and/or scientific standards, which a mere assertion from Cook clearly cannot do.]
John Cook: “The only forensic evidence I could find was the email from Stephan asking for me to post a link and my reply that I posted it on the same day.”
Skip, I don’t know whether you followed the previous exchange, but the issue was not whether Cook’s evidence was “forensic” quality, but that there were documents on the exchange and they showed that Cook’s claim was untrue.
On August 28, 2010, Cook replied to Lewandowsky about trying the survey:
Lewandowsky replied:
Cook replied:
Lewandowsky:
So Cook sent out the tweet. The above are from page 24 of Turnill-1 FOI.
In September 2012, Chambers and Cook corresponded about the non-existent SKS link, in the course of which COok said:
Simon Turnill then submitted an FOI for correspondence including the above, which Turnill got in mid-October. Lewandowsky appears to have sent Cook a prior copy of the emails to be released to Turnill in early October 2012. On October 29, 2012, Cook asked Lewandowsky (page 133 of Kile FOI volume 2) whether these emails had been released yet:
Lewandowsky:
Despite extensive contemporary discussion of the non-existent SKS link, in the SI to the published version of Hoax (March 2013), Lewandowsky doubled down, adding a section of the SI devoted to an analysis of SKS traffic to support the claim of skeptic participation in the Hoax survey. Curtis was annoyed by Lewandowsky’s false claim and didn’t understand why he would make it.
I published a blog post on Lewandowsky doubling down on his falsehood https://climateaudit.org/2013/03/28/lewandowsky-doubles-down. Cook’s lie to Geoff Chambers was discussed in connection with this.
Curtis agreed that Cook’s claim to have “forensic evidence” was false, but argued that it might have been an honest error or not, proposing a unicorn email in Cook’s email that wasn’t in the FOI correspondence https://climateaudit.org/2013/03/28/lewandowsky-doubles-down/#comment-408051:
I responded https://climateaudit.org/2013/03/28/lewandowsky-doubles-down/#comment-408076, excerpted below see original for full, observing that Cook claimed to have inspected an email, and did not claim to have been working from memory:
Later Tom wrote me lengthy email with an apology, also authorizing me to post his email at CA, which I did.
Thanks, Steve! As usual your care and rigor are admirable! This is a terrific review for those of us just catching up on these issues.
fyi, I did not mean to suggest that Cook’s “forensic” claim was a central issue, only that (I think) it shows another aspect of sloppiness in Cook’s work. How can he think that his mere assertion is “forensic” evidence? But I don’t suggest that this is any major issue, merely another embarrassment to Cook.
when I asked for the referring domain info for the participants in the LOG12 survey
(ie this would show the entry point to the survey, Tamino, Deltoid, etc) and prove conclusively whether or not SKS was a referring domain.
Prof Lewandowsky did not respond at my first request. Months later, my second more formal request, Lewandowsky referred me to UWA. and the VC of UWA refused point blank to supply the data.
“However, it seems to me that University administrators did not recognize the difference between the ethics application situation with the author self rating program (where there was one) and the SKS ratings program (where there was)”
Should one of these was’s be a wasn’t?
Steve,
At one point you said “guarantee ratings to coauthors” Did you mean guarantee anonymity?
This is an interesting discussion. Brandon makes some reasonable points.
But isn’t this all secondary or peripheral to the real issue Cook and McIntyre are both trying to get at? Overriding is this: Is a change in the concentration of long lived atmospheric gg gases to levels not seen on earth in at least several million years, ultimately likely to significantly shift our climate?
So as no nothing can be proven with absolute certainty, it would seem, until, after the fact, there are both arguments pro and con. (Though the main “con” ones I see are that “the earth has not warmed” short term, which wouldn’t be very relevant to the issue, and “the earth could have warmed on its own because it has done so in the past,” which also isn’t very relevant to the issue of whether we are causing it now.) So the idea of consensus comes in.
Regarding consensus, there are people who, as a profession, are scientists. Within this group are scientists, who as a significant portion of their professional work, study long term climate, atmospheric physics, or climate change specifically.
Of this group, how many – about what percentage – are in agreement with the theory that long lived anthropomorphic change to the atmosphere is already significantly affecting the climate right now, and is likely to increasingly affect it in the future.
Of this group, how many – about what percentage – are not in agreement with the scientific theory that long lived anthropomorphic change to the atmosphere is already significantly affecting the climate right now, and is likely to increasingly affect it in the future.
It seems from a reading of the literature and the fact that most of the practicing science skeptics who professionally study the issue seem to be the same people over and over, that the latter group is a very small amount. (Also, the idea of doing studies to find people in the U.S. or world with college science degrees who believe the latter, is not really relevant to the above question, nor, separately, even very valid when such targeted surveys or lists represent only a tiny fraction of all such people with such degrees.)
That doesn’t make the idea that our atmospheric changes are affecting the climate and likely to increasingly do so correct — although I think it is – but it would support the notion that most scientists who study the issue also think the same.
But the real issues are, what are the reasons why it would not? Is it a desire to have it not, because it’s complicated when/if our changes are likely having, and increasingly likely to have, this climate shifting affect?
Steve: everything is secondary to the “big picture”. however, as an editorial policy, I discourage efforts to discuss the big picture in a few paragraphs as otherwise all threads quickly become identical. There are other blogs which are more tolerant of this, but I try to keep threads on topic.
Trying to stay within the bounds of this discussion, rather than moving off.
It seems from a reading of the literature and the fact that most of the practicing science skeptics who professionally study the issue seem to be the same people over and over,
It seems to me that most of the practicing science consensus-ists who professionally study the issue seem to be the same people over and over. That is, if Cook et al tracked the pro-AGW statements back in the papers that they rated via the references, the actual number of original papers on sensitivity, paleo-climate etc would be tiny. We all know the names at the end of those chains — Hansen, Mann, Schmidt etc.
There are, to be sure, enormous numbers of scientists who take their word for it. But the number who actively provide evidence is small. That is why rebutting MBH98 is such a big deal — if it falls there isn’t a lot of support behind it. (And almost nothing which doesn’t use basically the same techniques with exactly the same evidence.)
If Cook et al was actually about something useful, it would be the number of papers that provide unrelated evidence supporting AGW, relative to those that find issues with the CO2 theory. We’d be in low hundreds of papers, at most.
Imagine if a history paper “proved” that the start of WWI was primarily caused by Germany by counting the number of papers that assumed that they did as their starting point! They’d be laughed out of the academy!
The bigger issue that is partly lost in the discussion over whether Cook lied or was simply deceptive is that if science is involved, particularly government funded science, the public is entitled to straightforward answers that would admit any issues that arise out of the paper. Feynman stated that: “The idea is to try to give all the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another”
If you have parse a supposed “scientist’s” statement to determine whether he is trying to evade your question, the speaker is not really a scientist.
JD
JD Ohio, I agree with your sentiment. I don’t care to defend John Cook. I just think it causes problems when people go further with their accusations than the evidence justifies. I am loathe to call anyone a liar because if they can provide any other explanation, even an implausible one, people will side with them.
Besides, nobody likes lawyer-speak. People distrust anyone who uses it. You may have a hard time convincing people someone lied, but it’s easy to convince them that person was cagey. Do that, and you’ve accomplished the same task. In fact, you may have accomplished more. People generally prefer bold lies to cagey truths.
Tom Curtis –
==> “I expect a retraction of your claim, and an apology for misrepresenting me again on a point on which you have already had to correct yourself.”
Good luck expecting that apology. It struck me as likely being inaccurate when I first read it (over at ATTP I asked you for verification). So it is interesting to find out from your comment that Steve wrote that even though he was quite familiar with what you’ve said and could easily have fact checked it in any case.
But now I read that he has misrepresented you in this way before, and already had to correct himself on the issue? So the question might be asked whether the error was inadvertent or sloppiness (as I originally assumed), as opposed to deliberate.
But even after all of that, you’d think that he’d respect your request for an apology. Instead, he offers that you misinterpreted him?
Too funny.
Methinks that auditors should audit themselves.
Steve: I try to be careful when representing the position of critics. I do not claim to be infallible and, if I make a mistake, I try to correct or clarify promptly. In the case at hand, there was overwhelming evidence that both Cook and Lewandowsky had lied. Tom agreed that Cook’s claim to have “forensic evidence” supporting his claim was impossible, but Tom felt that an innocent explanation of the falsehood was possible in the case of Cook, though, not, as I understand it, in the case of Lewandowsky. Tom unequivocally agreed that Cook’s statements on the matter had been untrue.
At another blog, Joshua observed that my actual words (when read slowly) did not actually say that Curtis had conceded that Cook had lied in this previous incident (though Tom had conceded pretty much everything else), but nonetheless, I had no issue in rephrasing my language to clarify the point as I did. I do not recall all the particulars of the earlier incident – if someone can provide a link, I’ll take a look – but if someone asks me to clarify language, I try to accommodate. Usually people say thanks in such circumstance.
My only record of an apology in the prior incident is an apology in the reverse direction: Tom apologizing to me. My email records show that Tom acknowledged to me offline that he had misrepresented Cook’s statement. Tom explained that he had been tired when he did so and apologized to me for the misrepresentation, as shown in the closing portion of an email from Tom to me at the time.
Skiphil has an excellent point over @ Judy’s. The passage of time while Cook fails to correct the record increasingly suggests moving the telling of the falsehood from unintentional to intentional.
==================
thanks for the h/t Kim, here is the comment if anyone is curious:
comment at Climate Etc. on unintentional vs intentional falsehood
and I added a slightly different take here
O/T
Steve, that link led me to this link: http://tamino.wordpress.com/2011/01/30/amo/#comment-47666
Interesting conversation back in 2011 about AMO with Vaughn Pratt, Robert Way and Tamino… 🙂
Joshua:
Please describe the flavor of old, damp Adidas.
Mr. McIntyre is this the beginning of a new carrier as a lawyer?
a couple of syllogisms (premises must be argued separately, of course)
=====================================
If John Cook could demonstrate his honesty in the issues related to “Lewandowsky’s LOG12 survey and SkS” he would have done so
John Cook has not demonstrated his honesty in the issues related to “Lewandowsky’s LOG12 survey and SkS”
John Cook cannot demonstrate his honesty in the issues related to “Lewandowsky’s LOG12 survey and SkS”
=====================================
If John Cook could demonstrate his honesty in the issues related to his “ratings” paper he would have done so
John Cook has not demonstrated his honesty in the issues related to his “ratings” paper
John Cook cannot demonstrate his honesty in the issues related to the “ratings” paper
Stepping back a bit, and considering the value of Cook et al’s analysis:
The clearest-eyed analysis I’ve seen is by Jose Duarte, a PhD candidate in Social Psychology at Arizona State University:
http://www.joseduarte.com/blog/ignore-climate-consensus-studies-based-on-random-people-rating-journal-article-abstracts
“Jesus. This is a joke. A sad, ridiculous joke. And it’s exactly what you’d expect from raters who are political activists on the subject they’re rating.”
Error 7 of Cook et al 2014, “24 Critical Errors in Tol (2014)” appears patently false. The rebuttal claims, “Timestamps for the ratings were not collected…”
The Consensus Project was conducted via web forms which entered responses into a database, presumably mySQL. The TCP rating website, which was apparently on the same server as SKS (until the 3/2012 hack), was hosted by a third party. That company would have a record of every transaction which would include a timestamp for each transaction. The administrator of the Consensus/SKS sites would have access to that record. Indeed this post at SKS shows the form of the Apache server logs contemporaneous with the TCP rating effort.
http://www.skepticalscience.com/hack-2012-5.html
Note the timestamps.
For each TCP rating record in Apache,the mySQL logs would have a corresponding transaction. Even if John Cook didn’t collect the timestamps in a database field that information might have existed in the SQL server logs.
Further, and with thanks to the Lacatena SKS Hack Tale Parts I-VII, we know that SKS kept their own log of SQL transactions (in a public directory!). John Cook hoped to use that log to analyze and defeat regular SQL injection attacks.
The SKS site apparently hosted the Consensus Project until the site was hacked in March of 2012. Indeed the ratings suddenly slowed to a crawl when the hack was announced. Meanwhile the domain for the top SeKret Site was registered within days of the hack. Coincidentally that’s the same time the team was completing the first round of ratings. Accordingly the TCP SQL logs may contain a third record of timestamps for the first round of ratings.
UoQ claims to own the Consensus data. That begs, do they own the server and SQl logs? Are those logs subject to FOI?
Steve: Nice spotting. Cook also told University research integrity officials that he didn’t collect timestamp information. As to the University title to SKS documents, I recently wrote to the University solicitor and copyright administrator, asking them whether they also claimed copyright over Cook’s Nazi and Spartan images, and for copyright permission. Just so there would be no misunderstanding of which images I was requesting, I included copies of the images in my request letter. I haven’t received an answer yet.
Re: Steve’s inline comment to dgh…. Oh my! Did you really do that!?!
Steve: yes.
@DGH
I indeed always worked on the assumption that timestamps are automatically collected, and that only in exceptional circumstances would a programmer make the effort to delete the information.
Anyway, in (what he thought was) a private conversation, Cook has admitted to having collected the timestamps.
“Cook has admitted to having collected the timestamps”
Once again, it pays me to read to the end before comment; I was about to argue that I find the idea that timestamps haven’t been collected is perfectly credible – a timestamp recorded automatically is still a timestamp, but if I take no care to log things as they happen (maybe I batch them up for input at the end of the day for instance), it’s of no value in describing reality etc. etc.
But an admission trumps speculation.
@richardtol
Yes, it becomes clear that Cook did collect the timestamps. It’s quite an account that Steve provides in today’s post.
If there were any possibility that Cook cannot provide timestamps linked to rater IDs, wouldn’t he simply have said so?
I realize that absence of denial is not necessarily conclusive, but would it not be extraordinary that Cook would not simply say, “sorry but that information does not exist”??
@skiphill
Not quite. Cook is a PhD student in psychology. Every psychology textbook on survey methods tells you that you must record details about the interview itself.
Cook knows that I know that he knows this.
It’s not quite that simple. For apache, the standard common log format doesn’t capture variables for POST requests, only GETs. So it’s possible that the HTTP requests would be difficult to link back to database records, it would depend on the design of the forms. I’m less familiar with MySQL, but the docs suggest that the logs capture the SQL text. A properly implemented database layer should use bind variables (instead of building SQL statements, which can lead to SQL injection), which would mean variables are not in the logged SQL, again preventing linkage to individual records. However, MySQL has a timestamp datatype, which is explicitly there for automatically capturing timestamps. I’d be surprised if that wasn’t used, we’d have to see the database schema to find that out.
Steve: elsewhere I’ve seen references to Cook collecting information in SQL. I believe that it’s a safe assumption to assume that he used SQL.
m.t.
Read the entire 7 part series that I linked in my comment. You’ll be left with now illusions about “properly implemented database layers” in regards to SKS.
m.t.
Sorry for multiple short comments.
No we don’t know for certain that the survey was on an SQL database but we do know that SKS uses mySQL and that John Cook has used that platform in other projects. It’s exceedingly likely he used mySQL in this case (at least until the site was moved).
Regarding POSTS and GETS, the page I linked shows that their apache logs recorded both.
Steve: I think that we do know for sure that the survey was done on SQL. See here where COok says
Uh, guys. We do know SQL was used. You can see a SQL query in one of the pages I mirrored (here). The page threw an error when I visited it because of a problem with a SQL query, and the error was printed to the page.
But there is nothing about using SQL which requires collecting timestamps. I’ve used a number of different flavors of SQL, and in many cases, timestamp information wouldn’t have been available. Most of the time, I didn’t store that information in tables. Sometimes I had logs which would keep track of timestamps, but even then, I usually didn’t keep them as long as would have been required for Richard Tol’s requests.
I wouldn’t be surprised if it turned out John Cook didn’t collect timestamps. I’ve known for a long time he at least collected datestamps since I found images which showed ratings done by day (broken up by rater). As far as I know, that’s the only way he’s ever used time information. If that was the intended reason for collecting time information, datestamps would have been plenty. Cook could have easily used a data type like SMALLDATETIME for such a purpose.
Personally though, I don’t think the issue is interesting. As I’ve pointed out before, Cook told a University of Queensland representative he had collected timestamps. In their response to Richard Tol, Cook et al denied having collected timestamps. That’s a clear and direct contradiction. Cook’s only possible defense (aside from claiming amnesia) is to say he considered datestamps to be timestamps before, but now he doesn’t. That would be all sorts of silly.
Brandon, a comment on your “truthiness” exegesis of Cook. You observe that Cook took the position that rater IDs were of “no scientific value” and argue for truth-y sake that some of his subsequent statements are internally consistent if you grant him this position. Then the argument is whether this is right or wrong.
However, in respect to the author self ratings, Cook takes the position that the author IDs are of scientific value but withheld due to ethics approval requirements.
I do not believe that anyone could seriously hold the view that one set of IDs were of scientific value, while the other set of IDs weren’t. In my opinion, the “scientific value” arguments were developed ex post, opportunistically and dishonestly, even if some of the statements are truth-y.
I don’t agree those two positions are irreconcilable. Suppose all the raters did a perfect job. In that case, Rater IDs would be irrelevant as no matter who did a particular rating, the rating would turn out the same.
Author IDs would not be irrelevant though. Even if authors rated all their papers perfectly, there would still be the question of who those authors were. Perfect ratings would not inherently mean the distribution of authors was representative. For instance, perfect ratings would give biased results if the response rate were biased.
This isn’t a trivial matter either. A number of people raised concerns about the author ratings, suggesting the self-selecting nature of them was a potential source of unquantifiable bias. That, combined with a belief the Skeptical Science raters did a consistent job, would make the “contradictory” views you list perfectly natural.
By the way, I think you may have missed the real problem. John Cook’s writing heavily implies ERL told him Rater IDs were of scientific value. If that’s true, it would undermine his entire trick. It’d mean every time he referred to anonymization and confidentiality, he must have been including Rater IDs.
To consider this possibility, let’s pay attention to John Cook’s description of what ERL told him. First, he said:
He then lists some of the data which was released. However, Rater IDS (labeled 4) were listed in the next batch (the other requested data):
The line I bolded is key. Two of the seven items were not released because they “were deemed unnecessary to replicate [the] results.” Rejection of a third item was considered, but rejected, on confidentiality grounds. If that third item wasn’t “of scientific value in reproducing the results,” there would have been no reason to even consider confidentiality issues. It could have just been withheld as being irrelevant. Instead, it was published when confidentiality was found not to be an insurmountable obstacle. That strongly implies the data was considered of scientific value.
In the same vein, Rater IDs were not withheld on the ground of having no scientific value. They were withheld for an entirely different reason. That strongly implies ERL felt Rater IDs were of scientific value, just like author IDs. That would mean Cook could not possibly argue Rater IDs were irrelevant.
Of course, that is not proof. It is, theoretically, possible other reasons to withhold Rater IDs were given by ERL despite ERL saying Rater IDs were of no scientific value. Further, it is possible Cook failed to say ERL judged Rater IDs were of no scientific value while repeating a different reason ERL gave for withholding the material. That means we cannot be absolutely positive ERL judged Rater IDs to be of scientific value. The only way we could be absolutely positive is if we saw what ERL actually said.
Still, it seems incredibly likely ERL judged Rater IDs to be of scientific value. If so, John Cook knew it. In that case, none of the defenses of what he’s done could hold.* Every time he referred to confidentiality, anonymity, research participants or anything like that, he must have been including Skeptical Science Rater IDs. In that case, when he said efforts were:
He was apparently lying. He should have known Skeptical Science Rater IDs were not covered by any ethics application. Unless he just forgot or got confused, the only explanation is an outright lie. It couldn’t have even been a careless mistake as he repeated the same sentiment in another e-mail.
As I said, this isn’t proof. I do think it’s powerful evidence though. Short of ERL having sent a strange e-mail which Cook described in an incredibly strange way, Cook’s statements were not “truthy.” They were false. And short of Cook being incredibly incompetent or highly forgetful, they were intentionally false.
One problem with this interpretation is this comment:
If this statement is true, ERL told John Cook not to release Rater IDs as they weren’t necessary to replicate his results, thus they were of no scientific value. It’s hard to square that with what he said of ERL’s communication elsewhere.
*Unless one says Cook simply dismissed ERL’s judgment of what had “scientific value.” In that case, he had no basis for anything he said about what had scientific value, other than his own say-so.
Brandon, you don’t need to just be able to replicate somebody else’s result for data to be of “scientific value”. That’s a totally risible claim. Maybe there are things you weren’t interested in looking at, that somebody else could derive more value from, for example.
I have a data set that I generated back in 1991 that was reused in somebody’s thesis just recently (third or fourth person to use it). I can assure you this person interested in other things than I was interested in, when she requested the data set.
IMO, having Cook being be by ERL that he wasn’t required to publish the timestamps set doesn’t remotely imply that there is no remaining information of scientific value in the part of the dataset that he bizarrely refused to publish.
The way I see this (and these are always judgement calls) either Cook is so incompetent and not bright that he has no business publishing peer reviewed papers or he’s knowingly behaving in a deceptive behavior on this.
I would really be interested to see the full correspondence between Cook and ERL. I suspect this something that could be gotten with an FOI request.
In any case, had Cook a shred of personal ethics, he would release this correspondence without being requested to.
I took a crack at explaining the use of the word “fake” in the title to this thread, responding to comments at Climate Etc. which maintained that there is something horribly wrong with employing the word “fake” in this context. Hope I didn’t cock it up, anyone is welcome to do better:
Climate Etc. comment on criticism of McIntyre
Some of the terminology used here is misdirected (as well as sickening). Was Cook “lying by omission”, merely “stonewalling”, outright “dishonests”, “intentionally” or “unintentionally misleading”? To quote Stephen Schneider, “On the one hand, as scientists we are ethically bound to the scientific method, in effect promising to tell the truth, the whole truth, and nothing but — which means that we must include all the doubts, the caveats, the ifs, ands, and buts.” This certainly applies to all aspects of the “Consensus” paper, especially questions from other scientists, such as Richard Tol, about the work. IMO, the discussion should be centered on how ethical scientists should behave, not politicians and ambulance-chasers.
Schneider goes on to discuss how a scientist might behave outside the scientific arena: telling scary stories, oversimplifying, hiding doubts, etc. Outside the scientific arena, phrases like “lying by omission” are appropriate. If Cook believes that “blog science” falls in Schneider’s ethics for scientists, perhaps he should post a warning at SKS.
Reblogged this on TrueNorthist and commented:
The following pretty much sums up why I am a climate heretic.
It appears that Jane Malloch, counsel to the University of Queensland, made false statements in her letter to Brandon Shollenberger. Perhaps Mr. Shollenberger could file and ethics complaint with the University of Queensland against Ms. Malloch. And also file an ethics complaint against her with whatever ethics group monitors attorneys in Australia. It appears that either she knowingly made a false statement or she failed to do a proper investigation and request copies of the contracts with third parties before writing her letter.
Exactly, and this is the most interesting aspect for me. Malloch failed miserably when she did not ascertain the truthof the matter. Perhaps she accepted the word of someone else as to the actual situation. There is a can of worms in this letter. Can you imagine Malloch’s reaction when she realized that she had been duped? That her egregious professional failure is now before the whole world? I wonder who prompted the letter to Brandon- was it the same person who assured Malloch that contractual obligations were at stake?
So it appears that Malloch threatened action through ignorance, at the prompt of someone else, and relied on what she was told father than seeing for herself. Elsewhere, this sort of lapse could get you fired.
I would love to know the whole story on this.
mpainter, one interesting aspect of the letter which hasn’t really been commented on is John Cook told me I’d be contacted by a university of Queendland administrator (the Vice Chancellor of Research, if I remember right). Instead, I was contacted by this lawyer.
I assume what happened is Cook talked to the administrator, the administrator promised a response, and he fobbed it off to the legal department. If so, there’s no telling where the problems originated.
I’m still curious why Jane Malloch said the university had conducted a forensic investigation and concluded there was hacking. I can’t prove it, but I bet it stems from Cook telling them about the Skeptical Science hack. If I’m right, he either misled them about it, or they greatly misunderstood him.
Then again, Cook and a number of his Skeptical Science pals have directly accused me of hacking, even going so far as to claim I’ve covered up information about my “hack.” I guess it is possible the University of Queensland has somehow come to believe the same.
It does not seem that Malloch would have invented all of that garbage herself but was fed a lot of bs but by – who? Perhaps the administrator sent Cook to the lawyer. It makes sense and squares with what we understand about Cook and his methods. If the administrator was wise about Cook then he would have shifted himself out of the whole matter if he saw a way to. But what a sloppy attorney Malloch would have been to threaten legal action on the word of a man like Cook.
4 Trackbacks
[…] Cook’s Fake Ethics Approval […]
[…] Cook’s Fake Ethics Approval […]
[…] McIntyre in a post last Saturday writes of Cook’s Fake Ethics Approval and has this hilarious […]
[…] https://climateaudit.org/2014/07/26/cooks-fake-ethics-approval/ […]