Over the past few days, Eric Steig aka Reviewer A has made a series of increasingly puzzling and strident outbursts, as the inconsistency between his RC post of Feb 1 as Eric Steig and his conduct as Reviewer A has been exposed.
Yesterday, Steig placed his latest and wildest diatribe online at two blogs – John Nielsen-Gammon here and Josh Halpern here. Steig writes:
John Nielsen Gammon, while you seem to be supporting me, which I appreicate it, it isn’t very useful support.
I did not, repeat not, repeat not, forget “that this amounted to his own insistence on iridge as a reviewer.” I did NOT recommend iridge in the first place. I did NOT bring it up. I SIMPLY DID NOT THINK I COULD ARGUE WITH THE EDITOR OR THE KNUCKLE-HEADED REVIEWERS that is should not be used.
This is not complicated folks. O’Donnell and gang, not liking my criticisms of the way they used TTLS, and in particular the fact that the truncation parameter they wanted to use, suddently started using IRIDGE. This has the advantage of having a build in verification function, which means you can’t see what the verification statistics are, which means that it is much easier to NOT SHOW THE BAD VERIFICATION STATISTICS I was criticizing them for. Maybe that is not why they used iridge. I don’t know WHY they used IRIDGE but I did not suggest it to them nor endorse it.
GET IT?
P.S. Yes I am shouting. That’s what the CAPS mean.
P.P.S. O’Donnell gave ‘his word’ (that’s what he said) that would not reveal my name, nor print the reviews verbatim. He lied to me.
P.P.P.S So if anyone wants to speculate that hiding table S3 is O’Donnell lying again, go for it, since speculation is all most people seem to be doing these days.
Posted by: Eric Steig at February 12, 2011 04:36 PM
The full story began long ago when Steig appeared incognito in the review of O’Donnell et al as Reviewer A. I’ll review the chronologies in two states – the recent public debate involving Eric Steig and then the Reviewer A backstory.
The Public Debate
The public debate began with Steig’s RC post on Feb 1 here which contained the following criticism of the use of the iridge variant of RegEM in O’Donnell et al 2010:
in their main reconstruction, O’Donnell et al. choose to use a routine from Tapio Schneider’s ‘RegEM’ code known as ‘iridge’ (individual ridge regression). This implementation of RegEM has the advantage of having a built-in cross validation function, which is supposed to provide a datapoint-by-datapoint optimization of the truncation parameters used in the least-squares calibrations. Yet at least two independent groups who have tested the performance of RegEM with iridge have found that it is prone to the underestimation of trends, given sparse and noisy data (e.g. Mann et al, 2007a, Mann et al., 2007b, Smerdon and Kaplan, 2007) and this is precisely why more recent work has favored the use of TTLS, rather than iridge, as the regularization method in RegEM in such situations. It is not surprising that O’Donnell et al (2010), by using iridge, do indeed appear to have dramatically underestimated long-term trends—the Byrd comparison leaves no other possible conclusion.
This criticism provoked an angry response from Ryan O’Donnell who, unlike the audience, knew that Steig had been Reviewer A and who believed that Steig’s public commentary could not be justified given his conduct as Reviewer A.
On Feb 7, in a commentary published at CA and elsewhere, O’Donnell provided a chronology of Reviewer A’s requirements for O’Donnell et al, showing Steig’s suggestion of ridge regression in the Second Review, the adoption of this suggestion in the third version of O’Donnell et al, Steig’s agreement in the Third Review subject to concerns that were fully responded to in our Response to the Third Review. As it turned out, the editor of the Journal of Climate hadn’t sent Ryan’s Response to the Third Review to Steig – a point that Ryan was unaware of. Ryan promptly acknowledged and apologized on this narrow point here, while noting the Steig knew or ought to have known that the editor must have been satisfied by his response. The primary allegation of Ryan’s post remained unanswered and is what Steig is presently responding to:
So Eric recommends that we replace our TTLS results with the ridge regression ones (which required a major rewrite of both the paper and the SI) and then agrees with us that the iRidge results are likely to be better . . . and promptly attempts to turn his own recommendation against us.
On the day after Ryan’s post (Feb 8), in an email at 1:26 pm, Steig (from a personal rather than academic email address) stated:
1) I insisted that the editor require you to show what you thoughts was the ‘most likely’ result. I did not insist you use iridge. I was totally suprised when you wound up doing so, in version 3 of the paper.
2) I did not even suggest using iridge — I said that *perhaps* iridge is better, as YOU suggested. In the same paragraph I alerted you to the problems with iridge
I’ll comment later on the veracity of these statements. For now, I’ll merely note that it is, to say the least, disquieting that a reviewer, whose comments had occasioned two major revisions and a total of 88 pages of comment and reply, should be “totally surprised” that an author should pay heed to proposals in his review comments.
Later in the day (7:08 PM) in an exchange with Zeke Hausfather, Steig accused O’Donnell and his coauthors of “lying to their readers” in respect to the assertion that the original submission had relied solely on TTLS and that the choice of iridge instead of TTLS was in response to comments from a reviewer of the paper:
Eric,
O’Donnell and his coauthors argue that the choice of iridge (instead of TTLS) in response to comments from a reviewer of the paper. This, at least, seems to be a somewhat weak point upon which to critique their approach if their originally submitted work relied solely on TTLS.[Response: How can I put this succinctly? How about this; I have pointed out the facts of the matter to O’Donnell et al. They have not changed what they have written. They are therefore now lying to their readers. It’s actually about that simple. –eric]
Shortly afterwards, in response to a similar question from another RC reader, BPW, Steig again said that there was “no basis in facts” for Ryan’s statements:
3) Did you, as part of your review, ask that they change their method only to later criticize that method?
Like it or not, if you don’t directly address these accusations, the impression lay people are left with is that the whole thing smacks of dishonest use of peer review. I am not qualified to suss out the science, but i am qualified to understand O’Donnell’s accusations. I am willing to accept that there is something lost and that he is misrepresenting the situation, but if you waive away these type of questions and quash those who try to ask, what are we left to think?
[Response: Perhaps you should try thinking, instead of asking me what to think. Let me turn this question around on you: why do you take O’Donnell at his word? And now he’s my word: His allegations have no basis in fact. Now you have my word against his. Now try thinking,-eric]
The next day (Feb 9), at an RC post here, Steig made a longer response to the developing controversy, stating:
I never suggested to the authors that they use ‘iridge’. This was an innovation of O’Donnell and his co-authors, and I merely stated that it ‘seems’ reasonable. As O’Donnell’s co-authors are fond of pointing out, I am not a statistician, and I did not try to argue with them on this point. I did, however, note that previously published work had shown this method to be problematic:
“The use of the ‘iridge’ procedure makes sense to me, and I suspect it really does give the best results. But O’Donnell et al. do not address the issue with this procedure raised by Mann et al., 2008, which Steig et al. cite as being the reason for using ttls in the regem algorithm. The reason given in Mann et al., is not computational efficiency — as O’Donnell et al state — but rather a bias that results when extrapolating (‘reconstruction’) rather than infilling is done.
In keeping with RC practice, while Steig quoted from Ryan’s CA post, Steig did not link to it.
Steig’s post contained a trick (TM- climate science). It omitted any quotations from or consideration of his Second Review or the responses to the Second Review, which, after all, had been what provoked Ryan in the first place. Andrew Revkin noted up Steig’s response, but didn’t notice Steig’s trick.
The controversy reverberated through the blogs. Lucia, normally a genial observer of the climate wars, sharply criticized Steig, first in a post asking the question of whether Steig was the Rod Blagojevitch of Science, followed up by Steig the Shameless incorporating Josh’s witty image of the Eric the Red.
The matter has been extensively discussed on many blogs over the past few days, culminating for now in Steig’s explosion at Nielsen-Gammon and Halpern’s blogs.
I think that it is fair to say that Steig has vehemently denied that he, in any way, ever requested, recommended, proposed, suggested or put forward iridge as a methodology. I think that his position can be reasonably characterized as follows:
I want you to listen to me. I’m going to say this again: I did not have relations with that methodology, iridge. These allegations are false. And I need to go back to real climate research.
The Review Record
Iridge and TTLS are two variants of RegEM that build on two alternative regularization techniques in linear regression: ridge regression and truncated total least squares. Truncated total least squares – like truncated SVD and principal components regression – utilizes a rough, stepwise regularization scheme to allow the calculation of ill-conditioned problems.
In our First Submission, (which, in my opinion, contains much material that should be of great interest to readers and should have been published with minor revisions as the non-conflicted reviewers recommended), we made no mention whatever of the iridge variation of RegEM. This submission introduced the statistical issues involved in simultaneously infilling missing ground data and estimating past satellite principal components – a problem not considered by Steig et al, observing the dependence of results on a retention parameter for ground data (which we denoted k_gnd).
In the original version, we reported (using a TTLS methodology) a West Antarctic trend of ≈0.05 +/- 0.07 deg C per decade, noting the sensitivity to k_gnd. This compares, apples-to-apples, to 0.20 +/- 0.09 deg C per decade (95% confidence interval) for Steig.
For the correlation RLS and EW reconstructions pictured in the main text, the West Antarctic trend is ≈0.05 +/- 0.07 deg C per decade. Both the magnitude and statistical significance of the trend are highly dependent on the truncation parameter for the ground station infilling.
In Steig’s First Review (as Reviewer A), Steig made one unsubstantiated statement after another, all of which were biased towards coercing our results towards higher trends in West Antarctica. Steig characterized our optimized values of k_gnd as “suspect” and alleged:
The suspect values of kgnd are the only ones that fail to result in statistically significant warming in West Antarctica.
Ryan responded comprehensively to Steig’s allegation (the Response to the First Review totalled xx pages), observing:
As stated in both S9.a and S9.b, the panels showing the kgnd sensitivity tests are done using settings that result in maximum trends, without optimizing the other parameters for those particular choices of kgnd. They also do not reflect the set of reconstructions with the next highest verification statistics. When the other parameters are optimized for those values of kgnd, the overall trends and trends in West Antarctica are lower and comparable to the optimum settings for kgnd. The reviewer does not note that, in Table 6, it is clear that the higher trends for the other values of kgnd result in degraded verification statistics – both in West Antarctica and across the continent.
Ryan also carried out supplementary tests with iridge to check on the TTLS results, noting that the iridge method yielded a higher West Antarctic trend (0.10 deg C per decade) and proposing that a full analysis with the iridge method (rather than the TTLS method as in Steig et al 2009 and the O’Donnell submission) be done as “future work”.
Testing by infilling ground stations using ridge regression with the ridge parameter for each time step determined via generalized cross validation yields validation statistics, spatial patterns and West Antarctic trends (~0.11) very comparable to our main results, and performing RLS reconstructions after offsetting stations based on periods of mutual overlap (i.e., no infilling) yields validation statistics, spatial patterns and West Antarctic trends (~0.10) also comparable to our main results (¶4.C – U, 4.Y – AE). These additional tests – which form the basis for a future work [my bold] – have now been incorporated into the main text.
One important reason for deferring this to a future work was to change as little as possible from S09 to avoid the argument that the infilling method was the reason for the differences – thereby avoiding the sort of argument that Steig has now initiated.
In the second submission, the TTLS results were again reported as the primary result:
For the correlation RLS and E-W reconstructions pictured in the main text, the West Antarctic trend is ≈0.05 +/- 0.07 deg C decade-1. Both the magnitude and statistical significance are dependent on the truncation parameter for the ground station infilling.
In addition, as promised in the Response to the First Review, we reported the results of an experiment using iridge (a report that falls well short of “suggesting” iridge as our intended alternative.)
In one experiment, we perform the ground station infilling using the individual ridge regression option in RegEM (Schneider, 2001). We optimize the ridge parameter for each time step using a generalized cross validation function. This experiment yields similar patterns of change as shown in Fig. 3, with less intense cooling on Ross, comparable verification statistics and a statistically significant average West Antarctic trend of 0.11 +/- 0.08 deg C decade-1
In Steig’s Second Review, Steig seized on the slightly higher trend of the iridge experiment that we had reserved for “future work” (in the Response to First Review), suggesting that the iridge results, rather than our preferred k_gnd results, be shown in the primary visual (which Steig described as the “take home message”). Steig noted that the results were “relatively new”, attributing them to suggestions that he himself had made in the previous review, but stated categorically that there was no “compelling reason” to defer the iridge analysis to a “future work”.
My recommendation is that the editor insist that results showing the ‘mostly likely’ West Antarctic trends be shown in place of Figure 3. While the written text does acknowledge that the rate of warming in West Antarctica is probably greater than shown, it is the figures that provide the main visual ‘take home message’ that most readers will come away with. I am not suggesting here that kgnd = 5 will necessarily provide the best estimate, as I had thought was implied in the earlier version of the text.
Perhaps, as the authors suggest, kgnd should not be used at all, but the results from the ‘iridge’ infilling should be used instead. The authors state that this “yields similar patterns of change as shown in Fig. 3, with less intense cooling on Ross, comparable verification statistics and a statistically significant average West Antarctic trend of 0.11 +/- 0.08 C/decade.” If that is the case, why not show it? I recognize that these results are relatively new – since they evidently result from suggestions made in my previous review – but this is not a compelling reason to leave this ‘future work’.
In his decision letter on our Second Submission, editor Broccoli called for “further major revision” – “major revision” is a term of art for Journal of Climate requiring re-review. Since none of the unconflicted reviewers had made anything other than minor comments, editor Broccoli was clearly requiring us to make “major revisions” in response to Steig’s demands. By now, the correspondence file of review comments and responses with Reviewer A was nearing its eventual total of 88 pages. So while Steig now claims that he was “totally surprised” that we paid attention followed his review comments to heart, our perspective at the time was that we had little alternative given Broccoli’s position and Steig’s “total surprise” that we paid attention is more than a little grating.
In addition, Steig did more than merely suggest/recommend/propose iridge. Steig also insisted that the editor require us to show the “most likely” results. Nic Lewis explains the impact of this requirement as follows:
In the second version of the paper, we had said “Based on these results, we conclude that the West Antarctic regional average is likely to be ≈0.10 oC decade-1”, and made clear that conclusion was based on (a) the iridge and (b) the offset non-infilled direct RLS methods and (c) TTLS/TSVD using a truncation parameter one higher or lower than the 7, our existing choice based on it being optimal for the verification statistics.
Reviewer A recommended the editor to insist that we show as our main reconstruction one that displayed our stated most likely West Antarctica trend of ≈0.10 deg C decade-1 , and there was no reason to think that the editor would not go along with this recommendation.
Further, we were being forced to undertake major revisions for the second time, and in the circumstances it would have been foolhardy to go for a method that Reviewer A could readily attack. In my view, that ruled out sticking with truncation based methods (whether TTLS or TSVD, correlation or covariance), since Reviewer A was still attacking the choice of truncation parameter. Further, it would have been awkward to justify switching to a truncation parameter choice that did not provide the best verification statistics. That effectively constrained us, in my view, to switching either to iridge or to offset non-infilled direct RLS, as these were the other methods on which we had stated our most likely West Antarctica trend was based.
In practice, there was no contest between switching to iridge RLS rather than to offset non-infilled direct RLS; iridge was a well proven, mainstream method, whereas the non-infilled method was non-standard and gave Reviewer A scope for both arguing about its validity in general and the choice of overlap period. Further, it did not give as good verification statistics as iRidge RLS and since the purpose of our paper was to demonstrate what happens when an objective cross-validation criterion is used, this would have been incompatible with one of our main points.
In our “General Note to all reviewers (Second Review), we stated:
Based on a request from one of the reviewers, we have agreed to incorporate our “most likely” reconstructions into the main text. These reconstructions do not infill the ground station data using TTLS; instead, they utilize ridge regression. Verification statistics are mildly improved and solution stability is much improved. The smooth regularization and ability to adapt the regularization parameter to the number of predictors in ridge regression proves to be of significant benefit (which was noted as a possibility in Schneider, 2001). Because of this, the TTLS/TSVD reconstructions now serve only to show that cross-validation testing provides a superior means of determining a truncation parameter than the heuristic tool used by S09, and have been relegated to the SI.
In our specific Response to Reviewer A, we stated:
Toward the end of the review, the reviewer suggests that the editor should require us to display the “most likely” reconstructions in the main text, which the reviewer correctly assumes would be the ridge regression results. We agree that this is the most appropriate choice, and the manuscript has been revised to show the ridge regression results in the main text. The TTLS/TSVD results have been relegated to the Supplemental Information.
Additional changes to the manuscript to accommodate using the ridge regression results as the primary reconstructions have been made throughout Sections 6, 7, and 8.
Also note that the ridge results mentioned in the previous response were multiple (not individual) ridge regression results and were not optimized for the number of retained satellite PCs or regularization parameter. As this was originally intended as a second-check, we had chosen the faster multiple ridge regression method and used the same parameters for the ridge reconstructions as the TTLS/TSVD reconstructions. We have since performed reconstructions using individual ridge regression and have optimized the number of retained satellite PCs and regularization parameter for both individual and multiple ridge regression.
For the optimized individual ridge regression reconstruction, the resulting best estimate for West Antarctica is 0.10oC decade-1. Because the individual ridge regression results display equivalent or better verification statistics and least sensitivity to removal of individual station data (including the manned Byrd station) of all of the methods (TTLS, TSVD, multiple ridge regression), this is what appears in the main text.
To our knowledge, Steig received these responses to the Second Review (though he has stated that he did not receive our responses to the Third Review.)
Our Third Submission (which differed only by a word here and there from the final version), we re-cast our presentation to focus on iridge results, as we had undertaken in our Response to the Second Review.
Steig was the only reviewer who responded to the Third Submission. In his Third Review, the only one referred to in his realclimate post, Steig, while on the one hand agreeing that the iridge procedure “makes sense”, introduced fresh caveats about iridge referring to “Mann et al 2008” in connection with a supposed “bias”:
The use of the ‘iridge’ procedure makes sense to me, and I suspect it really does give the best results. But O’Donnell et al. do not address the issue with this procedure raised by Mann et al., 2008, which Steig et al. cite as being the reason for using ttls in the regem algorithm. The reason given in Mann et al., is not
computational efficiency — as O’Donnell et al state — but rather a bias that results
when extrapolating (‘reconstruction’) rather than infilling is done. Mann et al. are
very clear that better results are obtained when the data set is first reduced by
taking the first M eigenvalues. O’Donnell et al. simply ignore this earlier work. At
least a couple of sentences justifying that would seem appropriate.
Ryan responded at length in the Response to the Third Review
We have two topics to discuss here. First, reducing the data set (in this case, the AVHRR data) to the first M eigenvalues is irrelevant insofar as the choice of infilling algorithm is concerned. One could just as easily infill the missing portion of the selected PCs using ridge regression as TTLS, though some modifications would need to be made to extract modeled estimates for ridge. Since S09 did not use modeled estimates anyway, this is certainly not a distinguishing characteristic.
The proper reference for this is Mann et al. (2007), not (2008). This may seem trivial, but it is important to note that the procedure in the 2008 paper specifically mentions that dimensionality reduction was not performed for the predictors, and states that dimensionality reduction was performed in past studies to guard against collinearity, not – as the reviewer states – out of any claim of improved performance in the absence of collinear predictors. Of the two algorithms – TTLS and ridge – only ridge regression incorporates an automatic check to ensure against collinearity of predictors. TTLS relies on the operator to select an appropriate truncation parameter. Therefore, this would suggest a reason to prefer ridge over TTLS, not the other way around, contrary to the implications of both the reviewer and Mann et al. (2008).
The second topic concerns the bias. The bias issue (which is also mentioned in the Mann et al. 2007 JGR paper, not the 2008 PNAS paper) is attributed to a personal communication from Dr. Lee (2006) and is not elaborated beyond mentioning that it relates to the standardization method of Mann et al. (2005). Smerdon and Kaplan (2007) showed that the standardization bias between Rutherford et al. (2005) and Mann et al. (2005) results from sensitivity due to use of precalibration data during standardization. This is only a concern for pseudoproxy studies or test data studies, as precalibration data is not available in practice (and is certainly unavailable with respect to our reconstruction and S09).
In practice, the standardization sensitivity cannot be a reason for choosing ridge over TTLS unless one has access to the very data one is trying to reconstruct. This is a separate issue from whether TTLS is more accurate than ridge, which is what the reviewer seems to be implying by the term “bias” – perhaps meaning that the ridge estimator is not a variance-unbiased estimator. While true, the TTLS estimator is not variance-unbiased either, so this interpretation does not provide a reason for selecting TTLS over ridge. It should be clear that Mann et al. (2007) was referring to the standardization bias – which, as we have pointed out, depends on precalibration data being available, and is not an indicator of which method is more accurate.
More to [what we believe to be] the reviewer’s point, though Mann et al. (2005) did show in the Supporting Information where TTLS demonstrated improved performance compared to ridge, this was by example only, and cannot therefore be considered a general result. By contrast, Christiansen et al. (2009) demonstrated worse performance for TTLS in pseudoproxy studies when stochasticity is considered – confirming that the Mann et al. (2005) result is unlikely to be a general one. Indeed, our own study shows ridge to outperform TTLS (and to significantly outperform the S09 implementation of TTLS), providing additional confirmation that any general claims of increased TTLS accuracy over ridge is rather suspect.
We therefore chose to mention the only consideration that actually applies in this case, which is computational efficiency. While the other considerations mentioned in Mann et al. (2007) are certainly interesting, discussing them is extratopical and would require much more space than a single article would allow – certainly more than a few sentences.
We refused to make any further changes. By this time, even editor Broccoli was exasperated with Steig and approved the Third Version, more or less as is (asking only that a few words be changed.)
Reprise
Going back now and parsing Steig’s recent statements, one sees one untrue statement after another. Consider first his Feb 8 email:
1) I insisted that the editor require you to show what you thoughts was the ‘most likely’ result. I did not insist you use iridge. I was totally suprised when you wound up doing so, in version 3 of the paper.
2) I did not even suggest using iridge — I said that *perhaps* iridge is better, as YOU suggested. In the same paragraph I alerted you to the problems with iridge
Aside from the oddity of a reviewer being “totally surprised” that heed is paid to his comments, these statements are inaccurate on other points. Steig says “I said that *perhaps* iridge is better, as YOU suggested”. At the time of Steig’s suggestion/proposal/request, we had only reported iridge results as an experiment. We had not then “suggested” that they were “better”. Nor is the following true: “In the same paragraph I alerted you to the problems with iridge”. Steig suggested/proposed iridge in his Second Review; the supposed “problems with iridge” were not mentioned until the Third Review – after we’d done the major revision. Nor do we think that Mann and coauthors have actually demonstrated that there are problems with iridge that do not also apply to TTLS – an observation also made by Smerdon and Kaplan 2008. Ironically Mann himself vehemently denounced TTLS relative to iridge in 2006 in as Reviewer 2 here.
In commentary at realclimate, Steig accused O’Donnell and coauthors of “lying” on the following point:
O’Donnell and his coauthors argue that the choice of iridge (instead of TTLS) in response to comments from a reviewer of the paper. This, at least, seems to be a somewhat weak point upon which to critique their approach if their originally submitted work relied solely on TTLS.
[Response: How can I put this succinctly? How about this; I have pointed out the facts of the matter to O’Donnell et al. They have not changed what they have written. They are therefore now lying to their readers. It’s actually about that simple. -eric]
However, the record clearly shows that the originally submitted work “relied solely on TTLS” and “the choice of iridge (instead of TTLS) [was] in response to comments from a reviewer of the paper” as the reader had stated.
In his realclimate post, Steig stated:
I never suggested to the authors that they use ‘iridge’. This was an innovation of O’Donnell and his co-authors, and I merely stated that it ‘seems’ reasonable. As O’Donnell’s co-authors are fond of pointing out, I am not a statistician, and I did not try to argue with them on this point.
Again, this is not supported by the record, as discussed above. “Suggest” is defined as “To offer for consideration or action; propose”. Steig’s RC post omitted his Second Review comments in which he clearly suggested iridge as an alternative. Nor was iridge an “innovation of O’Donnell and his coauthors”. It was a RegEM variant in the original Tapio Schneider code.
Steig’s most recent diatribe departs even more from reality. Steig:
I did not, repeat not, repeat not, forget “that this amounted to his own insistence on iridge as a reviewer.” I did NOT recommend iridge in the first place. I did NOT bring it up. I SIMPLY DID NOT THINK I COULD ARGUE WITH THE EDITOR OR THE KNUCKLE-HEADED REVIEWERS that is should not be used.
Again, the record speaks for itself. In his Second Review, Steig obviously suggested iridge and argued that there was no “compelling reason” why it should be deferred for “future work” as Ryan had proposed in his Response to the First Review. Far from arguing in his Second Review with the EDITOR and THE KNUCKLE-HEADED REVIEWERS that iridge should not be used, Steig, at a minimum, suggested it.
Steig continued:
This is not complicated folks. O’Donnell and gang, not liking my criticisms of the way they used TTLS, and in particular the fact that the truncation parameter they wanted to use, suddently started using IRIDGE. This has the advantage of having a build in verification function, which means you can’t see what the verification statistics are, which means that it is much easier to NOT SHOW THE BAD VERIFICATION STATISTICS I was criticizing them for. Maybe that is not why they used iridge. I don’t know WHY they used IRIDGE but I did not suggest it to them nor endorse it.
Again, this has no relationship to the history. The TTLS results showed a lower West Antarctic trend than iridge; it was Steig himself who seized on the iridge results and wanted to highlight them.
Steig continues:
GET IT?
P.S. Yes I am shouting. That’s what the CAPS mean.
P.P.S. O’Donnell gave ‘his word’ (that’s what he said) that would not reveal my name, nor print the reviews verbatim. He lied to me.
P.P.P.S So if anyone wants to speculate that hiding table S3 is O’Donnell lying again, go for it, since speculation is all most people seem to be doing these days.
Table S3
As to Table S3, Ryan writes as follows:
Reviewer B did not like the length of the SI. We cut a few things out after the first round of reviews, but he still was clearly not happy with it. Broccoli also mentioned the length in the letter to us.
After the second round of reviews and after having switched to iRidge, we decided that much of the SI was no longer necessary, as our main results did not depend on TTLS. So we basically completely rewrote the SI and removed just about everything to comply with Reviewer B’s request. All that was left was the tables of full verification stats, the replication effort, and TTLS for various settings of kgnd.
Steig is complaining that he never saw this revised SI – in which Table S3 had been removed. This is false. He did see it. In fact, he makes this comment about it in the third review:
An unfortunate aspect to this new manuscript is that, being much shorter, it now provides less information on the details of the various tests that O’Donnell et al. have done. This is not the authors fault, but rather is a response to reviewers’ requests for a shorter supplementary section. The main thing is that the ‘iridge’
procedure is a bit of a black box, and yet this is now what is emphasized in the manuscript. That’s too bad because it is probably less useful as a ‘teaching’ manuscript than earlier versions. I would love to see O’Donnell et al. discuss in a bit more details (perhaps just a few sentences) how the iridget caclculations actually work, since this is not very well described in the original work of Schneider. This is just a suggestion to the authors, and I do not feel strongly that they should be held to it. {emphasis added}And now he mysteriously complains that the removal of the table was both unknown to him and…well… “mysterious” – when he not only saw that the table had been removed but admitted that it was not our fault and recommended only that we add some information about iRidge to explain how the calculations work. My response was:
We hold a rather different opinion of which algorithm is a “black box”. Tikhonov regularization (which is called ridge regression primarily in the statistical literature, but Tikhonov regularization elsewhere) has a substantial body of published literature dating back to the 1960s. Much more has been written concerning ridge regression than any other shrinkage estimator of which the present authors are aware. It is a far more common tool in applied mathematics, statistics, and signal / image processing than TTLS.
Schneider’s 2001 paper spends but two paragraphs (page 866) on TTLS in a 12,000+ word article. The remainder of the article is dedicated to EM and ridge regression. We disagree rather strongly that the ridge regression procedure in Schneider (2001) is not well described – it is quite thoroughly described. On the other hand, TTLS is hardly mentioned, and most of the important calculations that appear in the algorithm are not even shown, much less discussed.
The reason much of the supporting information is gone is because the algorithm that actually requires additional explanation is TTLS, and the TTLS reconstructions are no longer the source for the results and conclusions of the paper. We feel the ridge regression algorithm is well-documented – both in Schneider (2001) and elsewhere – and adding additional explanation would be redundant.
The SI was unchanged – except for formatting – between the version Steig saw and the one that was accepted.
Postscript: See Lucia’s commentary here.
Climate Audit 
214 Comments
Game. Set. Match.
OT
As usual, a thorough and detailed review. Nice to see you back in the saddle!
One minor nit: you left out the number of pages in the First Response to Reviewer A. The sentence reads:
“Ryan responded comprehensively to Steig’s allegation (the Response to the First Review totalled xx pages)”
(I think the first response, including excerpts from Reviewer A, is in the range of 47 pages).
Cheers,
Amazingly, I saw your post just after I wrote this comment:
http://rankexploits.com/musings/2011/eric-to-john-nielsen-gammon-it-isnt-very-useful-support/comment-page-3/#comment-69445
I’m trying to see how my response to Shub about how I read the order of the process compare to your report.
Looks like our reading of the chronology was pretty much identical.
My take concurs as well. Posted as a response to Shub just before Lucia’s. An obvious cross post, given the amount of time to prepare the responses.
Sorry for buggering up that tag. 😦
I have been waiting patiently to read your deliberations relating to this arguement and feel now that i can form an opinion on the matter.
Thank you for the time and effort required to present all the necessary details.
In the purported words of T.E. Lawrence, “Take no prisoners! No prisoners!” Cheers.
If he had been a bit smarter he could have taken some of your credit by suggesting that he recommended the use of iridge RLS.
they say never bring a knife to a gunfight, but perhaps Steig should not have brought mere petulance to a contest with people with photographic memory and a WRITTEN RECORD of what happened. Unbelievable.
I don’t think it matters to them what the witten record shows. What matters to them is the story they put out to their followers and their media contacts.
It’s difficult for Steig to maintain credibility with his pants around his ankles.
I had a feeling this detailed accounting of the timeline would be forthcoming from CA. Great detail and it describes well all the major arguments being forwarded. Would be ideal if Mr. Steig could read and comment on this…and perhaps we could narrow down any remaining points of contention. Thanks.
“Would be ideal if Mr. Steig could read and comment on this…”
LOL. Good luck with that!
Exposed and coming unglued before our eyes.
Perhaps Professor Jones could form some kind of support group for Climate Scientists at the end of their ropes ?
Maybe a shoulder ? “Sush, I know dear, me too”
Steig is right about one thing – he was not the one who FIRST proposed it as a methodology in O10. If that is what Steig is claiming (and that is what he seems to be saying) then Steig is correct but only on this narrow point.
But I think he is using this small truth to distract from the BIG question which is whether he contradicted his reviewer A comments in his public comments.
So if I look at what he said I see him criticise iridge publicly and say that it was worse than TTLS because it understates long-term trends. He never mentioned this in his comments. Indeed your results, which he had already seen, clearly showed the opposite (which I would guess is one of the reasons which made him encourage you to use it). On that basis I find him guilty of dodgy scientific behaviour.
thisisnotgoodtogo
Posted Feb 13, 2011 at 4:58 PM | Permalink | Reply
“Exposed and coming unglued before our eyes.”
My thoughts exactly – Steig has (from the tone of his collective comments on this matter) become manic and, again, I hope he receives some help for his problems.
He also seems fond of the punctuating remark “GET IT?”. Sounds somewhat threatening, actually…
Although this has turned into a sort of handbags at dawn isn’t there another story here that given the available data Steves own work backs a claim that warming in parts of West Antarctica is statistically significant.
“In the original version, we reported (using a TTLS methodology) a West Antarctic trend of ≈0.05 +/- 0.07 deg C per decade”
You do realize that means a temperature change of between .12c/dec and -.02C/dec don’t you?
Which means it could be cooling.
Sean – a trend of 0.05 with an error greater than that means that there is no trend.
There is a difference between a statistical test for significance and the meaning of such tests.
And when you are working with a few scattered data points around the coast, you think that says anything about the continent, with an increase of only half of 0.1 degrees?
You need to read Briggs more.
Hi. I just want to make the point that is being overlooked clear. I have been using the First Submission which states: “We cautiously conclude that the trend in West Antarctica is likely significant and may be closer to 0.10 deg C decade than 0.05 deg C decade.” Thats all I was stating.
ENTERTAINING! Steig must be a quite the sloppy record-keeper to get that tangled up.
Yea, his organizational skills look to be on par with his grammar skills.
It seems to me that Steig, through his review, was only going to like the paper if it, instead of criticising or ‘auditing’ his work, would show conclusions of its own for West Antarctic Warming– statistically significant at that. Obviously the conclusions you arrived at (0.05 +/- 0.07) is not statistically significant– and not near to what he got for an answer. Since you took the step of also calculating it using ridge (and got a statistically significant result that WAS closer to what he got), naturally he was going to jump onto that method as being ‘closer to the truth’ regardless of how valid an answer it is (which probably raises other concerns). It seems like his reply about ridge methods being closer actually had nothing to do with ‘which methods are better’, but rather ‘whatever happens to get the ‘right’ results’. This would allow him to say that he never suggested using ridge as the better method, and at the same time suggesting its results as being ‘closer to the truth’ and therefore worthy. I’m not sure this is the right attitude to approach data, is it?
Am I right in saying that Steig would have been happy (no matter what method was used) if the end results agreed with his in that it showed some Antarctic warming? Would that have been the ‘fundamental rework’ he was looking for that would have received his approval?
It seems clear that ‘significant’ contributions to the literature means anything that offers ‘new’ or ‘improved’ trendlines or reconstructions– and not criticisms of old ones, unless they are instead also published along with an author’s own conclusionary trendlines or reconstructions to debate.
I think you are exactly right, in particular and in general.
It’s amazing to see how everything is approached as though they already know what the “truth” or “correct answer is”. And that is how ES went about his review. That his methods were shown to be flawed was not all that concerning to him (maybe he understood there was nothing he could do about it) as long as the “answer” was close enough to his already pre-determined, and team approved “correct answer”, he could claim they were right all along and RO-10 provided nothing new. Nothing to see here, move along. As we already knew the world is warming………
Where have we seen that technique before?
Steig’s usage of statistical techniques reminds me of the quote my 6th form Maths teacher used at the beginning of our Stats lectures…”An unsophisticated forecaster uses statistics as a drunken man uses lamp-posts – for support rather than for illumination.” Originally from Andrew Lang I think.
Great summary. Thanks, Steve.
Seems a little strange that the authors would not have taken issue in their response with Steig’s characterization of iridge as the “most likely result,” given that we are talking about two competing statistical approaches. I would have thought that a statement in the response about the inherent subjectivity in selecting a particular approach would have made sense, together with a “However, at the reviewer’s recommendation we have reworked the paper for iridge.” That said, I know sometimes you just have to put your head down and concede certain non-central drafting points in the review process in order to keep it moving along.
Interesting to know that TTLS was only half the trend of iRidge . . .
Steve,
What about Steig’s comments regarding himself not receiving a final copy of the final draft (for his perusal)…AND comparing it to your ‘umbrage’ when in the same situation?
Is there a difference between the two cases that makes them not comparable? (Did you have a separate agreement with the editor where you agreed to review provided you got to see the final copy before press?)
Steve: the situations are not remotely comparable. I’ll try to visit this issue at some point in the near future.
Actually, Steig should not have been reviewing the paper. As the subject of the paper his proper approach should have been a “response” or rebuttal in which he compares and somehow shows why his results were actually correct. But, since O’Donnel et al. is critiquing his methods rather than his conclusions per se, that is quite a difficult tactic to achieve. Ideally, he would have to apply different methods and show that his original results were conformable. Instead he seems to have done his best to obfuscate and obstruct the publication of the paper, and then to minimize the apparent differences and their significance to his interpretations. And now, rather than address the issue as science, he is arguing the “record” of the review process, which, again, he should not have been party to.
Isn’t it up to the Editor as to who they feel could or should review the paper?
Also…It seems like there’s a difference of opinion as to what constitutes a significant rebuttal paper. Does Steig not argue that in order to criticize his methodology, O’Donnell needs to also supply a statistically significant conclusion of his own using a different methodology, otherwise whatever is written is just a review (a process that was already completed for that paper)?
This concept is just silly. It is not unusual to ask an author to review a paper that responds to or expands on his work. He was a very logical reviewer. Of course it was up to the editor to consider the background which seems like it was handled appropriately in this case.
Nicolas:
I agree Steig was a logical choice as a reviewer – but why as an anonymous reviewer. I believe knowing that Steig was definitely Reviewer A would have led to a different response to his review from O’Donnell et al. I am sure a fourth reviewer would have been more forcefully requested at an earlier point in the proceedings.
I remain puzzled why Steig chose to act as an anonymous reviewer.
Probably to avoid the kind of reactions that finally emerged when Ryan learned that Steig was reviewer A. Again it was up to the editor to determine what portion of a reviewer’s comments needed to be responded to as was seen in the third review comments that the editor agreed didn’t require a change to the paper.
However I feel that if Steig wanted to subsequently write a blog post on the subject it would have been better to acknowledge that he was one of the reviewers, and to carefully review his own review comments to be things in context.
Perhaps, but in this case you have to do some real mental jujitsu to ignore the fact that Ryan et. al. were paper was explicitly trying critique the method not the conclusion. One can only speculate why Eric wanted a conclusion, or guess from post publication behavior.
The question remains , did Steig act as reviewer should do in ensuring the O’Donnell paper was good enough to be published. The answer would seem to be no, the Seig seem to have regarded his role as the reviewer as being one in which the reviewer sort change this paper so it was more supportive of Steig09. In other words he took his own work as the bench mark of quality for this paper, which is frankly ridicules given this was review of Steig09 could have always have been in conflict with Steig09 .
Chances are he knew what effect O’Donnell paper would have on the academic value of Steig09 and having so much invested in that paper, sort to block O’Donnell or change it in a way so it’s less effective and more support of his own work. And is a real danger of using reviewers who may have a conflict of interest.
Steig as clearly made statements on hand they denied them on the other.
His acted in a way that is not fitting for a person reviewing a paper for publication. He may have broken the ethics involved if he involved the ‘team’ in this review without telling the editor. And in the end wither through arrogance or foolishness had made more trouble for himself. Steig09 got all the press , the front cover of Nature the BBC etc, in contrast O’Donnell’s paper was going to get none of that and reality would have despaired into the academic wilderness if it were not for his current actions.
Salamano:
I’m not sure Steig’s not having final review is really relevant to the current questions surrounding the switch to iridge as the principle regularizing technique… but regardless – for clarity’s sake can you specify what situation you are referencing in which his not receiving a final copy of the final draft provoked SM’s “umbrage”? I might then be able to determine for myself whether the two situations are comparable – if I can convince myself the question is interesting enough to pursue.
Great summary, thank you.
Others have pointed it out but I think it is worth repeating that there are (at least) two issues here:
1. Duplicity by Steig which this excellent post makes crystal clear.
2. That the results of S09 are random noise, not anything linked to reality, which O10 made abundantly clear.
It’s not pleasant seeing a lifetime of achievement evaporate into an embarrassing refusal to make eye-contact while shouting louder.
I felt sorry for ES. I now feel much more sorry for him. He may have been told to face up to the big guns of climate scepticism. It’s all gone terribly wrong. He took on the big boys and is now crashing and burning, all alone, on his own.
Poor Eric!
Is it within the realm of possibility that Steig as Reviewer A was actually Steig et al.? (Has he addressed this subject? I no longer read Realclimate.) Perhaps if there were too many fingers in the pie … multiple response variations may have been in circulation before the final reviewer submission was released, leaving open the possibility of saving/deleting inappropriate editions on Reviewer A’s computer.
Steve – Steig has said that he did the review all by himself, though he did say that he told Mann that he was a reviewer. The language of the review is not Mannian – compare Reviewer 2 of Burger and Cubasch at CPS, 2006
According to Dr. Steig’s inline comment, he did not show the paper to anyone else.
I personally am willing to take this statement at face value.
He has behaved arrogantly, he believes that his work is correct and being attacked by people who are not part of the climate science fraternity and who should be treated as uneducated outsiders, and he may have attempted to “protect” his work by overstepping the bounds of scientific integrity during the review process.
However, in my view, he is not in any way an inveterate liar. He could have deleted the entire comment that I referred to in my comment, but he allowed that one portion which he answered as he did. C’mon, he is not “evil” and I would gladly give the the benefit of any doubt in that regard.
Just a tyro compared to The Mann ?
“he believes that his work is correct and being attacked by people who are not part of the climate science fraternity and who should be treated as uneducated outsiders”
Agree. But is his faith that his work is correct even reasonable at this point? Regardless of all the back and forth about who is a good person and who is a bad person, I don’t see any grounds for him to argue that his paper is sound.
Just because he didn’t show it to someone doesn’t mean someone didn’t see it. It also doesn’t mean he didn’t show his review comments to someone before he sent them. I agree he was likely hiding / delaying his impending embarrassment, and he doesn’t seem sophisticated enough to use one of the sly deniability maneuvers.
No one has any evidence or basis for believing it . . . so let’s stick to what we know. Steig said the reviews were entirely his. I have no reason to disbelieve him (as I’ve said here and elsewhere).
No evidence maybe, but Team past history is a sound basis, although I understand your aim to stick to the facts.
Ryan-
my fault for pointing out the possibilities, and then saying they’re unlikely.
Salamo,
I fail to see how one can draw any comparisions between Steig and McIntyre…simply because McIntyre didnt delay Wahl and Amman for a year. In fact, McIntyre seems to insist on Wahl and Amman actually doing their job property (not relying on a rejected paper…including data in their submission etc). That to me seems to be nothing more than common sense.
Steig on the other hand appears to have been deliberately obstructing the publication of O’Donnells paper…most likely as a direct result of previous actions by the hockey team in stopping other papers from being published.
Mailman
DaveL,
Yes, Steig has addressed the Reviwer A comment by O’Donnell. While Steig publicly denied being Reviewer A in his blog…he has now set that straight and has confirmed that he was in fact Reviewer A.
Mailman
Steig quotes:
> Yet at least two independent groups who have tested the performance of RegEM with iridge have found that it is prone to the underestimation of trends, given sparse and noisy data (e.g. Mann et al, 2007a, Mann et al., 2007b, Smerdon and Kaplan, 2007) and this is precisely why more recent work has favored the use of TTLS, rather than iridge, as the regularization method in RegEM in such situations. It is not surprising that O’Donnell et al (2010), by using iridge, do indeed appear to have dramatically underestimated long-term trends—the Byrd comparison leaves no other possible conclusion.
> O’Donnell and gang, not liking my criticisms of the way they used TTLS, and in particular the fact that the truncation parameter they wanted to use, suddently started using IRIDGE.
> Perhaps, as the authors suggest, kgnd should not be used at all, but the results from the ‘iridge’ infilling should be used instead.
????????
In quote 1, he prefers TTLS and criticizes iridged. In quote 2, he criticizes TTLS. In quote 3, he says the results of iridge should be used.
I think others are right. Steig was just interested in getting the results that were closer to his own paper. Those are what he called “most likely” results. But no matter what you were going to use, he was going to criticize.
He’s not really criticizing TTLS, bit the value of the truncation parameter. I’m guessing Iridge removes the necessity of using a truncation parameter?
Iridge doesn’t involve truncation. It does use a smoother regularization method (ridge regression) where the ridge parameter performs a broadly similar function to truncation, but that parameter is not set by the user. The iridge algorithm determines the optimum ridge parameter, separately for each pattern of available data and each variable with data missing, using an objective technique (generalized cross validation).
Eric:
I did OK, didn’t I? Guys?
Gavin:
You did just great Eric. Great rebuttal. BTW, I’m offline for the next few days but will touch base ASAP.
KBO mate.
Xxx
A Team Mate.
Would it be possible to ask a question on the scientific import of this dispute between S09 and O10? As I recall, a contentious issue was the apparent cooling of Antarctica which contradicted the accepted climate models. This cast doubt on the utility of these models for predicting possible changes that could be attributed to AGW.
Am I correct in assuming that S09 tended to confirm the model predictions and O10 tended to dis-confirm them?
Is this the basis for the vehemence of the dispute about these results?
What are the next steps that should be undertaken to resolve the issue with the models and Antarctica?
My understanding:
S09: statistical garbage. its results have no meaning and should be ignored. similarities with models are irrelevant.
O10: mathematically correct but that does not mean the method is a good way to estimate temps. The results could be accurate and tend to confirm existing estimates.
The response from the team is likely ego driven. They don’t like it when their “novel” statistical methods are shown to be nothing but an exercise in data mining.
Actually, IIRC there have been claims that a ‘cooling’ antarctica was ‘consistent’ with some of the models. However, the warming antarctica is also ‘consistent’ with the models and of course actually makes sense in the context of ‘global’ warming. The model output apparently can be assessed as ‘consistent’ with a lot of different trends.
Any idea what the climate science equivalent of a one handed economist is?
A one handed “teamster” would still manage to have five ways to obfuscate — one for each digit
Regardless, the models show that the poles should be warming at a significantly greater rate than the rest of the globe. And this CLEARLY isn’t the case.
It reminds me of the lack of amplification in the upper troposphere at the equator. All the models show the upper troposphere near the equator should be increasing at a rate approx. 1.4x the rate of the lower troposphere in the tropics. The empirical record once(10yrs ago) showed that the upper troposphere in the tropics was cooling. After torturing the numbers, the record now shows that it’s slightly increasing. But the fact remains the same for both the upper troposphere at the equator and the lower troposphere at the south pole. The theory and the models show amplification, but, after much adjustment, the record shows the temps are increasing at a rate approximately equal to (or slightly less than) the long term global trend.
I don’t believe this topic has anything to do with climate models.
But ir permeates the canvas
I agree that it seems that the big picture of policy action to limit CO2 seems to cloud every scientific discussion on this topic. In fact elsewhere I have been called a “denier” by at least one blogger since I was critical of aspects of Steig’s blogging on this topic. I actually don’t have any particular view on whether Steig or O’Donnell is more scientifically correct at this point, although perhaps O’Donnell is an improvement. As to the big picture I can’t see this being probative at all. But at the same time, the Nature cover was very provocative given the uncertainty in the Steig result and the fact that it doesn’t change the big picture.
My point is….the Team is trying very hard to show that Antarctica isn’t cooling. And it may not be. But it’s certainly not exhibiting the amplification that is forecast.
I think that point is often lost in this discussion.
I thinking writing and reading ability are related. To the extent this is readable, it is silly. Steig did not raise the possibility of using iridge, since the authors did that on their own. Steig suggested the most likely results be written up in the text. The impact this had in the mind of authors, which is the sum total of your “evidence”, is irrelevant.
Re: Ali Baba (Feb 13 19:41),
The authors suggested use of iridge in “future work.” Steig explicitly urged use of it in THIS paper, not as future work.
It doesn’t get much more clear than that.
Re: MrPete (Feb 13 21:02),
You can’t go telling a reviewer or editor that you know of a better way, but this will do for now. Of course they will say, do it properly here.
But in fact it wasn’t that explicit.
Re: Nick Stokes (Feb 14 03:49), There is a very good reason why the iridge work should be put off.
But I agree that the recommendation is not that explicit. Its clear that Ryan and others thought it was explicit. If they had adopted a legalistic defense ( steig never explicitly said use iridge) as a reason how would that fly?
In the end it looks like steig was promoting a result he liked (nothing wrong with that) that led to the use of a method he didnt seem to fully endorse which he now criticizes. Clearly rejecting a method now that he endorsed with reservations then, would bear some explantion on erics part. whatever.
In any case we have a conflicted reviewer ( not problem in that) a misunderstanding (on best case reading for steig) compounded with a breach of anonymity (by steig) piled on by a shallow post at RC, following by a broken promise by Ryan and some further misunderstandings.
The human dimension of this thing is interesting to me on a cultural level, But I’d like to see folks get back to the maths.
another point to keep in mind – Broccoli had already blown off our request that Reviewer A be treated as a conflicted reviewer or that his review, at least, be sent to unconflicted reviewers for consideration before requiring us to make more major revisions. We had no reason not to believe that the editor wasn’t hand-in-glove with Steig (this had happened in other cases and is evidenced in Climategate correspondence). That the editor had required another “major revision” after the Second Version certainly suggested that he was hand-in-glove with Steig and would use any pretext to reject.
Re: Steve McIntyre (Feb 14 16:28),
Yes, If one was writing history about this thing, RO10 is a VERY significant paper, REGARDLESS of the science.
In the context of what we had been claiming about climategate ( climategatekeeping) and the whole battle of the blogs this is a major piece of history. I mean if one is writing a history of the climate science wars this episode has it all.
Imagine your a historian trying to make sense of this.. especially since the science ( the math) and the texts and the personalities are all so subject to “plausible” diametrically opposed readings. (not so much the math).. What able historian could actually understand the math?
Here is something provocative that Jerry ravetz said to me at dinner.
( ha chatam rules at dinner.. I think not)
We were talking about the suspect valorization of physics as an ideal of science
and he cocks his head and says. Climate science is more history than physics. that nugget has been pinging around my brain ever since.
AFAIK, they were NOT saying it is a better way to do the analysis needed for the present paper, just an interesting alternative method.
As has been repeatedly noted, all of the methods used in O10 got very similar results! The “alternatives” were simply confirmations. That’s one reason why it is so interesting that Steig was adamant that the editor require this switch! It was a lot of work with very little practical effect… a make-work requirement.
You are correct in one sense: it wasn’t that explicit that iridge is a “better way.”
However, Steig was quite explicit about “do it now”:
To insist on the change, and highlight what that change ought to be, and not delay iridge work for a future paper, is quite explicit. And not ameliorated by saying “perhaps kgnd should not be used at all.”
Ali, no. The first submission of the paper was based on TTLS. In the first review, Steig found fault. In the response to this review, O’Donnell ran a check by running iridge and reported the results showed slightly more warming than TTLS but still only about half of that reported by Steig. O’Donnell felt TTLS was still the best way to go and “future work” could look into iridge.
In the second submission, TTLS results were again reported. In Steig’s review, he wrote: “My recommendation is that the editor insist that results showing the ‘mostly likely’ West Antarctic trends be shown in place of Figure 3.” Notice the word “insist,” which is exactly what the editor did. In the decision letter, the editor called for “further major revision.”
In the third submission, after a major revision the iridge results were reported. In Steig’s review, he said iridge “makes sense” but then went on to talk about biases. By this time, the editor must have been pretty hacked off at Steig himself. Steig insisted the ‘mostly likely’ trends be shown and now he wants to talk about biases? The editor did not even show Steig the response to his review comments. What does that tell you?
A few minor word changes were made for the fourth submission and it was published. Come on. Be honest. Wouldn’t you be hacked off if a reviewer “insisted” you use a certain method and then publicly criticized you for it?
Ali – from Steig, second review (i.e. before the iridge changes were made to O10):
“My recommendation is that the editor insist that results showing the ‘mostly likely’ West Antarctic trends be shown in place of Figure 3. While the written text does acknowledge that the rate of warming in West Antarctica is probably greater than shown, it is the figures that provide the main visual ‘take home message’ that most readers will come away with. I am not suggesting here that kgnd = 5 will necessarily provide the best estimate, as I had thought was implied in the earlier version of the text.
Perhaps, as the authors suggest, kgnd should not be used at all, but the results from the ‘iridge’ infilling should be used instead. The authors state that this “yields similar patterns of change as shown in Fig. 3, with less intense cooling on Ross, comparable verification statistics and a statistically significant average West Antarctic trend of 0.11 +/- 0.08 C/decade.” If that is the case, why not show it? I recognize that these results are relatively new – since they evidently result from suggestions made in my previous review – but this is not a compelling reason to leave this ‘future work’.”
So please tell me how you can maintain your last position.
The behaviour of O’Donnell and co-authors in this matter is simply appalling. I guess Steig realises, too late, that trusting any of you with the fact that he was a referee was a huge mistake. The betrayal of confidence shows that you have no morals. The mountains you are making out of molehills only show that your motivation is to be gadflies, and not to carry out any real science. I, for one, am disgusted.
If this were a molehill, why comment?
Because O’Donnell et al are behaving shamefully. Wasn’t that obvious?
Ignore the man behind the curtain (of anonymity).
I guess that’s what you just have to do when dealing with Wizards when they are defending True Science from the unworthy infidels …
“Reviewer A” delayed the publication with 88 pages of criticism. In the end “Reviewer A” turns out to be the guy who published the paper that is practically rebutted by O’Donnell et.al. That’s not a molehill; that’s a reason to abandon the compromised process of peer review in climate science completely as it does not *add* scientific value but is -obviously!- intentionally used to *destroy* science.
Re: DirkH (Feb 13 20:18),
Reviewer A did not write 88 pages of criticism.
The number of words in the original O’Donnell submission and “Reviewer A”s first reply are as follows:
1 20100209 Submission.pdf (the original submission, sans references and figures): 6275 words
1A 20100305 Review A.pdf (Steig’s first review): 6987 words
(as word counted by the unix ‘wc’ command after exporting the PDFs as text). i.e. Steig’s first review was several hundred words longer than the article he reviewed. That is unusual.
The overall amount of review comment and reply to review comment generated in the publication of this paper is by any standard very high. It is also largely a result of the activities of “Reviewer A”, who seems to have been remarkably voluble.
Pedant,
In total after Steigs obstructionism the paper and the subsequent reviews and replies totalled to 88 pages.
There, happy now?
Mailman
The peer review process obviously quite fundamentally relies on peers reviewing the paper. “Delayed the publication”? Publish it on a blog if you don’t want it to be critically assessed. And yeah, don’t just make things up. There were not even remotely 88 pages of criticism.
Wow. You got some misplaced disgust in this matter. As always, step back to the wider view rather than staring at a single tree (ala Yamal) and see the forest.
The hockey stick of disgust has a wicked curve to the blade and is pointing back to the team.
RW, Do I have this right?
OK, let’s see – “A” is the victim of misconduct, but doesn’t know it. Person “B” says “Hey. don’t tell anyone, but I’m the perpetrator”. Now, the reviewer confidentiality is to prevent softball reviews. There wasn’t a softball review; the review was complete and the paper published.
So, “B” commits reviewer misconduct. “B” further violates the review rules by disclosing to “A” that he was the reviewer. So far two strikes against “B”. At this point, ethical considerations require “A” to disclose that “B” has engaged in misconduct. Maybe your ethics class didn’t make this clear, but if a friend of yours tells you don’t tell anybody, but they held up the 7-11 last Saturday night, the fact that you were the clerk on duty at the time does not present you with a dilemma – you tell what you know.
One wonders when Steig will stop the protests(?) This is the problem for some in the climatological arts. Facts have been malleable for so long, it is difficult to adjust to a world where shouting and bullying are orthogonal to the variables of interest.
“So please tell me how you can maintain your last position.”
Sure.
Steig: Anything but kgnd.
Authors: How ’bout iridge?
Steig: Since kgnd sucks, maybe that’s a good idea.
Authors: We agree.
Later:
WATBies: How dare Steig criticize the use of iridge!
Well, why not? Iridge merely presents the authors’ most likely results.
The results we presented were near identical no matter the method.
Ali Baba – this is the most egregious misuse of the review process I’ve even heard of. The fact that the other reviewers handled the paper in the same general way I’m used to while it received extra special treatment from the one says it all. The relevant comments weren’t directed at making the paper better, they were directed at changing the content for tangential, not central reasons. No reviewer in his right mind does this.
Steig: Since kgnd sucks, maybe that’s a good idea.
=========================
YUP! I can clearly see the insistence in your paraphrase
Hint, kgnd “sucked” because it gave results that weren’t “likely”, i.e., they didn’t show what Steig wanted to see.
Think. What does “likely” even mean here, when he has no independent way of checking the “real true” results? “Likely” here means not as in “closer to the truth”, but rather “closer to what I always thought as the truth”, which is utterly irrelevant, unimportant and unusable as an argument, except if you somehow believe in epistemological shortcuts like some religious fanatics do.
Eric couldn’t have used such a language, if he had been competent as a good empirical scientist. O Donnell et al were very friendly to accept such veiled endorsement of another method. Now Eric states that he never did such an endorsement, and he is right, for it was veiled, not explicit! This is the kind of stuff that makes me hate lawyers.
RW, do you believe the fact that Steig was a reviewer is at all relevant to Steig’s public blogosphere post-publication criticism of the O10 paper?
Why did Steig say the fact that he was a reviewer becoming publicly known could cause problems? Was there something to hide; what kind of problems could it cause?
I think the very idea that he was a reviewer of a Discussion published on a paper he wrote which he would have ample opportunity to formerly address in the Reply is shameful. No doubt both Steig and the editors of the journal were aware of how inappropriate this was and did not want it known.
Perhaps in climate studies this sort of thing is common. I’ve published numerous articles in the geologic sciences and sat on a couple of editorial boards and in my experience this scenario would have been completely unacceptable. As a reviewer your duty is to identify possible conflicts of interest and remove yourself from the process if those conflicts have any possibility of clouding your judgment. Of course this doesn’t always happen, hence the bad name that peer review currently has.
Shameful behavior from someone who is obviously a talented scientist and I believe the journal needs to review their policies to avoid this in the future.
As far as I’m concerned, and trying to be as impartial as possible (fat chance), the most important error of Steig’s procedure (the “take-home message” so to speak) was to wait until O(10) was published before criticizing the choice of procedure (TTLS vs. iRidge). This is the one aspect of the story that revolves around the identity of Reviewer A, which is why Ryan O’s faux-pas was definitely for the best. In an ideal world (and, actually, in many less-than-ideal worlds too), a reviewer would grab the opportunity to make negative comments about a methodological choice… yet, Steig preferred to, at best, be silent about it (I’m glossing over the many sentences that indicate he was suggesting it or arguing for it; they’re not important to the point I’m making), bide his time, and criticize the chosen method in (horror of horrors) a blog.
Is this supposed to be the indicated behavior for a reviewer? Gee, if it is, this is way further-from-an-ideal-world than I imagined. From where I stand, it looks like Steig was trying to use two hats at once — the “impartial reviewer” and the “blogland critic”. They are not really compatible. And that is the take-home message for me.
A good point. Those academics who are wringing their hands about the sacredness of peer review should ponder the complications that arise when an “anonymous” peer reviewer also wants to be involved in public controversy. Interesting issues.
And an issue for editors, as well.
You’ve captured what I see as a real problem with anonymous peer review in the internet age, especially as it relates to the broader public impact of “Climate Science” imperatives.
It seems unfair to allow a participant in a closed court session to perform as somewhat of a juror, at one’s own trial and then be permitted to maintain the outward profile of uninvolved outsider to the pleadings, free to respond in the *public* court without acknowledgment of private participation.
I think this is something the various societies and journals need to consider as I imagine the rules of engagement were established prior to the days of self-publishing/broadcasting capabilities. It would seem to me that if anonymous reviews are a necessary protective feature during the pre-publishing process, once completed, the names of those involved in the review should be published alongside the manuscript and the reviews/responses should be available, at least, upon request.
There may be additional benefits. One of them being, closer attention by reviewers to ensure their meaning is sufficiently clear and understood by editors, authors and subsequent readers.
I don’t think too much has to be read into Steig’s three reviews or his motives. The reviews are indeed rather confused and discordant, which is perfectly consistent with someone who is mainly casting about for some way (any way!) to have O(10) increase the extent of warming in West Antarctica. That is, anything change in process or interpretation that would preserve the ‘novel finding’ in S(09) of substantial warming outside of the peninsula region, and especially substantial warming in West Antarctica. Viewed this way, Seig’s “evolution” in thinking about TTLS versus iridge… it just boils down to “you have to use whatever method shows the greatest warming in West Antarctica”.
.
I do not know if Steig is even aware that his reviews give this impression; indeed, I suspect that he may not be consciously aware of what he was doing. But none of it matters, beyond noting that his input as a reviewer was mainly self serving and a waste of everyone’s time.
This really makes one wonder how many “Trojan Horse” peer reviews are out there where the anonymity of the reviewer is never revealed after the post-publishing attack.
It’s more likely when the hit-job reviewer doesn’t overplay his hand.
Editors don’t like reviewers who drag things out unnecessarily – eventually they try not to use them. They also know when a reviewer is acting differently than they normally do (if they’ve been used before), and when they’re acting different from the other reviewers. The editor definitely knew something was up. After these postings, many in the field also know the editor had to know something was up, and the editor is sure to know they know.
Comments?
ARRRGGGGHHH! another “Team” proposes and Steve disposes
I think whether or not Steig suggested the use of iridge is splitting hairs, but there is no doubt that he insisted on its use once it was raised.
I think that Steig was simply testing the limits of Prof Phil Jones’ assertion in Nature about peer review:
“The whole point about trying to pervert the peer-review process is that it is impossible to do it. There are so many journals and if people are persistent enough, they can get their papers published.”
http://www.nature.com/news/2010/151110/full/468362a.html
This controversy now has generated more heat than the Peninsula suffers. I am reminded of my daughter’s efforts to recall the name of the dessert, Baked Alaska, coming out as “Flaming Antarctica” – a perhaps appropriate coinage for this kerfuffle.
Serious demonstration as demanded by the situation: Dr. Steig’s argument thins…
Steig (or any of his co-authors) should not have been an ANONYMOUS reviewer of O10. It is really that simple.
My guess is that the authors of O10 would have quickly identified the fundamental flaw in Reviewer A’s approach, i.e., the effort to highlight results that support the results of S09 even if the statistical approach in S09 is shown to be seriously flawed, and, more importantly, quickly pushed for a qualified but more disinterested fourth reviewer.
Steig could have reviewed O10 but only if he was identified as the first author of S09 to O’Donnell et al. In my opinion, once Steig saw O10 he would have been better off declining to be a reviewer and asked to be allowed to submit a rebuttal. That he did not – and this is obviously my opinion – reflects exceedingly questionable behavior and judgement on his part. It is exceedingly difficult to read Steig’s reviews as trying to improve O10 as opposed to defending S09. That some of Steig’s suggestions improved O10 serve only to mask what he was obviously actually trying to do. The majority of Steig’s review comments would have been OK if made in a rebuttal (though they would then have been seen as failing to address the methodological points raised by O10.)
JoC should revisit its reviewer selection and anonymity of reviewer policy.
I take your point, Bernie. But I think (from my comment below) that Eric’s actions can be more easily explained by sloppy writing (in his review) and bad assumptions than by malice.
You speculate as to why he didn’t write a rebuttal instead. Consider this: he was just given a paper that – when published – will be a very public smack-down of a very public (cover of Nature) article. And let’s say he recognized the main result as being correct, and he now knows that his method was flawed. What if his extensive review, rather than being a malicious undertaking, was instead a conscientious effort to improve the paper and make sure it was solid? Even though his ego would take a hit?
I think the fact that he DIDN’T write a rebuttal and try to “defend to the end” bad results potentially speaks well of him. I don’t think, for example, that Michael Mann would have handled this situation so well.
Ted:
You may be correct (you certainly have written a coherent and persuasive case below)and IF Steig had kept his cool then I would have to agree with you. But Steig’s earlier reactions to technical comments by O’Donnell et al when S09 originally came out plus his recent outbursts do little to support your more benign interpretation of his role as a reviewer.
All that said, I have no problem with his being a reviewer, simply not an ANONYMOUS reviewer.
I am not sure how the Team will weave this post, but I expect them to do so to nonetheless.
I am pretty sure that Stieg will survive the current controversy. He is a tenured professor at UW and would have to do something a lot more egregious than tampering with peer review to get the boot there.
He might do well to go back and concentrate on his specialty of isotope geology though, and leave statistics to those who know something about it.
I find it interesting that one reviewer apparently didn’t understand the statistics and was relying on Steig or Mann to refute them. A Macchiavellian would surmise another person in the clique was hoping MM or ES would be another reviewer and can the paper so s/he wouldn’t have to do the dirty work him/herself but I would rather ask why the editor didn’t ask for a fourth reviewer then since apparently this reviewer was taking a pass?
Steve:
I’ve read your account very carefully (thanks for providing it). Given Ryan’s comments that Eric seemed reasonable in person (before the current kerfuffle) I tried to read his comments with that in mind. I think I have identified a few places where intent and meaning have been misconstrued. Based on this, I think you are giving Eric both too much and too little credit.
I think you are giving him too much credit in assuming he did his own statistical tests to determine that k_gnd = 7 is “suspect” and that this is his independent judgement. It isn’t. He seems, rather, to be relying on the original paper, where you state that “We cautiously conclude that the trend in West Antarctica is likely significant and may be closer to 0.10 C decade than 0.05 C decade” and that k_gnd = 7 “may be artificially low.”
So his whole point seems to be, since this is “artificially low” (which he labeled “suspect” as shorthand for the paper’s conclusions) it shouldn’t be the main “take home” message of fig. 3. He is insisting that the take home message be the likely conclusions, i.e. the 0.10 trend rather than the 0.05 trend.
Every other objection seems to flow from this misunderstanding … he’s not suggesting the use of iridge because he thinks is best from his independent judgement (as he has said, he isn’t a statistician). He is saying that iridge “makes sense” and “it really does give the best results” … but ONLY because it is more consistent with all the other results you reported, the 0.10 C trend.
Later, of course, he criticized iridge because he thought his objections, wrt Mann, et al., 2008 (2007) were ignored. And Ryan took objection to that because he thought Eric had seen the reply to the v.3 review AND because Eric seemed to recommend iridge. I think it’s pretty clear (now) that Eric didn’t mean iridge is inherently robust … he just thought the results as presented showed iridge to be better (more consistent) based on what the paper said. Later, when the paper surprisingly switched to iridge, he decided not to buck the editor or “the knuckle-headed reviewers” whom HE PRESUMED were responsible for pushing iridge … because in his head, he did not. Rather than go against the editor and some presumed reviewer, he simply added a few objections that he, as a non-statistician, didn’t really understand but thought needed to be addressed, based on Mann, et al. He did this after, surprisingly (to him) iridge became the main methodology of the paper.
So, my point is, Eric actually believes it when he says he “did not endorse” iridge, and was surprised that it became the central part of the paper. He believes that his only point was to present the strongest result – the 0.10 C trend, rather than the “suspect” (based on the paper’s judgement) 0.05 C trend.
I base my interpretation of his meaning on three things: 1) Ryan said Eric seemed like a nice and reasonable guy; 2) Eric has said that he’s not really a statistician; and 3) Eric keeps posting his own “smoking gun” quotes to prove his point. Clearly, he does not believe they are smoking gun quotes.
It seems that you didn’t need to completely re-write the paper … he only meant to insist that the most likely results be the main message in fig. 3. Later, when iridge was brought up by you guys, he thought “Well, it’s more consistent with the main conclusions … you might as well do it now rather than save it for later. I’m not a statistician, but it can’t be that hard, right?”
I think you, and the other co-authors, are also being completely honest that you believe he *was* insisting on iridge. It is very easy to read his comments as promoting the METHOD rather than the (more consistent) RESULT. But I think you are wrong in ascribing malice to “that which is adequately explained by incompetence” … or, if not incompetence, then sloppy writing and a not very on-the-ball editor.
By the way, it’s a great paper. Congratulations.
Ted:
This is an interesting and reasonable take the potential source of the immediate and subsequent misunderstandings of all parties. It certainly accounts for the vehemence of Steig’s responses. However, it actually reinforces a more substantive concern namely that Steig was trying to preserve the results of S09 as opposed to reviewing the O10 paper on its own terms – namely that S09 had used a flawed statistical methodology. Apparently this is what the editor concluded.
I still say the misunderstanding around which method to highlight would have been flushed out earlier if it had been clear that Steig was Reviewer A from the beginning of the review process.
Yes, it probably would have been flushed out earlier … especially since Ryan and Eric have had friendly exchanges during this time. (Just to be totally clear: my main point is, I think Eric was pushing for highlighting different *results* – the 0.10 C trend over the 0.05 C trend – and that this was interpreted as pushing for a different methodology, due to miscommunication.)
And thank you for the kind words above. (I’ll reply here rather than continue two separate chains.) I haven’t read the posts on the S09 technical comments, so you have more information than I on which to base judgement. However, after I posted here, I read some of Eric’s comments on RC. He mentioned spending “many, many, many hours” on the review, to be “as thorough and fair as possible.” And he also said, “I was reminded at least once by the editor that if I thought it should be rejected, I should say so. I did not say that, ever” and “What I said in my first comment to the editor was that although I thought the first draft read like a series of poorly thought out blog posts […] the authors should ‘not be discouraged from submitting a revised version.'”
Anyway, I don’t know the guy, and don’t care much at all for RC. But based on the evidence I’ve seen, I couldn’t reject the null hypothesis: that Eric was trying to be conscientious. I think this whole episode is an unfortunate misunderstanding, conflated with a few honest technical disagreements.
Anyway, the science has been improved. Hopefully, the dialog will get better, too.
Ted, an interesting perspective.
Other points to consider were the Climategate emails in which Team authors admitted “going to town” to block publication of critical articles – David Rutledge of Caltech, as an editor of over 10,000 pages of IEEE journal articles, condemned the conduct in the strongest terms. I personally had been the subject of such reviews – a reviewer of our submission on Santer et al 2008 promptly and smugly reported to Phil Jones that that “Fraudit’s” comment had been rejected at IJC. While the Team was corresponding with one another about our comment being rejected, Santer untruthfully proclaimed that we had not attempted to make any use of the Santer et al data set.
Added to that were wee taunts from RC about publishing in the peer reviewed literature. There was a vivid incident when Steig posted at RC here on overfitting. Steig’s use of North et al 1982’s analysis of eigenvalue separability as a criterion for the number of PCs to retain (also used in Steig et al 2009), particularly in the context of Chladni patterns (which Steig was unaware of), clearly demonstrated Steig’s extremely weak grasp of the statistical issues.
Ryan made a very sensible comment at RC, making points that were later included in the first submission on Chladni patterns and the inappropriate use in Steig et al 2009 of the North et al 1982 article. Steig refused to engage on this and challenged Ryan and others to show up in the peer reviewed literature. For example, Steig to Jeff Id:
This was amplified by RC commenters to more of a taunt:
From our perspective, Steig, as a reviewer, appeared to be acting consistently with the Climategate pattern – perhaps phrased a little more smoothly, but, in practice, was attempting to raise as many roadblocks as possible in the hope of blocking the article or obstructing it as long as possible. The context is not exactly the same as ordinary academic disputes – it was rawer going in.
The disposition of the Chladni patterns – the topic of the June 2009 taunts/challenge – is an interesting story as well. As Reviewer A, Steig called for the removal of this section of the first submission. No other reviewer did so. As a matter of expediency, Ryan complied with Steig’s call. The net result – Steig had challenged Ryan to publish the comments on Chladni patterns in academic literature and then worked behind the scenes to prevent it appearing in the peer reviewed literature.
Shortly after the publication of O’Donnell et al 2010, Steig commented:
I don’t know whether Steig ever said this in so many words, but the taunt was there. In my opinion, once realclimate had issued its taunts, Steig would have been much less subject to criticism if he had just let the peer review process run its course with unconflicted reviewers, without attempting to bend the process to achieve results that more closely resembled his own.
Steve, Nic & Ryan:
I know it is asking a counterfactual, but how would you have responded differently to Reviewer A’s first and second review IF you had known that Reviewer A was definitely Steig? Is there anything you discussed saying or doing that you didn’t do early on because it was unclear who Reviewer A was?
No. From day 1, we guessed that he was the reviewer. We even sent correspondence to Broccoli on this point with our very first review response.
Ryan:
Thanks again for a straightforward response.
We raised the Reviewer A issue with editor Broccoli after the First Review quite pointedly and were brushed off. I just took a look at our internal correspondence about how to respond – this is in April2010 – and there were differences of opinion about how direct we should be with Broccoli.
Steve:
I guess then that my question remains open. Are you saying that you were pretty convinced Reviewer A was Steig (or a co-author) and you chose to respond to the review as if it was Steig? Or would you as a group have responded differently if you had known definitely that Reviewer A was Steig? Certainly the editor seems to have joined Steig in pushing for increased emphasis and visibility of the iridge results – thereby creating more of a distraction from the fundamental points of the article. I am assuming that you could have made it more obvious to Broccoli what Steig was doing and why if the identity of Steig was on the table. It is hard to be definitve when you have to preface your statements with “If Reviewer A is a member of the team…”
If this is a distraction please feel free to ignore.
My opinion at the time was that it was a team review. Steig says it was not. As far as Broccoli another valid interpretation may be that Steig was his expert most familiar with the S09 paper so taking our observations about Reviewer A into consideration, he went with another round of major reviews. The second time Broccoli agreed for a major review was pretty painful though and he could have disagreed, but we weren’t required to make any changes for the third despite the fact that Steig still recommended against publication.
Did Steig recommend against publication?
I thought that he denied that in his comments, though I may have miss-read.
If nobody recommended against publication, why would a fourth reviewer be necessary?
Steve, one of the things I admire about you – and why I always read this blog – is that you have an incredible memory for details. Every time you draw a conclusion, you also provide quotations and a clear record, one that specifically illustrates what led you to that conclusion. It’s a wonderfully useful practice – one could sometimes disagree with your conclusions, but they could never claim that they are “baseless” accusations.
For example, I might not characterize Eric’s emphasis on publishing as “taunting,” but I can clearly see your basis for that characterization. This makes it easier to argue the point rather than the person, which helps avoid a common consequence of blogs (or any communication lacking non-verbal cues).
I completely agree that “The context is not exactly the same as ordinary academic disputes – it was rawer going in.” This called for a more careful handling of the paper and the review process than what occurred.
In regards to “going to town” on reviews, I can only offer a devil’s argument. What if they have a very low opinion of any science coming from skeptics? I believe the “going to town” quote was from Phil Jones? It doesn’t speak well of him, but it doesn’t mean that the criticisms he offered were invalid (in his mind) … only that the glee in finding weak arguments is unprofessional, even in a private email.
In that vein, I think it likely that Eric was: 1) wrong about the Chladni patterns, and 2) unaware that he was wrong. Part of the reason for (2) is almost certainly an unrecognized bias or tendency to believe that much of the science from CA is bad. Nevertheless, since (2) exists – which he can certainly be criticized for – it was probably natural for him to recommend removing that section.
I also think there is a disconnect between how Eric views the job of a reviewer and how the editor views it. I think the editor gave more deference to Eric than he intended to claim. His repeated statements, along the lines of “it’s up to the editor to do X” and “the authors didn’t have to take my advice” indicates that he was unprepared for how stringently his words were taken.
As an aside, my advisor has been a tremendous help for my own academic papers, in responding to reviews. There have been many instances where I perceived a huge amount of work needed to respond to a comment, and instead my advisor suggested a short phrase or one sentence, and said that is enough. I am still getting used to the idea that most comments can be dealt with quite easily. (And sometimes one can simply state disagreement in a “reply to comments” and THAT is enough to satisfy.)
“I also think there is a disconnect between how Eric views the job of a reviewer and how the editor views it. I think the editor gave more deference to Eric than he intended to claim. His repeated statements, along the lines of “it’s up to the editor to do X” and “the authors didn’t have to take my advice” indicates that he was unprepared for how stringently his words were taken.”
I disagree that any experienced scientist who has reviewed papers before and been reviewed themselves, as Steig is, can claim such a hands-off/I-did-nothing position given the comments and recommendations made by Steig. Reviewers are the gatekeepers to publication and the team knows this, otherwise there would not be e-mails from climategate indicating such behavior. For Steig to claim later that he didn’t realize his reviews would have such an impact on revision and publication can only be completely disingenuous to say the least.
If he really didn’t believe he could influence the content of the paper before publication, or keep it from getting published, why did he bother?
Why not just wait till it is out then attack it publicly? He must have believed his review would have an influence on either the final paper, or keep it from getting published. To me it looks like he settled on pushing the conclusion of the paper as close as he could to the conclusions of S09. Then a claim of independant confirmation could be made, which is exactly what was done at Realclimate:
http://www.realclimate.org/index.php/archives/2011/02/west-antarctica-still-warming-2/
But does not this taunt carry with it the reality that the Team has been able to exert considerable influence in the past on the peer review-editorial process;i.e., they feel they speak from a position of power, whereas you and your colleagues are considered as outsiders in the climate world by at least some of the power brokers. The way you have been treated by certain editors in the past, such as by Nature, speaks volumes about this. The playing field is not and has not been level. The peer review process has been hijacked by an agenda.
Steve- don’t go a bridge too far. You can’t extrapolate from one experience to the entire world. Nor have I (to date) commented very much about the peer review process.
Re: Steve McIntyre (Feb 14 07:54),
I think any assesment of the propriety of picking Steig as a reviewer must take this into account ( As I’ve said elsewhere )
One can make a defense for ‘conflicted” reviewers. However, The existence of the custom
whereby conflicted reviewers sign their review is indicative of it’s precarious nature. The editor of the paper put Steig into this position and he accepted it. Did he know about Steigs public challenge
What makes this case different from any other “conflicted” reviewer case I’ve seen is this: Steig had made a public challenge to meet the author on the battlefield of peer reviewed literature. And in the case of Ryan this is an author who has no track record. That kind of challenge has no analogue that I’ve ever seen. Let’s see if I can make one
Imagine, for example, that you are a grad student with zero publications.
Imagine you make a pointed criticism or two of Judith Curry at a public forum,
say an AGU Keynote. Imagine that Judy responds to you by saying, “go ahead
try to get that published kid”
If you were that kid would you feel it was appropriate to have Judith review the paper? Would you have any reason to wonder if she was doing more than defending the science if as reviewer she gave you a hard time? Heck, even taking the reviewer assignment would be a sign to you that she intended to defend two things: her published paper and her public challenge/reputation.
The Chladni patterns: So, “as a matter of expediency” the Chladni patterns stuff was removed. Wish you hadn’t done that–I thought the Chladni patterns argument was a killer right there; removing it, to me, seems like self-disemboweling your argument to some extent. AND, I thought the Chladni pattern argument was beautiful and “elegant”. I mourn its loss.
Thanks for posting these extraordinary comments by Steig.
While this: ‘I thought the first draft read like a series of poorly thought out blog posts […] the authors should ‘not be discouraged from submitting a revised version.’ shows an unbecoming arrogance, it also goes towards a point I find very odd: the other three reviewers look at the paper as written by other scientists, they review on a level playing field, if you wish.
Steig however knows full well that RyanO et all were most certainly not climate scientists, and goes from condescension to trying his level best to get more and more re-writes.
Also remarkable is this: ‘I was reminded at least once by the editor that if I thought it should be rejected, I should say so.’
Why did he not take up this suggestion or suggestions by the editor?
Odd …
Ted,
Your explanation seems plausible. If, however, Eric did not believe he had the statistical expertise necessary to truly evaluate iRidge (and wrote his review comments in that context), then I find it difficult to find justification for him writing (especially without having seen how we addressed his request for a “few sentences” of explanation):
“It is not surprising that O’Donnell et al (2010), by using iridge, do indeed appear to have dramatically underestimated long-term trends—the Byrd comparison leaves no other possible conclusion.” (emphasis added)
It is not possible for him not to have enough statistical expertise to evaluate iRidge, and, at the same time, state that the lower trends shown in our reconstructions were certainly due to the use of iRidge.
Ryan:
This is called, “trying to have it both ways.” On the one hand, Steig claims he doesn’t have the expertise to RECOMMEND iridge, but on the other, he does claim to have the expertise to EVALUATE ITS EFFICACY.
Let’s face it, Steig’s objective functions are pretty clear (in order of priority): 1) keep the maximum possible Antarctic temperature trend in play (particularly West Antarctica) to maintain the illusion of the iconic “Nature” cover, 2) keep alive any possible controversy regarding methodology, for purely obfuscatory reasons (and because it’s an area of team weakness), 3) get the science right.
Message = method.
Ted:
Thank you for your interesting comments. I am sure that, as you say, there have been some misunderstandings on both sides.
As an author of OLMC 2010, I can correct one or two apparent misunderstandings on your part.
You say:
“It seems that you didn’t need to completely re-write the paper … he only meant to insist that the most likely results be the main message in fig. 3. Later, when iridge was brought up by you guys, he thought “Well, it’s more consistent with the main conclusions … you might as well do it now rather than save it for later.”
We didn’t bring up iridge in connection with using it for our main results (displayed in Fig.3) – Eric Steig did, in his (Reviewer A’s) 2nd review. We had at that stage merely quoted iridge trends for West Antarctica in a textual discussion of uncertainties in that region, and stated a ‘most likely’ trend there that was in line with the iridge trend; our main results did not use iridge. The switch to iridge for our main results was the consequence of Reviewer A’s 2nd review comments, which amounted to a suggestion of using of iridge for Fig.3 (and hence for our main results) even if it didn’t amount to a recommendation of iridge for that purpose.
As per Steve’s quote from me in the above post:
“Further, we were being forced to undertake major revisions for the second time, and in the circumstances it would have been foolhardy to go for a method that Reviewer A could readily attack. In my view, that ruled out sticking with truncation based methods (whether TTLS or TSVD, correlation or covariance), since Reviewer A was still attacking the choice of truncation parameter. Further, it would have been awkward to justify switching to a truncation parameter choice that did not provide the best verification statistics.”
Therefore, it was in fact necessary to change the method used for our main results, and rewrite the paper.
Your comments about what Eric Steig believes may be correct, but, if so, IMO they seem to suggest that he was using an inappropriate approach to reviewing a paper that was about correcting the method used in S09. Surely a reviewer of such a paper should have been primarily concerned as to whether results using its method (which were substantially different from the S09 results) more accurately reflected the available data (that used in S09), not whether they agreed with the results the reviewer himself had published or his beliefs as to what the actual trends were. However, I don’t want to make too much of this; like Jeff Id, I’m a bit worried about Eric Steig’s state of mind at present.
Of course, as Eric Steig isn’t a mathematician, perhaps he wasn’t the best author of S09 to be appointed as a reviewer. Whichever of his co-authors Schneider, Rutherford or Mann (who all had responsibility for the actual reconstructions in S09 attributed to them) carried out most of the work on their reconstructions (Rutherford?) would have been better able to review the methodological issues involved. We could then, more appropriately, be tearing them, not poor Eric, to shreds for not understanding the mathematical deficiencies of the S09 method and how they were overcome by our improved method.
NicL (and Ryan above) thanks for the reply. I’m sorry I have only a few minutes to post just now, so I’ll only write a quick word.
I want to make sure it is clear that (in the most generous interpretation I can come up with that is still reasonable) I don’t believe Eric “insisted” on the main message being a 0.10 C trend over a 0.05 C trend because it “agreed with [were closer to] the results [he] himself had published or his beliefs as to what the actual trends were.” I think, rather, he based this upon comments in the first draft itself; it was stated by the authors that 0.10 C was more likely that 0.05 C, and that k_gnd = 7 was the only value that yields an insignificant trend.
In other words, simply reading the paper (and not fully grasping the statistical details perhaps) caused him to say that the k_gnd = 7 result shouldn’t be so prominent. In effect, he was saying, “You guys think the trend is probably higher? Then you need to provide a basis for this speculation. And you need to explain why the ‘suspect’ value of 7 is suspect. And since it is suspect, this value shouldn’t be used as the take-home message.”
I don’t preclude the possibility that his preconceived notions biased him towards 0.10 C over 0.05 C. But it also isn’t necessary, given the statements from the paper, to infer this as an explanation for his “insistence.”
Cheers.
Another quick note: I hope I’m not coming across as rude above. I’m only trying to understand the underlying cause of Eric’s “outbursts,” and his insistence that he wasn’t being duplicitous. -t
Steve – not at all. Your comments are lucid and interesting. I think that the exercise of interpreting Steig’s conduct is the most favorable light is something that is much preferable to the obvious piling on.
One of the other problems with the Journal of Climate review is that none of the reviewers was a qualified statistician or remotely knowledgeable about the issues. Steig himself observed that he wasn’t a statistician, He had no understanding of the difference between an eignevalue being separable and being significant or other issues of that type. Nor apparently did any of the reviewers of the initial article in Nature. None of the JClim reviewers showed any grasp of what we were doing as a statistical point. Reviewer B looked to Mann and Steig – of all people – as statistical authorities.
John N-G in correspondence observed that the pool of reviewers available to JCLim in atmospherics was small as justification for relying on Steig’s statistical review comments. The issues were purely statistical and JClim should have had reviewers who understood statistics – as Wegman had observed several years ago. Had they had qualified reviewers, much of our frustration would have been avoided.
Why do you think the Climate Science community continues to fail to engage with the statistics community? They could surely find some sympathetic (and qualified) practitioners that could save them some pain and embarassment.
it’s may not be all that easy to involve academic statisticians as one might think – academic statisticians tend to be interested in Borel measure and that sort of thing. The stuff here is very low-brow in academic terms.
Except that most stats departments are starved for funding. Plus, there’s plenty of competent grad students and post grads who would love to get their name on a high profile climate paper.
The alternative, a cohort of climate scientists who are competent at statistics, is at least a generation away.
My guess is that the Team is unhappy with the degrees of freedom available when using a statistician.
Borel field is introduced at the first lecture, just to scare the incompetent students away. This method is used almost everywhere.
It’s complicated beyond statistics. Some of the comments on the interaction of CO2 with radiation, both on blogs and in climate papers are simply “very low brow in academic terms” as well. In case there is doubt, those who are top of field in such topics, who post here, are rendering a valuable public education service. We hope that they do not underestimate their voluntary importance or the thanks that many must be giving them, spoken or not.
It is hopeful that education through reading will help many and that the peer review process might be gently modified to overcome some examples illustrated here on CA.
snip – editorializing on other topics. sorry.
RC has a new post up regarding going ‘from blog to science’…emphasizing the use of peer-review.
All questions about whether it’s possible that an author who challenges another to publish rebuttals in the peer-review could actually sit on the review board of that same paper– and recommend it be rejected…All those questions aren’t making it past moderation yet.
… and all in all, peer review remains a good thing and is for the most part positive.
That being said, I would say the same thing to Eric, that he said to O’Donnell et al. If you want to talk the talk over at RC, lets see you walk the walk with a rebuttal paper, and not just one reviewed by a bunch of powder puff friends. I am sure we could possibly even tempt Jeff out of retirement with the possibility of reviewing a Steig rebuttal. Somehow I suspect , like most consensus policies, that this form of “peer review” is only meant for others. Actually having to justify all those unsupported assertions is a lot of work and when you are actually forced to explain the math and your reasoning, there is not a lot of room for weasel like dodging.
Eric is currently engaging in what I term “Seagull science”. Fly in at RC, make a lot of noise, poop all over everything and fly away. I would hope for some actual science in the form of a formal rebuttal, but I am not holding my breath.
Gavin has responded…And I must say the delay seems to have nothing to do with censorship, and that I appreciate his comments.
In many respects, the ‘main characters’ at CA and RC take time to participate for free in this discussion in the arena, and endure various abuses, precisely because of the passion they have behind the conclusions at which they’ve arrived. In the face of the persistence, it would be hard to level certain charges (disingenuousness, nonseriousness, etc.)
Anyway, here’s what he said to my prompt (above):
“[Response: Of course. And the editor is free to ignore anything I say – especially if it conflicts with the other reviewers. In dealing with comments on a few of my papers, it has been clear (to me at least) that the comment had nothing very much to add, or was greatly deficient in some way – in those cases, I’ve said so plainly. Some of those comments were published, others were not – but in every case it is the editor’s decision, not the reviewer’s. Editors are not stupid, and they know they need to balance multiple interests at once. On the whole they do it well, but there are clearly some occasions when they lose the ball (the Soon/Baliunas fiasco, a couple of comments James Annan was involved with, etc). – gavin]”
Ted,
Your comments are far from rude, and they are much appreciated.
There is one difficulty I have with your explanation.
Eric spent several sentences in his second review specifically on iRidge. This is fine; he was making an assumption that we felt the iRidge results were better than the direct RLS, and he was correct. He certainly left the impression that using iRidge instead of the truncation methods would be his preference.
Then, in his third comment, he agreed that he felt the iRidge results seemed to be the most accurate, but had a lingering concern that he asked us to spend a “few sentences” justifying. This, too, is fine . . . he may have had some time to think about iRidge while he was awaiting our rewrite. I think it would have been better had he noted this in his second review – but it is a complex subject with relatively arcane procedures being used, and it is perfectly reasonable to assume that he simply thought of an additional concern in the interim. This is especially true since he has freely admitted that he is not thoroughly familiar with the iRidge algorithm.
Where my problem starts is not there. Where it starts is when his RC post states that we got the Byrd / West Antarctic trend wrong (which he has every right to claim, as he presented evidence supporting his opinion) and that the reason we got it wrong was unequivocably the use of a procedure he recommended (where “recommended” is taken based on a plain reading of the review). If he does not have enough experience with the algorithm to make a comprehensive judgment during review (as he has stated recently), then it would seem inappropriate to make public statements asserting that use of this very same algorithm is unequivocably the source of a problem.
Re: Ryan O’Donnell (Feb 14 12:57),
“Where my problem starts is not there. Where it starts is when his RC post states that we got the Byrd / West Antarctic trend wrong (which he has every right to claim, as he presented evidence supporting his opinion) and that the reason we got it wrong was unequivocably the use of a procedure he recommended (where “recommended” is taken based on a plain reading of the review). If he does not have enough experience with the algorithm to make a comprehensive judgment during review (as he has stated recently), then it would seem inappropriate to make public statements asserting that use of this very same algorithm is unequivocably the source of a problem.”
Thats a good way of putting it.
I’d lead with that
Steve Mc – exact quotes and links would help as well.
Hi, Ryan. Sorry for the delay – I had to teach a class.
In terms of trying to “assume the best” wrt his actions above, I can only think of two hypotheses. I know you put the quotation in a different thread already – I’ll add it again here for easy reference. Eric said, “It is not surprising that O’Donnell et al (2010), by using iridge, do indeed appear to have dramatically underestimated long-term trends—the Byrd comparison leaves no other possible conclusion.”
H1: Eric felt to be a bit out of his depth on the deep statistical issues (based on his own words), so in terms of being a reviewer (authority figure) he didn’t feel like he could ignore the perceived wishes of the editor, the other reviewers, and all the authors (since you did say something like “we agree these results are improved with iridge). So he hesitantly went along to get along. When he was no longer an authority figure, just an academic offering an opinion on iridge, he could write a post stating his objections, and why. Some academics see this as the preferred method for advancing the science – you make your case as strongly as you can, even if you aren’t completely sure, and then see if someone refutes it. Regardless of whether or not this method is wise, it wouldn’t be appropriate to do this as a reviewer. (disclaimer: I haven’t read his post in full. But I have seen this sort of thing before in other fields.)
This hypothesis – if true – was confounded by: to you guys (the authors) he was the only one to “recommend” iridge, even if he didn’t believe he had. Plus the fact that he hadn’t read the v.3 review reply, so he still didn’t know what was wrong with Mann, et al.
H2: He’s not saying iridge is less robust in and of itself. He is saying he’s “not surprised” it didn’t work as well (based on faulty information from Mann, et al.); AND he’s saying the Byrd comparison confirms it. So, his opinion on iridge is conditional on the “proof” of the Byrd comparison. (which I believe another post by you guys has already addressed in full).
These two hypotheses aren’t mutually exclusive, of course. Anyway, that’s my attempt to explain it by assuming no intended malice.
I really hope this situation doesn’t permanently poison the well. I think the most that Eric is guilty of is sloppy writing, faulty assumptions (in how his words were interpreted), and probably a little unconscious bias. I think there are others on “the team” who I wouldn’t trust at all, scientifically. But based on all the things Eric *could* have done if he really were malicious, I would probably (80% confidence) recommend him as a reviewer, for any non-statistical climate paper that I may write. (Ryan – a lot of that confidence is based on your characterization of Eric based on your previous interactions with him.)
“I would probably (80% confidence) recommend him as a reviewer, for any non-statistical climate paper that I may write.”
But isn’t it ironic that the lead author on a statistical paper is not an expert in statistics, placing him in a perceived position of being such an expert and thus being called upon to be expert reviewer on the statistical rebuttal? At least during the review he should have consulted the expertise behind the statistical methodology in his own paper. But he claims he did the review by himself.
Hi, Dave. You know, I’ve been thinking more about some of the comments and discussion back and forth, and I think your comment speaks to a good take-away from this whole situation.
It seems, to some degree, that this was primarily a statistics paper that was taken to be primarily a climate paper. That is, Eric Steig, et al., don’t really care about the statistics. Well, they care, but the more important thing is to get the outcome correct. That’s the ultimate goal: what does Antarctica look like.
Conversely, Ryan, NicL, Steve, and Jeff don’t care about the temperature in Antarctica. Well, they care that the outcome is correct, but the important thing was fixing the methodology. The outcome is what it is, as long as the method is right. And correcting the method will help in lots of other work, regardless of where it is.
So Eric took their comment – the trend is likely to be slightly higher than this method produces – and basically said, well, put the best answer forward, if that’s the best answer. The statisticians – who were focused on fixing a method – fixed the method, came up with a better answer, and then – almost as a throw-away comment – basically said “preliminary tests (iridge, etc.) show it’s likely a bit low, but the important thing is the statistics are now better.”
All in all, I think the revisions – even though they were a major pain – produced an even better paper (based on authors’ comments). I’m just saying, part of the mis-communication, it seems, derived from a subtle difference in focus and intent … which will be helpful to know in future statistician / climate scientist communications.
Ted, you say:
“a subtle difference in focus and intent …”
Well, that is indeed a generous view. O’Donnell et al were quite explicit with their intent, ie. to fix the method and let the chips fall where they may.
Reviewer A cannot have missed what was the main thrust of the paper. And his intent with it is not relevant as it wasn’t his paper. If indeed he had an intent (other than just reviewing its content for errors and relevance) he is already treading on thin ice, he basically says ‘I wish you had written a different paper instead… about some other things ..’
I would assume that the written reviewcomments (all three of them) fairly well describe the content of his intent, what he wanted to accomplish … And as to motive, we can only speculate but not disregard what now is available.
Given that he says he doesn’t understand the subtler nuances of statistics, his pressing of these points are actually a little peculiar, methinks ..
Very insightful comment Ted. However, with this in mind I think it could be argued that O10 would have been a better paper had it stuck with TTLS (with an subsequent iridge version as a ‘part 2’) precisely for the reasons you suggest. This of course leads back to the question of whether a TTLS version of O10 was ‘painted into a corner’ (and how that came about) from the standpoint of publication.
”
All in all, I think the revisions – even though they were a major pain – produced an even better paper
”
No, they produced a different paper. There’s also a bog difference between revision and rewroite
Ted C.: “Some academics see this as the preferred method for advancing science–you make your case as strongly as you can, even if you are not completely sure, and then see if someone refutes it.” Well, I guess we must indeed be living in the post-Feynman era. Feynman would say, “Sure, make your case as strongly as you can, and then make, in the same breath, the strongest case against it.” Are we in the pursuit of TRUTH here, or are we in the pursuit of something with our own glorious name attached to it?
Ted,
You said:”I don’t preclude the possibility that his preconceived notions biased him towards 0.10 C over 0.05 C. But it also isn’t necessary, given the statements from the paper, to infer this as an explanation for his “insistence.”
Thanks for your comment, which is well made. I am sorry not to have seen it until now. I think you have a fair point: it is indeed difficult, if not impossible, to apportion the cause of his insistence between the stated reasons and (perhaps unconscious) bias arising from preconceived notions. In fact, in my earlier comment I didn’t say that preconceived notions biased Steig towards 0.10 C over 0.05 C to any extent at all. What I said was:
“Your comments about what Eric Steig believes may be correct, but, if so, IMO they seem to suggest that he was using an inappropriate approach to reviewing a paper that was about correcting the method used in S09.”
In other words, I was reading your post as implying that Steig’s preconceived notions biased him towards 0.10 C over 0.05 C. On re-reading your post, I see that I was mistaken in this.
No worries, Nic (Nick?). Everyone here at CA – and the authors in particular – has been very gracious in regards to my attempts to address the communication issues, and discern Eric’s “state of mind,” etc. I am often surprised at my own blinders when it comes to unconscious assumptions, especially in a forum with no non-verbal cues. In fact, I very much distrust “the team,” as well as most anyone at RC, based on numerous interactions, writings, etc. So I was ready to “pile on” Eric myself. The final piece that made me take a deeper look was his post at RC, where he quoted his review recommending iRidge and said, “See? I didn’t recommend iRidge.” Wha?? That was weird enough, finally, to make me stop and wonder what the hell was going on.
Although I imagine some folks (on both sides) may regret some intemperate remarks, I think (or, at least, hope) that the entire discussion has been a net positive. For example, as I said above to Ryan, if I ever write a climate science paper, I would strongly consider recommending Eric (if asked) as a reviewer. That wouldn’t be possible if Ryan, in his main post, had never reported (even when he was really, *really* mad) numerous nice things about Eric, including that he was “professional and helpful” during preparation for the first draft.
I also, now, tend to believe that Eric cares more about the science than other, less noble aspects, and that he is, on balance, not too quick to give in to bias. There are numerous other people on the team that I would say the opposite is true, and they would have to do quite a lot to convince me otherwise.
The open discussion has also helped illuminate the slightly different focus some people have, which I mentioned above in reply to Dave L. (In regards to Jonas N.’s subsequent comment – “Reviewer A cannot have missed what was the main thrust of the paper.” – I would say: well … assuming he *did* miss it, then the question becomes: why? Why did he miss what was so obvious to us? Which leads again to: Eric was focusing more on the reported results, and less on the reported method. Further, he probably has his own blinders on, and simply didn’t see how his comments could be read to mean: Change your methods.)
So hopefully this whole discussion will help everyone, at least in communicating with Eric, if not climate scientists in general.
Based on his public comments and intolerance, Steig has not impressed me as a nice guy. See his baseless accusation that an author of a book about weather lied about climate. http://noconsensus.wordpress.com/2010/12/05/warnings/ (Look at posts, 5, 10, & 33)
Nice catch. A comment about a benign review of a rather good book triggers nastiness and pettiness from Steig. Steig appears to be on a hair trigger. The “knucklehead” retort is a perfect example of the pattern that is beginning to emerge around how Steig operates. Ryan might have been too generous in his assessment of Steig. I think Steve is a bit more hard-nosed – having had to deal with team shenanigans for 5 or 6 years probably sensitize his bs meter.
Ted, my thoughts echo yours.
To Steve, Ryan, Jeff and the gang. I’ve followed this since Steig’s paper was published and was conversed about in the blogs. While I won’t bother re-pasting the quotes, I think it is apparent that Steig’s knowledge of TTLS and iridge are on par with mine. That is to say, I have no practical knowledge to its use or recognition when seeing either. His quotes state as much. I think in his mind, he wasn’t urging the use of iridge, but rather the results given by iridge. I know this strikes incredulously with you guys, because this isn’t the way we consider things. But, remember what Stegman said about Mann’s work?
“…MBH…a cardinal rule of statistical inference is that the method of analysis must be decided before looking at the data. The rules and strategy of analysis cannot be changed in order to obtain the desired result. Such a strategy carries no statistical integrity and cannot be used as a basis for drawing sound inferential conclusions.”
Obviously, the team doesn’t see it that way. While I am in no manner a statistician, I’ve a fondness of the purity of math and numbers. Other people simply use them as ends towards means. You all have seen numbers and math abused to the point of numerology, and it seems you were addressing such a case in your addressing of Steig09.
This puts Steig in a conundrum. He really can’t go on record (even though the record is fairly plain) as not being aware that the changes he requested/insisted/endorsed (whatever) would necessitate the use of iridge. Consider the implications of such a pronouncement. It wouldn’t just invalidate his work, but call into question many others. He will not do that.
I think the only question remains, is how much are you willing to punish Steig for his ignorance. Given his conduct, I don’t believe anyone else can sit in judgment. It is for you guys to decide and you alone.
SUYTS Said:
“I think in his mind, he wasn’t urging the use of iridge, but rather the results given by iridge”
I think you have nailed it. The sad part is that that is probably why ES keeps insisting he did not advise/recommend/encourage/insist on… the use of iridge.
He wanted something that results as close as possible to his. The engineers are making the scientists look bad…..or their lack of math skills has done it to themselves.
That’s probably a fair take, suyts. At this point the only explanation left is that the team tends to operate with a mindset that method doesn’t matter, only results matter. I’ve seen this kind of mindset in other groups in entirely unrelated fields of science. It’s sad when you see it, and very frustrating to deal with. Science is about your method and why it is closest to reality, always has been. Some people don’t get this and think that they’re being clever by discovering a method that gets them the result they want, rather than thoroughly justifying that same method.
Re: suyts (Feb 14 11:33), I’m pretty sure suyts is essentially correct. Like many hard scientists I have spent time working at the softer end of the spectrum (easier to find money down there), and learned fairly on that many soft scientists, even very senior and experienced researchers, have little or no understanding of how their tools (whether statistical analysis methods or complicated physical instruments) actually work, never mind what their limits are or how they can mislead. They’re just treated as black boxes which give results which are then judged by how “desirable” they are.
It’s a mindset which can be highly effective, but also has a terrible potential for self deception. When interpreting Steig’s reviews it is important to remember that he probably doesn’t understand what most of the long words mean. When he describes a result as “most likely” it has no technical sense at all: he just means “most consistent with my personal understanding”. He never meant that iridge was technically better because it would never even occur to him that a method (rather than a result) could be better or worse.
I completely agree, and this is also what has struck me in the aftermath of the O’Donnell publication.
Eric Steig insists on what he claims to be “the ‘mostly likely’ West Antarctic trends” are shown, and repeatedly alludes to this notion of already knowing what reality would display if there only were many more stations/data.
In his comments at RC, he gets even bolder and also claims to know the future temperatures:
“I may not bother with a rebuttal to Journal of Climate, because in a couple years temperatures in West Antarctica will probably have reached such an extreme that none of our ‘reconstructions’ will matter
Now, I don’t consider it wrong to hold beliefs, even firm ones, about both reality and future. But those have nothing to do with science. The might motivate sciencific efforts, but never replace science itself, nor even be used as ‘infilling’.
What baffles me is that Steig, even after the fact, after having been found dabbling with improper statistics, continues to argue his case from the wrong end. That te ‘proper’ result is end and that the methods used are just a way to get there. Not the other way around. But then again, this is all to often the impression I get when ‘climate science’ is argued …
Not being an academic, but used to the ‘outside world’ where structures and processes exist especially to prevent/avoid conflict of interest, I observe that the peer/pal/enemy review process stinks to high heaven.
Regardless of the rights and wrongs of the specifics, this sorry history, and all the similar shenanigans that we have learnt about in the last few years show that this is a process that does not do what it says on the tin. It is not ‘fit for purpose’.
Can anyone from academia – with a straight face – justify the existing process as giving the wider public assurance that reviewing academic papers in this way provides any assurance at all that the content is reasonably accurate? And if not, what value does it bring?
I have lost any faith I ever had that it is anything more than an outwardly polite version of gaming with added career advancement.
In my experience, it is common for editors to send papers to referees with a personal stake. I’ve been involved at both ends (as author and referee) many times. I know why editors do this, though it always makes me uncomfortable and I’ve declined a couple of refereeing invitations when my discomfort has been very great. Once I was invited to referee an attack on a recently deceased colleague’s work. No thank you.
But, but… other things equal, editors understandably want the most knowledgable referees, and there is a tradeoff there against interests, almost unavoidably.
What I would say is that good editors who do this also have a good bs detector and will act on it. You will notice how Steve, Ryan and others surmise that the editor Brocolli eventually became exasperated with the referee, and made a decision not to ask for a fourth report from the referee.
The trouble is that some editors don’t have the spine to stand up to an interested referee who has crossed the line into obstruction.
NW:
I agree that editors should get the most knowledgable reviewers which may well mean the reviewers have a CoI. I think that the whole matter gets clearer and more manageable when such reviewers are not ANONYMOUS. Were you opposed to your identity being known when there was a strong potential CoI?
Also: As an author, you can make a case to the editor (in your private correspondence) that a referee has crossed the line into obstruction. I have succeeded at that at least once.
The peer review ‘process’ on display here is not normal. Steig’s contribution made it abnormal, while the two other reviewers were behaving sensibly (regardless of the construction of their heads). Typically a reviewer says: publish: yes, yes with modifications, or no don’t publish; and if publish with modifications, what the modifications are and whether or not they are mandatory, and whether the reviewer should see the paper again after modification. It is that simple. It seems to work in physics, chemistry, electronics, for example. The only instances I have seen of peer-review perversion (like this) are in climatology. I must admit that there was another thread at Climate Audit where a commenter protested that this type of peer review problem was common in their field. When asked what area of science he or she was experienced in, the reply was ‘social sciences’. So, possibly there are worlds of malleable facts, modifiable data, and politicized bullying, were at least some climatologists are trained, but those fields, like climatology if it adopts their practices, aren’t science (no matter what they call themselves).
Well, where peer review is being perverted (as here), we can probably also reasonably assume that science is not flourishing. My experience is in chemistry and physics, in those fields the peer review ‘procedures’ made apparent by climategate and incidents such as this Steig behavior do not happen – as far as I have seen.
Made apparent by Climategate? Please elaborate. I reviewed the emails myself after the full texts were released and found humanity, yes pettiness too, but nothing I haven’t heard of others criticizing those on opposite sides of opinion. There’s a vast gulf between wishing to not see something published though and actually having the power to make it so. I’m not convinced that they had such power.
I’m still trying to sort out the premises of those commenting on the “Steig behaviour” however. See my questions and comments about peer review above.
You can send your thoughts on the inadequacy of the peer review process to the parliamentary inquiry.
http://www.parliament.uk/business/committees/committees-a-z/commons-select/science-and-technology-committee/news/110127-new-inquiry—peer-review/
I wonder if Steig’s latest hissy-fit is a portent of a new Team attack on the JoC’s editorial board. I wonder if their anonymous email accounts have been in overdrive seeing what contacts they can use to oust the current management and censure the editor.
While I hope that science hasn’t descended to this level, they do have form.
This is useless speculation.
It occurred to me, reading Steig’s vehement but incorrect assertions about the content of Reviews Two and Three, that he may not actually be fully familiar with their content. Is there any precedent in peer review for an eminent reviewer to delegate some of the work to someone else, perhaps a post-grad student or something?
No its supposed to be confidencial to the reviewer, the editor and the authors.
Actually it depends. Many journals have policies of explicitly allowing for “assisted reviews” (often a postdoc or grad students). All of the journals that explicitly allow this require that you state this in submitting your review.
I once wrote a short paper that, among other things, criticised work by Professor A. One of the reviewers objected on the grounds that Prof A was a “scholar and gentleman”. I pointed out to the editor that howsoever scholarly and gentlemanlike Prof A might be, the sad fact was that his mathematical model failed to conserve mass when it really, really should have done.
This stuck in my mind because shortly before I’d pointed out at a conference that Prof B was claiming as original a result of his that had, in fact, been in the literature for years. Again, someone criticised my remark by alluding to B’s qualifications as a “scholar and gentleman”. Was this phrase fashionable in the US a decade ago? Was it a translation from the Chinese? It was very odd to come across it twice in a few years.
Anyway, not much doubt about Steig now, is there?
How rigorous were the reviews of S09 before acceptance for publication, I wonder?
There is a general problem in climate science which this debate shows, which is that a paper is often viewed as good if it gets the “right” results, even if proxies are upside down or the statistics are garbled. And if one gets the “wrong” results you (like I have) immediately get on the enemies list. This is itself upside down. If you don’t know what you are doing don’t do it. Always correct your data (rain in spain and all the rest). Then the results are what they are.
This is why I find it so funny when I read comments in threads that decry Global Warming as a “hoax” as if the team secretly doesn’t believe in any of it. The crux of the problem is that they “believe” in it too much, they are “sure” it exists and then set out to prove it.
Any person who is genuinely trying to conquer confirmation bias (esp. scientists) should be persistently trying to destroy their own preconceptions and expose themselves to countering information. This is nothing new of course, and I’m sure they profess to do this, it just doesn’t look like it (perhaps due to my own biases 😉 ).
In the words of Chinese Zen master Sent-ts’an:
If you want the truth to stand clear before you,
never be for or against.
The struggle between “for” and “against”
is the mind’s worst disease
Sorry if I’m getting OT.
Fascinating to review this thread nearly two years later….. Strikes me that CraigL’s comment is right on target, that there are (a variety of) double standards operating in climate science. Methods and results evaluated post hoc depending upon some separate, prior expectation of the “correct” answers….. aka confirmation bias.
I agree with Craig. A method which gives the “correct” result is used without criticism while one which produces a “less correct” result is attacked with charges that do not necessarily stand up to scrutiny. This is not science. It’s a scientific version of “the end justifies the means”, … where “means” do not mean averages ;0)
The end justifies the means, or Noble Cause Corruption. For example, if one believes in the cause, it is ok to hide the decline – hide the uncertainty in the proxy from policymakers – so as to present the “correct” story.
Reading this I find myself wondering how Eric gets his shoes on the right feet each morning. All apologies for piling on, but c’mon, how can one internally rationalize such contradictory stances being taken on the public record?
I would submit that Steig’s legion of typos and grammatical errors are reflective of his overall sloppiness and lack of professional demeanor. He’s much the KNUCKLEFINGERED REVIEWER.
My interpretation of the entire matter is that Steig was hoping the authors would eventually give up on the original paper if he reviewed it to death, and when he lost out on that gambit, doubled down and lost. Truly shocking behavior.
“The peer review ‘process’ on display here is not normal.”
I don’t mean to pick on ZT. This is certainly what I would prefer to think. But how would we know? There is no data. If there was, it would have subjectivity problems. It would also depend on the specific journal, what their peer review policy was, and the potentially subjective determination of whether such journal was rigorously applying or simply “robustly” applying their review standards.
I’m a total layman here. Am I off base? Could we use another WUWT to start collecting data?
There is some further information on this thread: https://climateaudit.org/2010/12/15/mckitrick-and-nierenberg-2010-rebuts-another-team-article/
There you’ll see various opinions on this topic. E.g. Pat Frank said “I’m a chemist, and have never seen anything remotely like what’s happened in climate science. I’ve controverted others’ work, and have never experienced a lock out.” (This was specifically on the topic of peer-review).
I would highly recommend reading:
Classical peer review: an empty gun
Richard Smith*
Click to access bcr2742.pdf
This too is pure speculation.
But suppose Eric was pushed to go much further in SO9 than he wanted to by his co-authors, especially MM.
Now he has to defend that position on his own.His co-authors silence is deafening.
Might explain some of his more intemperate remarks.
While I understand the impulse to call out behavior that can rightfully be called duplicitous, I do not believe there is anything to be gained by further responses to Steig’s ranting on other blogs. You got published, after more than the usual troubles. You were right to inform us how the review process was flawed.
(Although in my personal opinion, and based on my personal experience with journal peer review, selecting Steig as a reviewer falls within the range of accepted practice. The problem was with the editor: he was too slow to admit to himself that Reviewer A’s criticisms were mostly not constructive and not aimed at improving the quality of the journal; a good editor should have allowed the other reviewers to outvote Reviewer A earlier.)
You accepted the Team’s challenge to meet them in the peer-reviewed literature. My sincere congratulations and respect for that. There is more than enough on the record that posterity will have little difficulty coming to a just conclusion. Let Steig rant. Show us that you realize he has now reached the point of hurting himself more than anyone else possibly can. It’s your turn to say “meet us in the peer-reviewed literature.”
Bruce, it seems reasonable to concluded that within the various analysis of the whole Steig affair in this thread there has been a great deal of careful thought and insight displayed trying to get to the truth of what happened and how it happened. The civility has been outstanding, and the worth of the project undeniable. Your well crafted contribution, as I see it, goes that step beyond and encourages truth being projected forward—the proper word for that is, I think, “wisdom”. Thanks.
Ron
“This is especially true since he has freely admitted that he is not thoroughly familiar with the iRidge algorithm.”
As I read the O(10) reviews from Reviewer A and the replies, I got the distinct feeling that Reviewer A was in a conundrum since iRidge got a little of that West Antarctica warming back that Reviewer A was looking for, but at the same time did it using a method not used in S(09) and one, as I recall, that was not necessarily recommended by other Team members (Mann). He may merely have wanted better literature justification by the authors of O(10), who by now he knew were doing literature searches and reading in good detail and understanding. He may have wanted to see the graphs that iRidge produced as opposed to those with lesser warming, but thought there better be ample justification in going against the Team. In summary a reviewer desperate for warming concessions yet conflicted by the use of iRidge.
An alternative explanation might be that since Reviewer A, after all, was still pushing for a change in the truncation parameter with TTLS that he might have judged would gain back even more warming than going to iRidge? I am rather certain he would liked to have seen the O(10) authors go back the drawing board so he could see what algorithm and what truncation parameter resulted in the most West Antarctica warming and then perhaps argue for that method.
If indeed Reviewer A was on a mission to get as much WA warming as possible out of O(10) and let the O(10) authors do the work (with the editor hopefully going along) I think his actions here are consistent with that picture.
Gossip? Seriously?? Seriously, gossip???
What ever happened when you used to post science here. This is like an episode of Greek. Or 90210 for you older folk. Or whatever high school/teenage drama tv show you like.
Like this is the only paper to ever undergo review by someone who doesn’t like the authors… riiiiight.
Quit the _____ and get back into the kitchen.
Go do some science.
OMG – did you see what bender and mosh were wearing??? Oh no the didn’t copy my outfit.
Re: Willy Roberts (Feb 14 22:14), Mosh doesn’t copy, especially when it comes to fashion. Don’t know about bender, when I fill the pool and he comes around to clean it I’ll let you know.
No, I read your comment twice. I thought I should pass, but then…. R U kidding me?!?!?!?!??! So the scientist of the paper that these scientists are rebuking, publicly states in a science blog a duplicitous statement regarding the scientists that wrote the rebuking paper. And you say they should get back to science? Take a look around. Smell the air, and understand, they aren’t just engaging in science, they are framing science. Remember when the person running this blog wasn’t allowed to engage in “science”? Or perhaps more succinctly, wasn’t allowed to engage in “climate science”? Gossip indeed. It isn’t gossip if the facts are laid out on the table.
Questions serve as poor conversation starters. An introductry statement allows you to begin sizing up your opponent.
Correct me if I’m wrong but the original purpose of O’Donnell et al is to demonstrate the weak methods of Steig et al’s 09 paper. Both TTLS and iRidge methods accomplished this feat. I am sure RC was ready to bring it hard against either method.
Currently the debate seems to be whether iRidge or TTLS was the best way to shoot holes in Steig et al ’09. I have followed this since it erupted (thanks to Watts). I’ve seen bits and pieces making the claim that both methods show the problems with Steig et al.
Now the question, Is there or will there be a post to demonstrate that using either TTLS or iRidge brings into question the methods and results of Stieg?
The answer to your question is “yes”.
Thank you, that is good to know. I’ll stock up on beer and popcorn! This episode has been very informative and I look forward to the next post and round of comments.
It does seem that the Reviewer A comments are a rather nasty stain on the blue dress of Steigs reputation.
This is a bit off target, Steve, so sorry.
I think that when the CRU leak happened, it so appallingly revealed what, in your pessimistic moments, you merely feared, that you were stunned into silence? For, after all, if people behave so badly, who have the power to ‘behave badly’, what is the point? I, therefore, want to plead with you, please start your work again. For without you (or JeffId!) all we will have is the nutters who wrote that absurd ‘Dragon’ book. I’m listening again, to Lord Lawson’s rather rambling (perhaps, he is intoxicated?) replies to our parliamentary Committee. Lucidity is what I miss, which I’m sure, if you were asked (!), you would have supplied! We miss this.
Once again it’s clear that getting around in Climate Science you need also a degree in psychology and philosophy…
What is it with these climate guys and their PCA plots? I posted links on skepticalscience.com, trend plots that show no increase in tornadoes (F3-5), hurricanes (Cat 3-5), droughts, precip, and snow. In reply, I was directed to a paper by Dai, Trenberth, and Qian (2004), concerning drought trends. But the PSDI temporal trend plots (PC 1) contains both precip and temperature data for 1950-2000, precip only 1900-1950. Why add temperature to only the 1950-2000 data? The spatial plots show dry areas associated with El Nino’s, not a clear AGW signal as suggested by the authors. The Steig 2009 paper seems to smear warming from the Antarctic peninsula onto the western mainland. I’m very skeptical of the use of PCA by climate critters to detect the AGW smoking gun. This reminds me of an old joke that I’ve expanded, bad mathematicians become physicists, bad physicists become meteorologists, bad meteorologists become climate scientists.
Leaving aside the scientific arguments of who is right and who is wrong, what I find absolutely fascinating is the issue of whether Steig should have been a reviewer or not.
I do not know any other field of learning/business/endeavour where it would be accepted that that such a conflict of interest would be accepted and that the reviewer could hide behind a cloak of anonymity. It doesn’t matter whose call it was – the editor’s, or Steig’s, by any standard it is ethically inappropriate, whether it is accepted practice or not. Why this is not immediately obvious to even the most partisan reader is lost on me.
No wonder there is such debate over the process of peer review. No wonder there is such suspicion!
The more I read about the review process of this paper (O’Donnell et al (2010)) the more I feel that the paper was made weaker by the changes required by the editor in response to Reviewer A’s comments. The changes that Reviewer A requested/insisted upon and the editor enforced were for the worse. The resulting paper is still pretty good but not as good as the original and that’s a pity.
Steve- glad that you think so as well.
For other CA readers, Nicholas (not Lewis) was the originator of some of the scripts that I’ve used to access data. I’ve posted on some of these technical methods in the past.
Speaking personally, I think that while some of the changes required by the editor in response to Reviewer A’s comments were bad, others ended up for the best. Losing the section discussing Chladni patterns was bad, for instance, and the original paper (and SI) provided a great deal more detail in a number of areas. But, having spent quite a lot of time working on the Antartica data with both methods, I have little doubt that using ridge regression (the result, even if at the time he didn’t realize it would be, of Reviewer A’s comments) rather than a truncation-based Total Least Squares (TTLS or TSVD) method produced a significantly more stable reconstruction, and one that is likely more accurate.
I agree Nic but the problem is that if you are using ridge regression, even if it gives a superior result, the result is not quite as instructive about where Steig went wrong because he wasn’t using ridge regression.
In other words I don’t think it’s as strong of an “apples to apples” comparison.
I’m not saying that, on balance, the final paper, using ridge regression, is necessarily inferior because of that, more that the switch is a double-edged sword which has both benefits and drawbacks.
It is that, in combination with the loss of some interesting points (especially regarding the Chlandi patterns) that made me say that on balance, the final version is not as strong as the original submission.
Forgive me if I’ve missed this as I started skipping about toward the end, but since Reviewer A (Steig) is being criticized for his lack of knowledge in statistics, as is it seems to me is the case too for the Journal of Climate by Mr.McIntyre, does this indicate that the identities of Reviewers B, C and D are known as well?
As a person involved in research in an entirely different field (Molecular and Cellular Biology) I’m not a complete stranger to the peer review process, which must be somewhat similar to the case here. We often submit papers to journals and endure a grueling process of back and forth, sometimes are requested to generate additional data, and often face some mild rancor as there are also areas of controversy and camps with differences of opinion. We are allowed by certain journals to request that the submission NOT be sent to a particular person for review as we are aware they are working on something similar close to publication for instance, and sometimes I’m certain, there are personality factors in play. Likewise we can also suggest reviewers who are more knowledgeable in our particular field, which is cutting edge . However, the editor &/or review assigner are under no obligation to acquiesce to our requests.
If I may be frank though, there seems to be a bit of naivete on the part of many posters as to nature and purpose of the peer review process. it is not a rubber-stamping committee and most journals that I know of are under no obligation to publish a submission. Also, the word “peer” is there for a reason, and in fields such as mine, as I imagine is true in atmospheric science as well, there is a limit to the size of the community available to review which does often result in an article or grant going to the person who knows the science and is a competitor.
Rigourous and robust questioning and commentary ensues, even in our field, but this only serves to hone the work. At first some of the criticisms can feel harsh but often the reviewer is just busy with a million other duties in his/her own research and has no time to hone the review for feelings, sometimes not even for spelling error correction!
Somewhere, it was suggested that reviewers are paid . . this is not true, at least to my knowledge. For granting agencies as well as non-commercial publication one is expected to serve a stint if nominated, especially if one has received grants from said agency or attained a certain respect in the field.
And finally, most of the time reviewers do remain anonymous, although if one is familiar with the writing style of a particular scientist through scientific articles or their particular emphases, one can sometimes guess as to the identity of the reviewer. As for outing them publicly though, well, all fields have their kurfluffles and I’m sure after a few drinks at a conference it happens with us too, but it seems rather counter-intuitive. To return to my original thought though, I wondered what B, C, and D thought of this controversy.
Just my 2¢
## 2011-02-14, 2011 21:45 CST – Eli Rabett:
> How ridge regression could ever have been his idea after it was not mentioned in the paper, not mentioned (at least now according to O’Donnell) in the Supplementary Information and not mentioned in Steig’s first review is one of those mysteries known only to blog scientists.
http://blogs.chron.com/climateabyss/2011/02/steig_this_is_not_complicated.html#c2048120
## 2011-02-15, 02:22 CST – Steve McIntyre:
> Re First Submission SI – version attached to first submission is now online (dated 20100121).
http://blogs.chron.com/climateabyss/2011/02/steig_this_is_not_complicated.html#c2048169
Mr McI;
Please consider threading those contemporaneous ES posts (after each of his responses) into your chronology.
They’ve gotten lost in the fray, and I thought they were an important color in the big picture.
In retrospect, his failure to follow his pattern (of posting a signed blog comment after each of his ‘anonymous reviews’ ) might have been a clue that he didn’t see Rev 4.
Maybe Ryan O could apologize for not noticing the subtle shift in ES behaviour, that ES didn’t continue to tweak the authors by posting non-anonymously at (one of) their weblog, after having done it consistently before.
TIA
RR
I am not at all certain I know what the objections are to the “revelation”. I think those of us who have studied the reviews of Reviewer A and had an initial interest in the subject matter from the analysis of S(09) found those public reviews helpful in better understanding some of the more nuanced statements in S(09) , the literature that was discussed at length in reviews and pertained to S(09) and O(10) and materials that put were forth in the initial O(10) submission but not published. I think much can be learned in general by making the review process public even with the reviewers being anonymous. Is anyone protesting about making the reviews public? Surely reviews can be an important learning tool and particularly so in taking advantage of the communication that the internet permits.
.
If we get by making the reviews and submissions public, we have only to deal with revealing the identity of Reviewer A. I assume that no bylaws or rules of the publisher were infringed with the revelation and since it is difficult to get an agreement on the terms of the gentlemen’s agreement involved all that is left is to determine if anyone was individually injured by the revelation.
.
O’Donnell made the revelation and I assume, save his wanting to injure himself, he was not hurt by it. That leaves Eric Steig. First we have a situation where the reviews when first made public were going to raise some very serious speculation by interested parties that Steig was Reviewer A. Given blogs propensity to search for all the clues in cases like this one there was a good chance most would have a strong inclination who the reviewer was. Certainly Reviewer A revealed a strong ownership to S(09) through his reviews.
.
The issue here in my mind then comes down to what harm was done to Steig by interested parties knowing that he was Reviewer A. It was rather obvious that Steig had specifically a strong interest in maintaining as much of the West Antarctica warming claimed in S(09) as he could. He wanted every avenue explored in the review/reply process that he could get out of that process to afford alternative methods to find one that might have the greatest WA warming, i.e. in his mind the “correct” warming. Has not that been the thought process revealed outside the review process by Steig? Was there really anything in the review we know about Steig and his position on S909) versus O910) that he had not already revealed to the public? Actually the way Steig attempted to make points in the reviews versus his non review approach would I think give higher grades (given we forgive the tenacious tenor of his reviews) for Steig, the reviewer, than Steig, the blog commenter. I, therefore, judge that any harm that has come to Steig has been from his actions outside the peer-review process.
What a waste of time and talent.
Steig admits he isn’t a statistition. “… As O’Donnell’s co-authors are fond of pointing out, I am not a statistician, …”
I think this whole fiasco elevates Steig to the level of Mann et all. Completely debunked.
Great to see the reasoned, patient summary by the Master.
Thanks Mr. Steve for all you do!
and JeffID and of course, Ryan
EJ
17 Trackbacks
[…] Steig and the “KNUCKLEHEADED REVIEWERS” […]
[…] Steig and the “KNUCKLEHEADED REVIEWERS” Over the past few days, Eric Steig aka Reviewer A has made a series of increasingly puzzling and strident outbursts, as […] […]
[…] Climate Audit by Steve McIntyre Skip to content Hockey Stick StudiesStatistics and RContact Steve McProxy DataCA blog setupFAQ 2005Station DataHigh-Resolution Ocean SedimentsSubscribe to CAEconometric ReferencesBlog Rules and Road MapGridded DataTip JarAboutCA Assistant « Steig and the “KNUCKLEHEADED REVIEWERS” […]
[…] history of the affair is quite lengthy and will not be reviewed in this post. (See here for example.) Two very distinct issues arise in regard to the Steig […]
[…] Here’s the two minutes of hate in response in the comments: don Posted Feb 13, 2011 at 4:17 PM | Permalink | Reply […]
[…] the two minutes of hate in response in the comments: don Posted Feb 13, 2011 at 4:17 PM | Permalink […]
[…] Source: https://climateaudit.org/2011/02/13/steig-and-the-knuckleheaded-reviewers/ […]
[…] though the authors of the O’Donnell paper wrote directly to the Editor (Broccoli) wrote to ask that Steig “be treated as a conflicted reviewer or that his review, at least, be sent to […]
[…] Apparently, Steig’s post-review change-of-heart includes forgetting that he saw the revised SI with the table removed, forgetting that he knew why it was removed, and forgetting that the only additional request he had with respect to the SI was that we add a few words on iRidge (which we declined to do for reasons stated at the end of this post here). […]
[…] […]
[…] Warmteperiode werd weggepoest, of de creatie van warmte op de Zuidpool zonder meetstations (Steig et al op cover Nature in 2009, en zijn invloed op de rebuttal).Polarbeargate is nieuwe beeldenstorm in klimaatkerk Laten we nog eens op een rijtje zetten, hoe […]
[…] Steig et al (Antarctica is warming) where Steig himself inappropriately served as a reviewer, and a hostile one at […]
[…] Steig et al (Antarctica is warming) where Steig himself inappropriately served as a reviewer, and a hostile one at […]
[…] those were good times! Of course, with this more current map of temp trends, we can see part of the […]
[…] but the whole of the continent is not.” And forget all of the back and forth and Steig demonstrating his cluelessness about statistics. What did the paper state that he […]
[…] Steig-O’Donnell conflagration continues to be volatile, with little hope for […]
[…] Hmm, so those 1100+ papers at poptech’s don’t really exist? More than that, Dr. Lindzen isn’t published? Dr. Christy isn’t published? Dr. Spencer isn’t published? and on and on ….. For those who may not know, yes, they are all skeptics, experts in the climate field, and collectively they probably have somewhere between 500-1000 published themselves. But, about those non-experts …. people should ask Dr. Steig if they can or can’t get papers published. […]