Iridge versus TTLS. What if a key text on this conundrum of the day resided in an anonymous open peer review? Would we, within the ethical standards of modern climate science, be entitled to speculate on the identity of the author of these pearls? Or would that be an ethical violation “as bad as possible”? Even if we merely speculated privately, alone here on the internet, just you and I, dear reader? Sometimes truth is stranger than fiction.
Burger and Cubasch, a rejected submission to Climate of the Past (discussion paper here), tested a variety of multivariate methods on a standard set of proxies (MBH98), including the two RegEM variations of Schneider 2001 (Ridge, TTLS) that are the source of the present dispute.
The comments of Reviewer #2 here are of particular interest as he commented on the advantages and disadvantages of RegEM Ridge versus RegEM TTLS – the topic of today’s controversy.
Reviewer #2 featured RegEM Ridge as his primary case:
The RegEM method of Schneider (2001) accomplishes regularization through the use of ridge regression, introducing a “regularization parameter” h that specifies the degree of inflation (1 + h^2) of the main diagonal of the covariance matrix . The parameter h determines the degree of smoothing of the estimated missing values. Schneider (2001) uses Generalized Cross Validation (“GCV”) to determinate an objective estimate for h….
Reviewer #2 continued with a powerful condemnation of RegEM TTLS relative to RegEM ridge (see bolded section below), observing that “an optimal regularization of TLS leads directly to ridge regression, and not truncated TLS”:
the implementation of RegEM as defined by Schneider (2001) and employed by Rutherford et al (2005) involves only one statistical model, the solution of the above equation above using ridge regression for regularization, using GCV to select the regularization parameter h. It is true that there are a number of other possible ways to regularize the EM algorithm, including Principal Components Regression (PCR), truncated total least squares regression, and ridge regression. Schneider (2001) however specifically favors one unique regularization approach, ridge regression, since it arises as a regularization method when the observational error in the available data is taken into account.
Ridge regression regularizes a total least squares regression, provided the relative variance of the observational error is homogeneous. This assumption is appropriate when, as in applications to paleoclimate reconstruction (e.g. Rutherford et al, 2005; Mann et al, 2005;2006), available data series are standardized prior to their use in CFR. Even if this assumption is not met, the true regularized estimates are close to the estimates provided by ridge regression. The alternative models proposed by Burger and Cubasch are likely to provide estimates with greater variance.
Consider, for example, their use of truncated “Total Least Squares” (TLS). This is not an optimal approach to regularizing TLS. Under the assumption of homogeneous relative errors in the standardized data as discussed above, an optimal regularization of TLS leads directly to ridge regression, and not truncated TLS, which is indeed the reason Schneider(2001) employed ridge regression in the RegEM algorithm.
Given Reviewer #2’s endorsement of RegEM Ridge over RegEM TTLS, there is obviously a strong temptation for O’Donnell co-authors to try to figure out who he is and perhaps appeal for his endorsement of RegEM Ridge in the present controversy.
Unfortunately for this option, we have recently learned (through the kind intervention of Nielsen-Gammon here), that, even in the case of open review comments, ethical standards within the climate science community forbid speculation on the identity of Reviewer #2 (except in private). (See also here) Nielsen-Gammon:
Regarding your first question, whether it is unethical in my opinion to speculate on the identity of reviewers in open review systems, I would regard it as either unprofessional or unethical, depending on the circumstances. The reviewer has chosen to be anonymous or is required to be anonymous (depending on the journal), and that choice or requirement should be respected. It rises to the level of unethical if it is done in the context of a criticism of the reviewer, because the subject of the criticism may be unable to respond effectively and truthfully while remaining anonymous. One of the key purposes of anonymity is to protect the reviewer from personal criticism.
Regarding whether speculating privately is equivalent to speculating publicly, I think not.
Thus, much as the authors of O’Donnell et al would like to invoke the assistance in the debate of Reviewer #2 (with his strongly advocacy of the technique used in O’Donnell et al 2010), it seems that, within norms of the climate science community, it would be “unprofessional or unethical” to speculate on the identity of Reviewer #2.
Unless, of course, we do so privately. Like the Climategate correspondents.