Re: “Can you imagine an econometrician satisfying an editor with such a statement?”

First, I think the answer is “Yes.” Of course, that “Yes” is dependent on testing for serial correlation in other, (presumably) more robust ways or that the data sequence is arbitrary as in cross-sectional data. The latter, of course, does not, by definition, apply to chronologies. As to the former, one would usually expect an author to say something like, “We tested the influence of serial correlation on the test statistic with the X-Y-Z test [standard alternatives are Breusch-Godfrey and Ljung_Box which can handle multiple orders of autocorrelation and one could come up with better tests for non-linear models] and did not use, or hence report, Durbin-Watson test values.”

I agree. Normally an author would not actually provide the code or data, but would run the requested test and report the results. This is standard when a referee asks for a robustness check. So, asking for the code is not standard practice.

But, Mann’s response to the request is a bit bizarre. I would expect the response to be: “The R2 statistic is …, but this is unimportant for the following reasons.” But Mann doesn’t report the requested statistic. Instead he only argues that it is inappropriate or wrong. Simply courtesy for the reader requires that the question actually be answered.

Mann’s response to the inclusion/exclusion of bristelcones is also a bit bizarre. If I were the referee, I would expect the response to be “Yes, results are sensitive to inclusion of the bristlecones which is ok because …” rather than “but you can’t think about excluding the bristelcones, you are throwing out data.” Of COURSE the analysis without the bristlecones throws out some of the data. That is the whole point, to see which data is driving the results and which is not. It is highly informative to know which data series we shold focus on and which are peripheral. The reader can then consider whether flaws in the key series might undermine the results.

]]>At the time of the initial request, there was not even an accurate rendering of what data was used – he said that in the original article that 112 series were used, then after our 2003 article, he said that 159 series were used, then the Corrigendum SI listed only 139 series. Some series listed in the SI as being used in the principal components calculations were not used; the principal components methodology was notoriously misrepresented; there are other undescribed methods. So it was (and remains impossible) to exactly replicate his calculations. Even now both Ammann and Wahl and ourselves (who do agree) can’t reproduce his early 15th century results other than in general terms.

The way that he tested for serial correlation is pretty odd-looking to someone coming at it from an econometrics perspective. He eyeballed the Fourier spectrum of the residuals as to whether the spectrum was consistent with white noise. The residuals from the AD1820 step probably are, but perhaps not the earlier steps. (This doesn’t deter them.) They report for calibration residuals, but not verification residuals. I’m looking into this methodology as we speak, as I’m not familiar with the properties of the method, which is also used in Rutherford et al, 2005.

I think that there may also be issues about whether usual tests for autocorrelation in residuals will be effective against ARMA(1,1) residuals with high autocorrelation – for the reasons outlined in Deng [2005].

However, issues of autocorrelation in the residuals got overtaken with the catastrophic and unexpected failure of the cross-validation R2. But at the outset, I had no idea or reason to suppose that MBH98 would fail such a simple statistical test. Hence the request for residuals (or something else that would be equivalent).

]]>First, I think the answer is “Yes.” Of course, that “Yes” is dependent on testing for serial correlation in other, (presumably) more robust ways or that the data sequence is arbitrary as in cross-sectional data. The latter, of course, does not, by definition, apply to chronologies. As to the former, one would usually expect an author to say something like, “We tested the influence of serial correlation on the test statistic with the X-Y-Z test [standard alternatives are Breusch-Godfrey and Ljung_Box which can handle multiple orders of autocorrelation and one could come up with better tests for non-linear models] and did not use, or hence report, Durbin-Watson test values.”

Second, if he had supplied the residuals, either explicitly or implicity [see third point], the D-W statistic is a pretty straightforward compuation:

DW = Sum[t=2 to N] (e(t)-e(t-1))^2/ Sum[t=1 to N] e(t)^2. Thus, it seems that the issue is not whether he supplied the D-W statistic, but whether he tested and accounted for serial correlation.

Third, why were the residuals needed? Did Mann fail to supplied the dependent variable and its estimates [the Y and “Y hat” values of the Y=F(X,e) relationship]? Maybe I should add, that I still haven’t figured out just exactly what variables are the Y and X’s or what is the “F(…)” of his calibration and verification.

]]>I apologise for my error in referring to Barton as a Senator rather than a Representative.

However as Steve has already noted, Rep Barton’s Energy Committee has seniority over Rep Boehlert’s Science Committee and that Barton has been unafraid to use that seniority in order to ask critical questions about the science that is supposedly shaping and informing future energy policy.

]]>Folks unfamiliar with the United States Congress might not realize that Barton stepped about as far into another committee’s turf as he could. If there’s one thing closer to a Congressman’s heart than perks from lobbyists, it’s turf. The NAS information is interesting, if true. It probably means that Boehlert’s committee (the science one) is either not going to hold hearings at all, or will only do so if the NAS report provides a reason. We shall see, but I doubt there will be a reason. In any case, once Boehlert had publicly stated the Barton letters were out of line Mann was in no danger of reprisal for being insufficiently responsive (if he actually was).

]]>