Rob Wilson has written in sharply criticizing me (Yamal Substitution #3) for a lack of a balanced presentation on the Yamal substitution, and, in particular, for not acknowledging the "clear statistical reasons (related to variance changes through time)" that he had provided me offline for why D’Arrigo et al 2006 made the Yamal substitution.
Also see here here here here
I take any criticism from Rob very seriously and regret that he feels that I did not represent his position on this adequately. The post in question only discussed Briffa – it made no reference to D’Arrigo et al 2006. In a previous post on this topic , I had referred to Rob’s in passing to this argument where I said:
Rob Wilson has written me offline, attempting to justify the switch on the basis that the variance of the Yamal chronology is more stable than the variance of the updated Polar Urals chronology.
Because (inter alia) he’d sent the details offline, is not an official, had not made any public comments about the matter and had not mentioned that I could post them up, I referred to the position (with which I disagree), but did not present it.
Here is a more detailed presentation of Rob’s argument (which I do not agree with). I did not present this argument since it had been sent to me offline, but I have no objection to presenting and discussing it. I don’t think that it’s very convincing and, far from settling the substitution, raises as many questions as it answers. If it’s been relied on to make an important substitution, I think that it should have been clearly presented in the original article accompanied by an impact assessment. None of this should be construed as a criticism of Rob personally who is earnest and diligent, but who did not make all the decisions in respect to this article.
Reviewing the bidding a little. I first encountered D’Arrigo et al, 2006 last fall in connection with the IPCC 4AR review. It was unpublished at the time. I asked IPCC to provide the supporting data. They refused. This led to considerable correspondence which is a long story in itself, which I’ll write up on another occasion. D’Arrigo et al 2006 was published almost concurrently with Osborn and Briffa 2006, which took much of the publicity away from it. It’s too bad. D’Arrigo et al 2006 is a vastly superior paper. I published some first comments on Feb 11 focussing on bristlecones. In response to that note, Rob wrote to say that he used Briffa’s (2000) Yamal series because he "could not develop an RCS chronology that had homoscedastic variance through time using the Polar Urals data." He said that Briffa would not give him his Yamal raw data but "said that the Yamal series was a robust RCS chronology".
On Feb 12, I wrote back as follows:
Rob, what test did you use for homoscedastic variance? Did you get homoscedastic variances in the other reconstructions? Cheers, Steve
On Feb. 20, Rob wrote back:
re. variance this is a sticky issue and there is no definitive way to say if the variance over time in a chronology is right or wrong. In general, I try to develop chronologies which have as stable variance as possible. My decision is generally through eye balling the final chronology. For example, in the attached word file, you will see one of the RCS chronologies that I developed using the Polar Urals data. Clearly, the variance is not stable through time and RCS detrending has not done a good job in developing this chronology. Of course, using this chronology would greatly change the story of the past 100 years, but hopefully you would agree that this version would be wrong. Hence my use of Yamal, which at least had a roughly stable variance through time
Figure 1. Polar Urals Chronology. Rob’s Caption: One of the many RCS chronology versions that I developed. They all looked similar, although some iterations were better than others. The problem, I think, are the raw data around ~1000 and ~1400 with higher RW values, which needed to be properly processed to as not to bias the final chronology. The raw non-detrended chronology looks very similar to the above graph suggesting that RCS simply was not adequately removing the growth trend in the data.
Rob’s RCS version of Polar Urals looked somewhat like a version that I had previously posted up at CA.
Figure 2. My Version of Polar Urals Chronology
On Feb 21, I replied to him as follows:
Rob, here’s an RCS version [SM note: the one shown in Fig 2 above] that I got from the Polar Urals data using the best nls-fit to all the data. I applied a Goldfeld-Quandt test for heteroskedasticity (used in econometrics) to this series and to the Yamal series with a breakpoint at 80% through the series and got better results for the Polar Urals version than for the Yamal data. A p-value towards 0 implies heteroskedastic. This would not support the replacement. Cheers, Steve
#Yamal GQ = 1.4711, df1 = 399, df2 = 1595, p-value = 1.938e-07
#Polar Urals RCS GQ = 0.5621, df1 = 970, df2 = 241, p-value = 1
On Feb 21, Rob promptly replied:
I guess we need to agree to disagree on this subject. Attached are two 101-yr running variance plots for the RCS series I showed you yesterday and the Yamal series. I have never used the Goldfeld-Quandt test, so cannot judge the results, but from the attached figure, the Yamal series is clearly superior to the RCS version. I do not deny that there could be problems with the Yamal series especially at the recent end. But at the time of the analysis, I needed to generate a STD and RCS version for each record. As I could not generate my own version, the Briffa Yamal series was the next best option.
Figure 3. 101-year running variance. Briffa Yamal RCS versus Wilson Urals RCs.
While this particular graphic may be an argument for using the Yamal version rather than the updated Polar Urals version, it hardly settles the matter, especially in Osborn and Briffa. Retention criteria are stated differently in Osborn and Briffa; they do not mention anything about variance stabilization as a criterion. So even if this criterion was used in D’Arrigo et al 2006, it does not mean that it was used in Osborn and Briffa. We have just learned that the Polar Urals Update was used in Esper et al, 2002.
If variance stablization is important, why wasn’t it applied in Esper et al.? If a variance stabilization test is used in site decision-making, why isn’t it mentioned in any of the papers? What is the test – the Yamal variance doesn’t look all that stable? Given that the Yamal substitute is the most extreme hockey stick in the entire selection, shouldn’t the impact of the substitution be pointed out? Maybe it would be relevant to look at the underlying Yamal data to see what’s going on – oops, can’t do that. Were variance stabilization criteria applied to the other sites? On what theoretical grounds does one expect tree ring site chronologies to have stable variance over time – financial series don’t have stable variances over time, hence the development of ARCH and GARCH methods (see Engle and Granger Nobel prize) – maybe stable variance isn’t an appropriate criterion? Even if it is, it should be stated and justified.