DF criticized my post on principal components yesterday as follows:
Most of your figures for conventional PC analysis are misleading. You are comparing PCA1 to mean as if PCA1 has an intrinsically meaningful scale, when it does not. If you rescaled your comparison plots so that PCA1 and the mean had the same variance, then the results would be nearly indistinguishable (aside from questions of orientation). I do not believe such near equivalence holds for the Mann, offset-centered method.
I disagree with this comment on a number of grounds. In fact, I think that the scaling appropriately illustrates the near-identity of the PC1 and the first HS-shaped series. I agree with DF that, in the circumstances of this example, the rescaled mean approximates the PC1, but I take home an entirely different message: this illustrates the well-known non-robustness of the mean and illustrates the need for climate scientists to use a robust measure of location.
In my example, the weight of X[,1] in the PC1 is near unity – empirically it’s about 1-àÅ½àⴞ2 here where àÅ½àⳠ~ 0.025 and the weights of the other series are close to 0 – empirically about àÅ½à⳯sqrt(n)*flip[i], where flip[i] is +1 or -1, roughly half and half here. The PC algorithm is not sensitive to n in picking out X[,1] from white noise; it is sensitive to the standard deviation à?à’ of the noise series, as à?à’ increases, the PC algorithm will eventually find random patterns in the white noise instead of the signal. In this toy example, the HS blade is big enough that it separates itself from the noise. In such a case, the pc1 is approximated by:
(1) pc1~ (1-àÅ½àⴞ2)* X[,1] + àÅ½à⳯sqrt(n) àÅ½à⡠(i=2:n) X[,i] *flip[i]
Figure 8 below shows a slightly re-stated Figure 1 from yesterday, this time with the median (blue) added in the right panel. It notiveably separates from the mean (red).
Figure 8. As with Figure 1 yesterday. Blur is sd=0.05. Blue – median.
In this particular example, it’s the difference in variance that makes the HS stick out (rather than its HS-ness per se, although the HS-ness comes into play in the Mannian method.) To show this, I randomly permuted the values of the HS-series in Figure 9 below, leaving everything else unchanged. Obviously the PC1 recovers the high-variance series. You can also see where the mean and median yield different results.
Figure 9. As with Figure 8, with permutation of order.
DF stated that the rescaled mean would have similar properties to the PC1 in this particular case and this is, in fact, true. Figure 10 below shows the PC1, mean and median. As noted above, I take a very different moral from this. We know that the PC1 is to a very close approximation, simply the high-variance outlier series (the HS series in this example). In this example, 9 of 10 series are constant at 0 up to white noise and there is one outlier. The weights in the PC1 recover the outlier rather than the null value of 9 of 10 series. The re-scaled mean also recovers the outlier. If you think about it for a minute, the approximation of the PC1 to the re-scaled mean can be understood by working through the algebra a little. Since pc1~ X[,1],
pc1 — n* àÅ½à⺠ ~ X[,1] — n* (1/n * àÅ½à⡠(i=1:n) X[,i])
= (n-1) * 1/(n-1) àÅ½à⡠(i=2:n) X[,i]) =(n-1)* mean(X[,i])
Since the X[,i] i=2:10 are by construction white noise (àÅ½à⺠here being the mean series and à?à’ the standard deviation of the white noise series):
var(pc1- n*àÅ½à⺩ = (n-1)^2 var(mean(X[,i]))
= (n-1)^2 * à?à’^2 /n ~ (n-1)* à?à’^2
sd( (pc1-n*àÅ½à⺩ = sqrt(n-1) * à?à’
I checked this formula empirically and it seems to be correct. So if you have one outlier series and (n-1) white noise series, the PC1 and n*àÅ½à⺠are going to be fairly similar, as DF observes. But that’s not the end of the story; it’s the beginning.
Figure 10. Scaled versions of PC1, mean and median.
The idea of variance re-scaling is one that is very peculiar to Hockey Team climate science right now. Variance re-scaling is implemented in Briffa, Mann; it’s discussed at length in von Storch et al  and Esper et al . But you can’t go to Draper and Smith or Ripley and find a discussion of this method or a recommendation that it be done. I could stand corrected on this, but it seems to be somewhat sui generis to Hockey Team climate science right now.
We know that the PC1 is not robust since it essentially only recovers the outlier series X[,1]. Since the rescaled mean approximates a non-robust measure, it is evident that the re-scaled mean is not a robust statistic either -using "robust" in its technical sense. . If you google this term, you’ll find many discussions e.g. Ripley here . Ripley (and similar studies) specifically identify the "mean" as a non-robust statistic. Considerable effort has been spent in statistics over the past 20-30 years (see Ripley references) to develop location measures that are not subject to breakdown. The median is a simple method, shown here, because it is easy to implement and illustrates the point nicely.
So, while as DF points out, the rescaled mean approximates the PC1 (which is nearly identical to the outlier series), both methods in this particular example are yielding non-robust "reconstructions" and a very different result is obtained using a "robust" statistic – the median. Obviously this issue affects not just to MBH98 but to the many Hockey Team studies using pocket ( say 5-15) subsets of proxy networks.