A question that Jean S inquired about before we were so rudely interrupted. The expanation in Steig et al was: Principal component analysis of the weather station data produces results similar to those of the satellite data analysis, yielding three separable principal components. We therefore used the RegEM algorithm with a cut-off parameter k=3…. A […]

“Noisy” covariance matrices have been discussed here on many occasions in a variety of contexts, largely because the underlying strategy of Mannian methods is to calculate the covariance of everything to everything else and then calculate verification stats using methods that ignore the data mining that effectively takes place with huge covariance matrices. Steig et […]

Yesterday, we discussed the remarkable decomposition of the AVHRR reconstruction into 3 PCs, initially observed by Roman and discussed yesterday by Jeff Id. I thought that it would be interesting to see what happened with the PC3 in the AWS reconstruction (and in the process, do some comparisons of the RegEM emulation (file-compatible between Jeff […]

Roman Mureika perceptively observed about 10 days ago that the AWS reconstruction consisted of only 3 PCs. Jeff Id has extended this to observing that the AVHRR reconstruction consists of only 3 PCs. Jeff’s demonstration of this is correct, but a little awkward. I’ll show an alternate demonstration which shows a useful linear algebra property […]

I’ve now ported my emulation of Schneider’s RegEM PTTLS to R and benchmarked it against Jeff’s Matlab as shown below. I caution readers that this is just an algorithm. There are other ways of doing regressions and infills. The apparent convergence to three PCs noted by Roman is still pending as a highly interesting phenomenon. […]

I’ve transliterated relevant Tapio Schneider code into R (pttls.m) and parts of regem.m that seem relevant at present. Jeff Id has extracted a variety of intermediates from his Matlab run and I’ve fully reconciled through two steps with remaining differences appearing to be probably due to transmission rounding. My dXmis statistic at step one was […]

Standardization in Mannian algorithms is always a bit of an adventure. The bias towards bristlecones and HS-shaped series from the impact of Mann’s short segment standardization on his tree ring PCs has been widely publicized. Smerdon’s demonstration of defects in Rutherford et al 2005, Mann et al 2005 and Mann et al 2007 all relate […]

The two Jeffs ( C and Id) have interesting progress reports on emulating Steig using unadorned Tapio Schneider code here. Check it out. One of the first questions that occurred to third party readers was whether RegEM somehow increased the proportional weight of Peninsula stations to continental stations as compared to prior studies. Jeff C […]

Roman M has already done one post on the impact of the Harry error. Ryan O has also done so [see comment here]. As has Steig. I show below some graphics that I’ve just done on AWS recon trends. At Steig’s website, he now states: awsreconcorrected.txt is a correction to the above file, using corrected […]

So how did Steig et al. calculate the AWS reconstruction? Since we don’t have either the satellite data or the exact path they followed using the RegEM Matlab files, we can’t say for sure how the results were obtained. However, using a little mathemagic, we can actually take the sequences apart and then calculate reconstructed […]