I’ll report on this issue in today’s post. I’ve been looking closely at Briffa et al 2013 over the past 10 days and unsurprisingly there is issue after issue. According to CRU, they’ve been working on this article for over seven years and, needless to say, it is impossible to fully observe the pea in only a few days, especially when the adjustments have become so baroque that the chronology style is most aptly described as East Anglia Rococo, making the weary reader long for the classic simplicity of earlier CRU illusions like the Briffa Bodge and Hide the Decline. But more on this on another occasion.

Briffa et al 2013 on Radial Deformation

Even though Briffa was the AR4 Lead Author responsible for assessing recent reconstructions, Briffa has more or less steered clear of bristlecones and radial deformation. However, one of the objectives of Briffa et al 2013 was to try to dismiss or discount the high medieval values of the Polar Urals series (see tag “urals”) when the additional data of the Polar Urals “Update” was included. Previously Briffa had just ignored the inconsistency between Polar Urals and Yamal, pretending that the issue had never occurred to them. This obtuseness has long been criticized at CA, where reconciliation of such inconsistencies has been recommended as one of the highest priorities for specialists.

B13 attributed the high medieval values (in part) to a bias arising from the inclusion of “root collar” samples:

The reanalysis of these data here show that the apparent evidence for high tree growth in medieval times in the Polar Urals region (circa 980-1040) was exaggerated due to inclusion of root-collar wood samples…We have shown here that that version of the Polar Urals TRW chronology (also combining larch and spruce data) is biased by the inclusion of multiple root-collar-derived sample data.

Briffa concluded that it was simply not “appropriate” to use radially dilated samples:

As these root collar samples appear highly variable in terms of cross-sectional dimensions, rather than being generally symmetric, it is not appropriate simply to process them with a separate (root-collar) RCS curve and it was considered necessary to remove these samples from the Polar Urals TRW RCS chronology despite the already low chronology replication (see SM4 description and Fig. PC02).

Passive voice exclusions and adjustments abound in B13 – a point that I will return to in another post. As an aside, the passive voice used to disappear the data reminds me of a famous phrase from the Vietnam War “It became necessary to destroy the town to save it” – a slogan that seems all too applicable to CRU’s approach to dendroclimatology.

Briffa’s interest in root collars arose from their examination of previously unreported metadata which showed that numerous medieval samples in the Polar Urals Update were taken from “root collars” rather than the more usual stems (because of more advanced rotting in the stems as opposed to the root collars). See in particular PU06 in SM4, which reproduces a letter to Schweingruber (presumably from Shiyatov). B13 summarized this as follows:

Closer examination of the details of the sample material reveals that the Polurula samples comprise a relatively large proportion of root-collar wood. The root-collar (or root crown) refers to the lower section of the tree bole (stem), generally near the soil surface, where the bole meets the upper parts of individual roots ..

Briffa observed that there were two related problems with samples taken from root collars: (1) increased width relative to samples taken higher up on the stem; (2) greater variability around the circumference, depending on the location of the major roots:

[root collars are] frequently associated with an expansion in the stem diameter at this point. It would be expected that ring width dimensions in such root-collar samples would be systematically larger than equivalent rings measured higher in the boles of the same trees. It is also the case that average ring dimensions in the root-collar vary greatly when measured at different positions around the circumference, according to the positions of the major roots. (91/9)

In section 6.1, Briffa re-iterated the issue in similar terms:

The root-collar samples have more variable (and generally larger) ring dimensions than regular stem samples, here sampled at varying levels above ground depending on the height of remnant tree boles.

And again in Supplementary Material 3, Briffa emphasized the variability around the circumference at the root collar:

The rings in the trunk of Siberian larch are generally concentric whereas the rings in the root-collar samples can vary considerably, being larger in the direction of a major root and smaller in directions between roots (Figure PU07). The allocation of material to individual roots will partly depend on mechanical strength requirements within the tree and growth can favour one root (direction) over several decades. Hence the dimensions of TRW measurements for one year taken in different directions from root-collar samples can be very variable…

These concerns seem entirely reasonable to me. Inhomogeneity between sample populations are a very serious problem in Briffa’s RCS methodology. B13 asserts that need to test subpopulations for homogeneity, but are completely and irredeemably at sea in actually carrying out statistical tests for homogeneity – procedures on which many statistical specialists could have assisted. Nonetheless, their concern over inhomogeneity between root collar and stem samples seems valid to me based on my own first examination of the data. However, they are obtusely blind to other inhomogeneities of equal or larger magnitude. One wonders whether inhomogeneities between sites are in fact so severe as to swamp whatever Briffa is trying to do. An issue for another day.

Figure PU07

B13 illustrated the supposed severity of radial asymmetry at Polar Urals in their Figure PU07 (Supp Mat 4) commenting as follows in its caption: “As sampling approaches the root the variation of ring-width measured along radii in different directions increases considerably.”

Original Caption: Figure PU07 Wood sample from the base of the stem of larch (Larix sibirica Ledeb.) from the Polar Urals. 22-43C 66°49.07′N 65°33.94′ E, 269 m.a.s.l., 1170-1412 CE. As sampling approaches the root the variation of ring-width measured along radii in different directions increases considerably. {Note the sample ID (22-43C) does not match any archived ID either in number or format.)

Strip Bark and Radial Deformation
By bristlecone standards, the radial deformation evidenced in figure PU07 is very slight indeed. Indeed, it is doubtful whether a single bristlecone chronology can survive the implicit standards of Briffa et al 2013.

The diagrams below show two examples (pages 32, 33) from Brunston of the extraordinary deformation in bristlecones – not even remotely “concentric”.

The effect of scarring as a proximate cause of radial deformation was neatly illustrated in a graphic below (ironically from the thread on which the first notice of Climategate occurred). The cross section is from an Engelmann spruce which was scarred (see bottom of graphic) by a glacier. In areas adjacent to the scar (dated to 1846), there was a huge growth pulse (see bottom right part of tree). Variability around the circumference of this tree is far more extreme than that shown in Briffa’s PU07. In addition, the growth enhancement is much larger than in PU07.

Figure 2. Deformation in engelmann spruce scarred by glacier in 1846. From presentation by Brian Luckman. See CA post ^.

At Climate Audit, strip bark has been a longstanding issue, but, since our work at Almagre, on the basis of mechanical deformation rather than the carbon dioxide fertilization hypothesized by Graybill, Lamarche and Idso (which may exist but as a much lesser effect.)

At Almagre, Pete Holzmann observed six-sigma differences in ring widths in cores drilled only a few inches apart on the same tree. See, for example, here and various presentations. The difference between adjacent cores was illustrated through the following diagram of Pete’s cores from Graybill Almagre tree 56:

Figure 1. Ring widths in two 2007 cores in Almagre tree 31 (Graybill 56). The difference in widths is due to radial deformation.

Pete’s theory – one that I endorse – was that the strip bark event caused or corresponded to mechanical deformation in the surviving part of the tree that was evident in the subsequent growth pulse. Curiously, in the Climategate emails, MBH coauthor Hughes privately expressed a similar thought to Briffa (though not in any public statements):

A further problem arises from the observation that radial increment may increase rather dramatically in the period after most of the bark dies back, but of course we don’t know when that was.

At Almagre, Pete observed that strip bark formation could be connected to bark being torn off by falling limbs due to snow weight, a hypothesis that might connect the notoriously severe winters in the US west in the 1840s to the apparent bristlecone growth pulse commencing in the 1850s.

As long ago as 2006, the NAS panel hadrecommended that strip bark data be “avoided” in temperature reconstructions, but this recommendation was flouted by Briffa in IPCC AR4 and subsequently by Mann (Mann et al 2008) and other specialists.

Commenter Salamano at realclimate, in one of the earliest comments, asked whether the strip bark problem was comparable to the root collar problem, but unfortunately did not receive a reply that was responsive to the important issue.

Nonetheless, it is an important issue.

I very much welcome the strong position taken by Briffa and coauthors against the use of radially deformed tree ring data. I look forward to the prompt application of these standards by Mann and others to strip bark chronologies. Given realclimate’s endorsement of B13, I presume that realclimate will urge that all reconstructions relying on bristlecones be recalled pending assessment of radial deformation and inhomogeneity according to B13 standards.

Ross has attracted an enviable representation from the econometric community. Invitations were widely extended to the climate community without the response that Ross had hoped for, though there will be some prominent attendees, including Carl Wunsch who will be giving a keynote address.

I’m giving a presentation on Friday on proxy inconsistency at a session chaired by Hu McCulloch. I am consistently amazed at how long it takes me to prepare a new presentation and this has been no exception.

Yamal has been a longstanding issue at Climate Audit. The new article appears to be their long awaited response to criticism from Climate Audit (though this criticism is not referred to anywhere in the aticle.)

In resisting FOI requests for their withheld 2006 Yamal-Urals regional chronology, CRU said that it was incomplete, as they were continuing to work on its development. However, they did undertake to disclose the 2006 regional chronology as part of the present publication. On my first reading, instead of living up to their undertaking to develop a regional chronology, CRU has instead provided reasons against using a regional chronology and do not present one in the paper – instead focussing on a variation of the original Yamal chronology.

In resisting the FOI, CRU said that production of the 2006 regional chronology would damage the reputation of CRU scientists. The 2006 version appears to be the “Urals raw” chronology illustrated in SM9 as Greater Urals (shown below), though it is not identified as such in my first reading. Readers can judge for themselves whether their foreboding was justified.

Figure 1. Compare to GU2 top panel.

Readers who are convinced by Briffa’s arguments against a regional chronology may well wonder whether, for example, the Avam-Taimyr regional chronology of Briffa et al 2008 would pass corresponding tests, since no similar analysis has ever appeared in Briffa articles in which he presented earlier regional chronologies. Or whether these tests only became of interest when the regional chronology went the wrong way.

CA readers will recall the original controversy in September 2009 over the Schweingruber Khadyta River series in Yamal, a controversy on which I’ll review in the present context on a subsequent occasion. Leaving nothing on the table, Briffa excluded the Khadyta River from the present reconstruction, pointing out that recent trees in this area had been growing poorly (thereby lowering the late 20th century uptick.) They state:

the site report (and statistical evidence) demonstrating the anomalous “signal” in the Khadytla data lead us to omit them from the new Yamal chronology constructed here (see SM5 for details)

There have been a number of posts, for example,  here, here and here at Lucia’s Blackboard or this one and that at WUWT which discuss the weak data gathering / data interpretation methodology and the truly incredible spin-one’s-head- around algorithm for generating a value of “97” which conveniently ignores a large proportion of the data.  My focus in this post will be to examine some of the other “quantifying” material.

Given the virtual absolute absence  of available data and statistics in this paper, this will not be that easy a task.  The authors have apparently bought into the Climate Science tradition of why should I make the data available if you will only use it to prove me wrong.  I should point out that I have spent much of my academic career doing just that and become reasonably adept at recognizing situations where work might  be shoddy.  This is due to my experiences with consulting on academic research projects, Ph.D. and Master’s theses work as well as outside the university.  When  interviewing the researchers about their project, I would tell them that I would attempt to find everything that was wrong with their planned research.  When it got to the point that this was no longer possible to do, I would be satisfied that the statistical aspects would be adequate for answering the questions that they wished to answer.

One of the items that caught my eye was Figure 2b:  Percentage of self-rated endorsement, rejection and no position papers.

This figure was discussed in the text of the paper:

Figure 2(a) shows the level of self-rated endorsement in terms of number of abstracts (the corollary to figure 1(a)) and figure 2(b) shows the percentage of abstracts (the corollary to figure 1(b)). The percentage of self-rated rejection papers decreased (simple linear regression trend  -0.25% ±0.18% yr^-1,  95% CI, R² = 0.28; p =0.01, figure 2(b)). The time series of self-rated no position and consensus endorsement papers both show no clear trend over time.

Figure 2(a) showed that the number of abstracts each year was increasing at a very high rate.  This implied that the variance of the percentages calculated for figure2(b) would change substantially from year to year.  Since a “simple linear regression” assumes homoscedasticity (i.e. equal variability) of the data from year to year, this would mean that the early years with very few abstracts would have an inordinately strong influence in the calculation of the parameter estimates as well as possibly distorting the interpretation of the significance of the results.

As well, the model for this regression (which for convenience we will do for a probability rather than a percentage) looks like:  P_k = α + βT_k where P_k is the probability that a paper in year T_k will belong in the category for which the regression is being done (e.g. “rejection papers”).  An observation for that year will look like p_k = n_k / N_k where n_k is the count of papers in years k and N_k is the total number of abstracts in that year.  One should note that p_k is assumed to have a binomial distribution with parameters N_k and P_k.  The full model  becomes  p_k = α + βT_k + ε_k where ε_k is a random variable with mean 0 and variance equal to P_k (1 – P_k ) / N_k .

The data needed to create Figure 2(b) was, you guessed it, unavailable, so I digitized the data in the graph using a simple R programme.  The large symbols used in the graph were not helpful to the process, but I managed to get what appeared to be a very reasonable replica of the various plots.  The replicated plot appears below near the end of the post.

Using creative rounding, I also turned these percentages into yearly counts for the categories each year for later use.  As a check, I calculated both the yearly count totals and the overall category totals.  The yearly counts matched the counts given in Table S2 of the Supplement except for the years 2009 and 2011 where they were each out be 1 in opposite directions.  The overall totals for the categories were

reject          endorse               nopos

40                 1321                    781   :  Estimated from Figure 2b

39                 1342                    761   :  Given in the paper

The differences of size 20 seem to be much too large to be explained as digitization errors (since a check on Photoshop of my reconstruction and the original graph showed an extremely close matchup of points, so this appears to be something that Mr. Cook could address by providing the “correct” values for the totals he gives in the paper.  However it is not the central issue here.

Using the digitized data, the “simple regression was carried out in R.  The comparable statistics to those above for the Reject group were (to all decimal places provided – round as you wish) :

-0.2501658% ± 0.1950723% yr^-1,  95% CI,                 R² = 0.2749;         p = 0.0147   Calculated in R

-0.25% ±0.18% yr^-1,  95% CI,                                         R² = 0.28;                   p =0.01  From the paper

The Cook bound for the confidence interval appears that it might have been calculated from a simple 2 times standard error calculation rather than formally using the appropriate t-value – something one should not do in a professional paper.

As I mentioned earlier, the “simple” regression method assumes equal variability from year to year which is clearly not the case.  In order to correct for this, we need to do a weighted regression where the optimal weights are equal to the inverse of the variance of the data value.  Thus the weights look like:

W_k = 1/( P_k (1 – P_k ) / N_k  ) =  N_k / ( P_k (1 – P_k ))

There is a slight problem here in that the P_k’s are not known.  One could substitute the sample proportions or better still, one can estimate the probabilities from the regression equation,  P_k = α + βT_k.   This is done by alternately calculating the regression and the weights until the process converges (iteratively reweighted least squares).  I wrote a short program for doing this and recalculated the above estimates:

-0.1524997% ± 0.1182535% yr^-1,  95% CI,                 R²  = 0.2772;        p = 0.0142

The magnitude of the slope is reduced by about 39% from the inappropriate unweighted regression.

I repeated this exercise for each of the other groups for which no results whatsoever were calculated because ostensibly  “the time series of self-rated no position and consensus endorsement papers both show no clear trend over time”.  One would think that for interest’s sake, such results would at least appear in the SI.

Endorse Group:

-0.16485% ± 0.7570545% yr^-1,  95% CI,                       R²  = 0.01081;     p = 0.654              Simple Regression

-0.4350717% ± 0.5419943% yr^-1,  95% CI,                 R²  = 0.1294;        p = 0.1093            Weighted Regression

No Position Group:

0.4149549% ± 0.7862663% yr^-1,  95% CI,                  R²  = 0.06034;     p = 0.283              Simple Regression

0.603483% ± 0.570961% yr^-1,  95% CI,                       R²  = 0.2048;        p = 0.0394            Weighted Regression

Using a more appropriate regression can produce substantial changes in the results.

So is this the end of the story?  Not quite.  Unbeknownst to some amateur statisticians, using regression to analyze this type of data is not recommended.  For example, if one were to use the Cook paper regression to “predict” the probability for the Reject group for any year from 2013 on, they would get a negative value ( possibly because retractions would exceed new publications???).   Methodology  termed Generalized Linear Models has been developed for exactly this type of situation.  In our case, we will use a slightly different model:

P_k = exp(α + βT_k) / (1 + exp(α + βT_k))

where exp(x) represents the exponential function e^x.  This particular model is also known as logistic regression.  Fitting is done using maximum likelihood techniques and interpretation of the estimated coefficients differs somewhat from the regression case.  Unlike linear regression the result is always between 0 and 1 with no negative probabilities.  Also, confidence intervals for probabilities are usually not symmetric about the estimated value.

Using the glm() function in R, the calculations were done   for all three groups:

Reject:

…………………………Estimate              Std. Error             z value             Pr(>|z|)

(Intercept)          144.97037            54.51333              2.659          0.00783

year                   -0.07427              0.02720                -2.731       0.00632

Mean annual rate of change from 1991 to 2011: -0.2007284

Endorse:

………………………..Estimate              Std. Error             z value          Pr(>|z|)

(Intercept)          38.788789            18.676384            2.077       0.0378

year                  -0.019095            0.009308              -2.051    0.0402

Mean annual rate of change from 1991 to 2011: -0.4384528

No Position:

…………………………Estimate              Std. Error             z value    Pr(>|z|)

(Intercept)          -55.417983          19.119586            -2.898    0.00375

year                   0.027343              0.009528              2.870     0.00411

Mean annual rate of change from 1991 to 2011: 0.60276

You will note that all three of the groups now seem to have statistically significant rates of change with the Endorse group showing a decrease of almost 9 percentage points over the 20 year period.

The plot below shows the distribution of the Reject group with associated regression lines and the fitted GLM.

All of the groups with associated GLM fits:

All of the R calculations and the data used are available by running the following lines in R (you may need to fix the quotes first):

dfile = “http://statpad.files.wordpress.com/2013/05/cookpost.doc “
download.file(dfile, “cookpost.R”, method = “auto”, quiet = FALSE, mode = “w”,cacheOK = TRUE)

Open the script cookpost.R .

http://blogs.nature.com/ofschemesandmemes/2013/05/02/npg-journal-club-how-has-earths-climate-changed-in-the-past-2000-years-npgjclub#comment-1503

Started at 11 am Eastern.

11:30. Open for questions. I have submitted the following:

Can you explain the decision to label the article as only a “Progress Article”, rather than a Research Article?

Nature’s definition of Progress Articles http://www.nature.com/ngeo/authors/content_types.html says that such articles are “commissioned by the editors” and associates them with “fields that might not yet be mature enough for review”. It also states that such articles do not include received and accepted dates and places more restrictive word and display limits than full Research Articles:

“When the discussion is focused on a developing field that might not yet be mature enough for review, a Progress article is more appropriate. Progress articles are up to 2,000 words in length, with up to 4 display items (figures, tables or boxes). References are limited to 50. Reviews and Progress articles are commissioned by the editors, but proposals including a short synopsis are welcome. Reviews and Progress articles are always peer-reviewed to ensure factual accuracy, appropriate citations and scholarly balance. They do not include received/accepted dates.”

Thousand-year paleoclimate reconstructions clearly do not qualify as a “developing field… not mature enough for review”. So why was this article classified as only a Progress Article?

Did Nature editors either commission the PAGES2K article or receive a short synopsis from the authors?

Given the above policy against received and accepted dates, why did Nature you include received and accepted such dates for the PAGES article?

Here is my surmise on the matter. The PAGES2K article presents eight different reconstructions using a variety of methods. Each individual reconstruction warranted separate peer review in specialist literature and it was impossible within the required time frame for peer reviewers to provide the peer review expected of a Research Article. As a way out of the review dilemma, one or more reviewers suggested that PAGES2K be published as a Progress Article, a recommendation that you adopted, even though the article did not fit within the definition. Can you comment on this surmise?

The links to the survey from SKS here is http://survey.gci.uq.edu.au/survey.php?c=1R9YT8YMZTWF and from Rabett hereis http://survey.gci.uq.edu.au/survey.php?c=II7WP4R4VRU7. More IDs are available at Lucia’s.

It is easy enough to access both blogs using hidemyass.com and then click on their link to the survey. In the survey, readers are asked to rate various abstracts according to their support for AGW. I urge readers to take as much care with the survey as the respondents to Lewandowsky’s Hoax , where Lewandowsky argued that fake responses should not be excluded.

The contaminated series is readily identified as an outlier through a simple inspection of the data. The evidence of contamination by recent agriculture in the specialist articles is completely unequivocal. This sort of mistake shouldn’t be that hard to spot even for real climate scientists.

.

Here is a plot of the last nine (of 22) Arctic sediment series. One of these series (top left – Igaliku) has the classic shape of the contaminated Finnish sediment series (often described as upside down Tiljander). Any proper data analyst plots data and inspects outliers, especially ones that overly contribute to the expected answer. The Igaliku series demands further inspection under routine data analysis.

Figure 1. Plot of last nine (of 22) Kaufman et al Arctic sediment series. The Igaliku proxy is total pollen accumulation.

The Igaliku series is plotted separately below. It is also available at a NOAA archive here , which actually contains one additional recent value plotted in red. The NOAA archive contains many other measurements: it is unclear why Kaufman selected pollen accumulation rate out of all the available measurements.

The resolution of the data set is only 56 years (coarser than the stated minimum of 50 years) and only has three values in the 20th century. The value in 1916 was lower than late medieval values, but had dramatically surged in the late part of the 20th century.

Figure 2. PAGES2K Igaliku series.

Igaliku is in Greenland and was the location of the Norse settlement founded by Erik the Red and is of archaeological interest. Sediment series from Lake Igaliku have been described in three specialist publications in 2012:

Massa et al, 2012. Journal of Paleolimnology, A multiproxy evaluation of Holocene environmental change from Lake Igaliku, South Greenland. (Not presently online). (Update: online here h/t Mosher. I’ve added a paragraph from this text referring to pollen accumulation.)

Massa et al 2012. QSR. A 2500 year record of natural and anthropogenic soil erosion in South Greenland. Online here.

Perren et al 2012, 2012. Holocene. A paleoecological perspective on 1450 years of human impacts from a lake in southern Greenland. Online here.

The three articles clearly demonstrate that the sediments are contaminated as climate proxies.

Igaliku has been re-settled in the 20th century and modern agricultural practices have been introduced. The specialist publications make it overwhelmingly clear that modern agriculture has resulted in dramatic changes to the sediments, rendering the recent portion of the Igaliku series unusable as a climate proxy. Here are some quotes from the original article.

The modern community consists of 60 permanent inhabitants and was founded in the late 1700s. Agricultural practices resumed in the 1920s, at the same time that the climate of southern Greenland reached its recent maximum (Box et al., 2009). Current sheep farming in the catchment is limited to one farm, established in the early 1960s, which has a barn for wintering sheep and summer hay production on a 30 ha field. A small ditch currently drains effluent from the barn into the nearby lake. The farm currently deploys 750-900 kg N fertilizer per year within the lake catchment to boost yields for winter fodder (Mikki Egede, personal communication, 2011)

A multiproxy sedimentary record from Lake Igaliku in southern Greenland documents 1450 years of human impacts on the landscape. Diatoms, scaled chrysophytes, and C and N geochemistry show perturbations consistent with recent agricultural activities (post- ad 1980), superimposed upon long-term environmental variability. While the response to Norse agriculture (~ ad 986-1450) is weak, the biological response to the last 30 years of modern sheep farming is marked, with drastic changes in diatom taxa, d 13 C and d 15 N isotopic ratios, and a sharp increase in scaled chrysophytes. Indeed, current conditions in the lake during the last 30 years are unprecedented in the context of the last 1450 years. The dominant driver for recent changes is likely an intensification of agricultural practices combined with warming summer temperatures. W

The PCA of diatom results show two major features: a major shift in lake ecology ~ ad 1980 as registered in the PCA axis 1… the rise in d15N in Igaliku is likely a result of the addition of fertilizers from manure and industrial sources, but some component of internal utilization of N, such as enhanced sediment denitrification, cannot be ruled out.

However, beginning in 1976, the method of farming shifted towards fodder production and higher yields at slaughter which introduced fertilizers
(250-300 kg/ha per yr) and effluent from winter sheep stables into the local landscape and lake (Figure 7: agricultural phase II; Greenland Agriculture Advisory Board, 2009). After 1976, sediments from Igaliku show a rise in planktonic diatoms ( Cyclotella stelligera, Fragilaria tenera ), as well as chrysophyte scales, d15N, and N, reflecting increased nutrient additions and the beginning of industrialized agriculture.

The digging of drainage ditches for hayfields caused a dramatic increase in MAR, which reached unprecedented values. The use of nitrogen fertilizers on thesefields (200–250 kg ha -1 yr -1of N, Miki Egede pers.commun.) have outpaced the natural buffering capacity of Lake Igaliku, resulting in a sharp rise in the mesotrophic diatom,

This is precisely the same sort of contamination that affected the Korttajarvi sediments in Finland – for which, Kaufman, Mann and others were rightly criticized at Climate Audit. Kaufman conceded that the prior criticism was justified by issuing a corrigendum to Kaufman et al 2009 (but conspicuously failed to acknowledge Climate Audit or myself by name). It’s ludicrous that Kaufman has made an identical error with a different site. And that peer review at major journal was unequal to the identification of an error that Kaufman’s made in the past.

Now it is not evident to me that Kaufman’s varvology lends itself to multiproxy sausage-makers in any event. Varve compaction was not addressed in Kaufman et al 2013 and has the potential for a very serious bias. Nor is there any direct physical connection between temperature and varve thicknesses. The traditional interpretation of varves requires presence of a nearby ice cap and thin varves have been interpreted as evidence of warmth and thick varves as evidence of cold (Miller et al 2012) – the exact opposite of Kaufman. Until such issues are resolved, varve thickness data is unusable for temperature reconstructions that are destined for policy-maker consumption.

The network was unusable in the first place. However, the unusability is made much more evident when the authors and peer reviewers are once again unequal to the small task of separating out contaminated data.

Does this sort of error “matter” to the reconstruction? It’s hard to say.

It did in the case of the no-dendro reconstruction of Mann et al 2008, though it was never formally retracted. In that case, Mann toughed it out and continued to use the contaminated no-dendro reconstruction of Mann et al 2008 even after conceding it did not validate prior to AD1500 without the contaminated Tiljander data: see 2012 RC here; also cited in the EPA response to the Petition for Reconsideration). On the other hand, the Kaufman et al 2009 reconstruction was able to survive the correction of contaminated data.

While critics will be quick to say that it is my responsibility to show the impact of the error, I view today’s post as part of extended peer review: no author will tell a peer reviewer that it was their job to figure out the impact of using contaminated data. It’s the responsibility of the author to correct contaminated data, not the responsibility of a reviewer, either at the journal stage or in the present “extended” review. I presume that Kaufman will do so, once he has satisfied himself that there is a problem.

In the present case, it may well be that varve compaction – which impacts multiple series – is a more serious problem that a single contaminated series. But one really wonders at the quality of work when such gross errors are made.

Update: 6 pm. The Journal of Paleolimnology article which Mosher located also stated in respect of pollen accumulation:

Despite the possible influence of land use, pollen accumulation appears to document climatic changes of the last millennia nonetheless. PAR reached minimum values during the Little Ice Age from 1500 to 1920 AD, consistent with maximum glacial re-advance at Qipisarqo (Kaplan et al. 2002) and elsewhere in south Greenland (Weidick et al. 2004; Larsen et al. 2011). It is also coeval with high rates of isostatically driven transgression, which caused the inundation of a Norse graveyard at Herjolfsnæs (Mikkelsen et al. 2008). The sharp increase of Salix/ Betula pollen accumulation rate after 1920 AD (Fig. 6) suggests a rapid warming, which reversed the Neoglacial cooling trend similar to other locations in the Arctic (Kaufman et al. 2009).

Nick Stokes has argued in comments below that this is sufficient to qualify the contaminated sediments as a climate proxy. I disagree. The sediments are clearly contaminated by human activity. Can pollen accumulation within contaminated sediments be separated as an indicator? I’ve got a better idea: the Arctic is a big place. Don’t use contaminated sediments.

Postscript: Here is the longer Igaliku pollen accumulation series as plotted from the data at NOAA. Values are low in the mid-Holocene despite other evidence of mid-Holocene warmth. My interpretation of this is that glacier retreat took a long time in this area (think LIFO accounting) and had not retreated sufficiently to permit pollen accumulation until rather late in the Holocene.

Update 2: Here is a plot comparing pollen accumulation rates to organics accumulation rate. (Organics accumulation is the Korttajarvi proxy.)

Update 3:

Igaliku age model calculated by Paul Dennis. Red- age model of authors.

Update 4: here is a plot of pollen sum at Unit Lake, Manitoba, one of the data sets published in the Kaufman 2012 JOPL issue.


Update 5: here is a plot of pollen accumulation vs mineral matter accumulation in the same interval for the 7 pollen measurements since 1750. The pollen intervals (1 cm) do not exactly overlap the mineral intervals (0.5 cm) and so weighted averages were taken. There is an obvious relationship between erosion (indicated by mineral matter accumulation) and pollen accumulation. Massa could just as easily “suggested” that erosion was a proxy for temperature.

PAGES2k expanded Kaufman’s 2009 network to 22 series, adding a number of new series, including Hvitavatn, Iceland, where Kaufman once again has used the data upside down to the interpretation of Gifford Miller, the original author and a very eminent paleoclimatologist. Miller’s report on Hvitavatn was previously discussed at CA here.

Here is a plot of the first eight Arctic sediments (log basis). WIthin these eight series, the series on the right second from the top (Larsen 2011) is the only one showing a dramatic increase in 20th century values relative to medieval values.

Figure 1. First eight Arctic sediment proxies(log scale). Hvitavatn is the “Larsen2010″ series on right side second from top.

Larsen et al 2011 is a report on Hvitavatn, a dataset that was also discussed at length in Miller et al 2012, an interesting article previously discussed at CA here.

In that blog post, I presented the graphic of Hvitavatn varve thicknesses from Miller et al, as shown below. Miller interpreted the increased varve thicknesses at Hvitavatn after 1275, together with radiocarbon information on kill dates at Baffin Island, as demonstrating the development of LIA cold and the narrow MWP varves as evidence that ice caps were not in close proximity. Miller stated:

Here we present precisely dated records of ice-cap growth from Arctic Canada and Iceland showing that LIA summer cold and ice growth began abruptly between 1275 and 1300 AD, followed by a substantial intensification 1430–1455 AD.

From Miller et al 2012 Figure 2C-D. (c) Ice cap expansion dates based on a composite of 94 Arctic Canada calibrated 14C PDFs. (d) 30-year running mean varve thickness in Hvítárvatn sediment core HVT03-2 [Larsen et al., 2011].

Miller et al discussed varve changes at Hvitavatn at length, including the following:

Thus, supra-decadal changes in Hvítárvatn varve thickness track the intensity of Langjökull erosion, and serve as a proxy for ice-cap size [ Larsen et al., 2011]. The response time of Langjökull outlet glaciers to abrupt summer cooling is approximately a decade and the estimated ice-cap equilibration time is 100 years (H. Björnsson, unpublished data, 1997- 2011).

Consequently, Langjökull outlet glaciers will begin to advance within a decade following abrupt summer cooling, although the ice cap will not attain its new equilibrium dimensions for a century. We therefore expect that times of abrupt snowline lowering derived from the Baffin Island kill dates should correspond with the onset of multidecadal trends of increasing varve thickness in Hvítárvatn. To test this prediction we analyzed replicate varved sediment cores from Hvítárvatn, where the past 1200 years is contained in the upper 8 m of the sediment fill. The varve chronology since 800 AD is constrained by seven historically dated tephras, providing 6 year temporal precision [ Larsen et al. , 2011]. The 30-year running mean varve thickness integrates the interannual to decadal variations in hydrologic efficiency of the delivery systems and tracks the evolution of Langjökull’s growth and decay in response to summer temperature changes over the past 1200 years..

From both the Canadian evidence (many sites became ice-covered in the late 13th Century and remained so until the past decade) and Icelandic evidence (consistently thick varves following the late 13th Century), we can conclude that multidecadal average summer temperatures never returned to those of Medieval times until the 20th Century…

The expansion and subsequent retreat of Langjökull recorded by a peak in varve thickness between 850 and 950 AD also coincides with ice-cap growth in Arctic Canada. This is followed by two centuries of unusually thin varves between 950 and 1170 AD, indicative of reduced glacial erosion in response to increased summer warmth. This suggests that the lack of ice-kill dates on Baffin Island during the same interval is the result of ice-melt rather than extended cold, a finding that could not be determined from the dated vegetation record alone.

Turning now to the PAGES use of this record. The PAGES2K SI says that its proxy records must “exhibit a documented temperature signal” and be “published in peer reviewed literature as a proxy for temperature”:

The proxy records selected by the Arctic2k group for the Arctic continental-scale temperature reconstruction (Fig. S7) meet the following criteria: (1) situated north of 60°N, (2) extend back in time to at least 1500 CE, (3) have an average sample resolution of no coarser than 50 years, (4) include at least one chronological reference point every 500 years, (5) exhibit a documented temperature signal, and (6) are published in peer-reviewed literature as a proxy for temperature, although not necessarily calibrated to temperature (i. e., some records provide only a relative measure of temperature with unknown transformations between the proxy measurement and temperature).

Unfortunately, they did not also require that the use of the record be in the same orientation as the published literature and, in the case of Hvitavatn, the SI states that they used the record in a positive orientation i.e. opposite to the interpretation of Miller et al 2012.

PAGES2K also used a number of Baffin Island varve series. In my previous post on Miller et al 2012, I observed that the varve data on Baffin Island lent itself to the interpretation that Miller had placed on the Hvitavatn, Iceland data – i.e. thick varves indicating glacier proximity, an interpretation of varves that is consistent with interpretations of Ice Age recession.

BTW I strongly commend Kaufman for actually reporting the orientation of each proxy. Previous authors e.g. Neukom et al 2011 have failed to do so, enabling Neukon’s upside down use of Quelccaya data to remain undetected until I noticed it the other day because of PAGES2K disclosure.

Readers should not conclude that Miller has argued that MWP warmth exceeded modern warmth. Miller has argued that recent Arctic glacier melt is exposing sites that were ice covered through the MWP:

The 24 Canadian sites that became ice-covered 800 – 900 AD (Table S4) and did not melt again until the past decade demonstrate that multi-decadal
average summer temperatures in Arctic Canada now exceed those of Medieval times.

This is a line of argument that, in my opinion, might well be used to argue that modern warmth has surpassed MWP warmth. Assembling such facts would be far more persuasive to me than multiproxy varvology with upside down data.

The appearance of Steig’s bladeless hockey stick was apparently so horrifying that he dared not show it in the RC post. However, I believe that CA readers are made of sterner stuff and will be able to withstand the sight of even a bladeless hockey stick, which is shown below.

Figure 1. Steig et al 2013 Figure 3. Original caption: Decade-average d18O from the WAIS Divide ice core for the past 2,000 years. Grey shading shows 2 s.d. about the decadal mean, based on the upper 100 years of the multi-core d18O composite, providing an estimate of the 95% confidence range. The dashed line shows the 97.5 percentile value relative to the average linear trend.

Steig described d18O values in “recent decades” as “highly unusual”:

Our results thus show that, indeed, recent decades in West Antarctica, which have been characterized by very rapid warming, and very rapid loss of ice from the West Antarctic Ice Sheet, are highly unusual.

Steig also asserted that there was a “strong trend” in O18 values in the past 50 years, which was, according to Steig, “largely driven” by high values in the closing portion of the series. Here’s his exact language:

Our results show that the strong trend in δ18O in West Antarctica in the last 50 years is largely driven by anomalously high δ18O in the most recent two decades, particularly in the 1990s (less so the 2000s).

It seems odd to say that the supposed trend was “largely driven” by higher values in the closest portion: how would one get a trend without higher closing values. For comparison, here is a detail of the WAIS d18O record (plotted from PAGES2K data) for the past century. Values in the 1990s were locally elevated, but values in the 1970s were the lowest in the entire record, contradicting Steig’s claim about “recent decades“. Nor is the “trend” since 1950 even statistically significant. Indeed, the values in the 1990s appear more like a fluctuation, as opposed to a “trend” (let alone a “strong trend”), particularly given the subsequent downtick in the 2000s. Nor is this data set is one that any reasonable person would compare to a Hockey Stick.

Figure 2. Modern portion of WAIS d18O.

Even with the most liberal allowance for imprecise language and lack of statistical acumen on the part of “real climate” scientists,

Considering decadal averages (as Steig did), the 1990s were far from being “highly unusual”. They were at the 77th percentile within the data set: slightly elevated but not “highly unusual”. The average of values to date in the 2000s were at the 15th percentile!

Although Steig conceded at RC that the recent results can’t be considered “unprecedented”, he purported to deduce from this data that similar results occurred only once per century and were “probably” a harbinger of ice sheet collapse in west Antarctica:

What we’ve observed is unusual, but it is also dominated by decadal climate variability, and can’t be considered “unprecendeted”…
Looking at the very long term record from the WAIS Divide ice core, it appears that similar conditions could have occurred about once per century over the last 2000 years. Hence our answer to the question, “are the observations of the last few decades a harbinger of continued ice sheet collapse in West Antarctica?”, is tentative: “Probably”.

While the 1990s did indeed have the highest value in their century, as noted above, they were far from being “highly unusual” in the context of the data itself. Even within the past three centuries, the slight downward trend persisted and the 1990s appear more as a fluctuation than a “trend”.

From this very unpromising bladeless hockey stick, Steig was hard-pressed to support alarm, but did his best. Steig even claimed that the long record showed that recent values were “anomalous” over the past two millennia:

Analysis of the long record at WAIS Divide shows that d18O in West Antarctica is anomalous not only with respect to the past two centuries, but also with respect to the past two millennia.

… Decadal-average 18O values comparable to the 1990s in the WAIS Divide record are reached on only four occasions in the past 1,000 years (Fig. 3). Before 1,000 years ago, modern decadal-average 18O values are reached more frequently, but these are superimposed on a declining trend attributable to the influence of Milankovitch orbital forcing and ice flow. Assuming that the decadal variability is independent of orbital forcing, we calculate d18O anomalies relative to the long-term trend (dashed line in Fig. 3). Anomalies in d18O similar to those of the 1990s occur just twice in the past 2,000 years; assuming sampling error estimated from the multiple shorter records, comparably elevated 18O values were reached about 1% of the time.

Watch the pea here.

Steig is not talking about the d18O values that people are actually interested in, but in values after subtraction of the long-term trend. These are plotted below. The residual for the 1990s is indeed relatively large, but there is nothing to suggest that it is “significant” – particularly allowing for the unexceptional nature of the residual for the 2000s.

Figure 3. Trend residuals.

Steig’s RC post glossed over the long-term decline in d18O values. A commenter challenged him as follows:

1. Would it be correct to say your δ18O data indicates a decline over the past 2,000 years, thus the WAIS has cooled over the past 2,000 years?
2. The abstract says, “However, δ18O anomalies comparable to those of recent decades occur about 1% of the time over the past 2,000 years.” This appears to be only in relation to the declining δ18O trendline. Would it be correct to say that if compared to the mean δ18O of the past 2,000 years, the anomalies of recent decades would actually still be negative in comparison to the mean?

Steig responded testily that pointing out this relevant point was “tiresome” when the answers were given in the (paywalled) paper that few readers would have purchased:

[Response: Yes and yes. It is a bit tiresome answering questions whose answers are given unambiguously in the paper. Read it, please! The mean cooling is consistent with Milankovitch forcing, and is not particularly relevant to the question of atmospheric circulation and glacier anomalies.--eric]

The article itself made an armwaving reference to Milankovitch forcing as follows:

Before 1,000 years ago, modern decadal-average 18O values are reached more frequently, but these are superimposed on a declining trend attributable to the influence of Milankovitch orbital forcing and ice flow [13, Neumann et al 2008 JGR].

Unfortunately the citation did not establish that the declining trend was “attributable” to Milankovitch forcing and ice flow. Indeed, it contained no reference whatever to Milankovitch forcing.

Steig’s RC article also discussed a dataset from the Antarctic Peninsula which showed increased melt in the 20th century. Steig rhetorically asked

Why the difference between the Peninsula and the WAIS?
After all, both locations are warming at about the same rate.[linking to Steig et al 2009]

Of course, Steig et al 2009 is no authority at all for the proposition that the Peninsula and West Antarctica are warming at about the same rate. As discussed in blog posts at the time and in O’Donnell et al 2010, Steig et al 2009 had simply spread Peninsula warming to West Antarctica through faulty math. But even when confronted by his own adverse d18O data, Steig refused to raise the possibility of the correctness of the criticisms of his flawed math.

There are a few points on which I agree with Steig. Steig argued that there was merit in single proxy rather than multiproxy studies:

Amidst the continuous chatter in the blogosphere about the strengths and limitations about “multiproxy” studies, these studies may be a refreshing return to simpler methods relying on just one type of “proxy”: data from ice cores. While ice core data aren’t perfect proxies of climate, they come pretty close, and aren’t subject to the same kinds of uncertainties that are unavoidable in biological proxies like tree rings.

I substantially agree with Steig that there would be great benefit of focusing on single classes of proxy. Steig also pointed out the potential benefit of focusing on d18O data without translating it into temperature- a tactic that seems eminently sensible to me and which I’ve employed from time to time at Climate Audit. Reconstructions of past d18O seems like a very worthy enterprise to me.

Steig claimed that his paper placed recent changes in a “longer-term” context:

Both our paper and that of Abram et al. add to our understanding of recent climate, glacier, and ice sheet changes in Antarctica by placing them into a longer-term context.

Again, I agree that Steig et al 2013 places recent changes in a longer-term context though not necessarily with the conclusion that Steig desires.

Forest et al. 2006 (F06), here, was a high profile observationally-constrained Bayesian study that estimated equilibrium climate sensitivity (Seq) simultaneously with two other key climate system parameters / properties, ocean effective vertical diffusivity (Kv) and aerosol forcing (Faer). Both F06 and its predecessor Forest 2002 had their climate sensitivity PDFs featured in Figure 9.20 of the AR4 WG1 report. I started investigating F06 in 2011, with a view to using its data to derive estimated climate system parameter PDFs using an objective Bayesian method.  That work eventually led to my paper ‘An objective Bayesian, improved approach for applying optimal fingerprint techniques to estimate climate sensitivity’, recently published in Early online release form by Journal of Climate, here.

Readers may recall that I found some basic statistical errors in the F06 code, about which I wrote a detailed article at Climate Audit, here. But those errors could not have affected any unrelated studies.  In this post, I want to focus on an error I have discovered in F06 that is perhaps of wider interest. The error has a source that will probably be familiar to CA readers – failure to check that data is zero mean before undertaking principal components analysis (PCA) / singular value decomposition (SVD).

The Forest 2006 method
First, a recap of how F06 works. It uses three ‘diagnostics’  (groups of variables whose observed values are compared to model-simulations): surface temperature averages from four latitude zones for each of the five decades comprised in 1946–1995; deep ocean 0–3000 m global temperature trend over 1957–1993; and upper air temperature changes from 1961–80 to 1986–95 at eight pressure levels for each 5-degree latitude band (8 bands being without data). AOGCM unforced long control run data is used to estimate natural, internal variability in the diagnostic variables. The MIT 2D climate model, which has adjustable parameters calibrated in terms of  Seq , Kv and Faer, was run several hundred times at different settings of those parameters, producing sets of model-simulated temperature changes on a coarse, incomplete, grid of the three climate system parameters.

A standard optimal fingerprint method, as used in most detection and attribution (D&A) studies to deduce anthropogenic influence on the climate, is employed in F06. The differences between changes in model-simulated and observed temperatures are ‘whitened’, with the intention of making them independent and all of unit variance. Then an error sum-of-squares, r2, is calculated from the whitened diagnostic variable differences and a likelihood function is computed from r2, on the basis of an appropriate F-distribution. The idea is that, the lower r2 is, the higher the likelihood that the model settings of Seq, Kv and Faer correspond to their true values. Either the values of the model-simulated diagnostic variables are first interpolated to a fine regular 3D grid (my approach) or the r2 values are so interpolated (the F06 approach). A joint posterior PDF for Seq, Kv and Faer is then computed, using Bayes’ rule, from the multiplicatively-combined values of the likelihoods from all three diagnostics and a prior distribution for the parameters. Finally, a marginal posterior PDF is computed for each parameter by integrating out (averaging over) the other two parameters.

The whitening process involves a truncated inversion of the estimated (sample) control-run data covariance matrix. First an eigendecomposition of that covariance matrix is performed. A regularized inverse transpose square root of that matrix is obtained as the product of the eigenvectors and the reciprocal square roots of the corresponding eigenvalues, only the first k eigenvector–eigenvalue pairs being used. The raw model-simulation – observation differences are then multiplied by that covariance matrix inverse square root to give the whitened differences. As many readers will know, by setting the number of retained eigenfunctions or EOFs (eigenvector-patterns), k affects how much detail is retained upon the inversion of the covariance matrix. The higher the truncation parameter k, the more detail is retained and the better is discrimination between differing values of Seq, Kv and Faer. However, if k is too high then the likelihood values will be heavily affected by noise and potentially very unrealistic. There is a standard test, detailed in Allen and Tett 1999 (AT99), here, that can be used to guard against k being too high.

The Forest 2006 upper air diagnostic: effect of EOF truncation and mass weighting choices
My concern in this article is with the F06 upper air (ua) diagnostic. F06 weighted the upper air diagnostic variables by the mass of air attributable to each, which is proportional to the cosine of its mean latitude multiplied by the pressure band allocated to the relevant pressure level. The weighting used only affects the EOF patterns; without EOF truncation it would have no effect. It seems reasonable that each pressure level’s pressure band should be treated as extending halfway towards the adjacent pressure levels, and to surface pressure (~1000 hPa) at the bottom. But where to treat the top end of the pressure band attributable to the highest, 50 hPa pressure level, as being is less clear. One choice is halfway towards the top of the atmosphere (0 hPa). Another is halfway towards 30 hPa, on the grounds that data for the 30 hPa level exists – although it was excluded from the main observational dataset due to excessive missing data. The F06 weighting was halfway towards 30 hPa. The weighting difference is minor: 4.0% for the 50 hPa layer on the F06 weighting, 5.6% on the alternative weighting.

However, it turns out that the fourteenth eigenvector – and hence the shape of the likelihood surface, given the F06 choice of kua = 14 –  is highly sensitive to which of these two mass-weighting schemes is applied to the diagnostic variables, as is the result from the AT99 test.  Whatever the physical merits of the two 50 hPa weighting bases, the F06 choice appears to be an inferior one from the point of view of stability of inference. It results, at kua = 14, in failure of  the recommended, stricter, version of the AT99 test, and a likelihood surface that is completely different from that when kua = 14 and the alternative weighting choice is made (which well satisfies the AT99 test). Moreover, if  kua is reduced to 12 then whichever weighting choice is made the AT99 test is satisfied and the likelihood surface is similar to that at kua = 14 when using the higher alternative, non-F06, 50 hPa level weighting.

AT99 test results
The below graph plots the AT99 test values – the ratio of the number of degrees of freedom in the fit to the r2 value at the best fit point, r2min, for the two 50 hPa weightings. To be cautious, the value should lie within the area bounded by the inner, dotted black lines (which are the 5% and 95% points of the relevant chi-squared distribution). The nearer it is to the unity, the better the statistical model is satisfied.

Using the F06 50 hPa level weighting, at kua = 13 the AT99 test is satisfied (although less well than at kua = 12), and the likelihood surface is more similar to that – using the same weighting – at kua = 12 than to that at kua = 14.

Upper air diagnostic likelihood surfaces
The following plots show what the upper air diagnostic likelihood consistency surface looks like in {Seq, Kv} space using the F06 50 hPa level weighting, at successively kua = 12, kua = 13 and kua = 14. Faer  has been integrated out, weighted by its marginal PDF as inferred from all diagnostics. The surface is for the CDF, not the PDF. It shows how probable it is that the upper air diagnostic r2 value at each {Seq, Kv} point could have arisen by chance, given the estimated noise covariance matrix. Note that the orientation of the axes is non-standard.

Notice that the combination of high climate sensitivity but fairly low ocean diffusivity is effectively ruled out by the kua = 12 likelihood surface, but not by the kua = 14 surface nor (except somewhere below sqrt(Kv) = 2) by the kua = 13 surface. It is the combination of high climate sensitivity but moderately low ocean diffusivity that the F06 surface and deep ocean diagnostics, acting together, have difficulty well-constraining. So the failure of the upper air diagnostic to do so either fattens the upper tail of the climate sensitivity PDF.

Why didn’t the Forest 2006 upper air r2 values reflect kua = 14, as used?
I couldn’t understand why, although F06 used kua = 14, the pattern of its computed r2 values, and hence the likelihood surface, were quite different from those that I computed in R from the same data. Why should the F06 combination of kua = 14 and 50 hPa level weighting produce one answer when I computed the r2 values in R, and a completely different one when computed by F06′s code? Compounding the mystery, the values produced by the F06 code seemed closely related to the kua = 13 case.

I could explain why each F06 r2 would only be only 80% of its expected value, because the code F06 used was designed for a different situation and it divides all the r2 values by 1.25. The same unhelpful division by 1.25 arises in F06′s computation of the r2 values for its surface diagnostic.  But even after adjusting for that, there were large discrepancies in the upper air r2 values. I thought at first that it might be something to do with IDL, the rather impenetrable language in which all the F06 code is written, having vector and array indexing subscripts that start at 0 rather than, as in R, at 1. But the correct explanation was much more interesting.

A missing data mask is generated as part of the processing of the observational data, based on a required minimum proportion of data being extant. That mask may have been used to mark what points should be treated as missing in the (initially complete) control data, when processing it to give changes from the mean of each twenty year period to the mean of a ten year period starting 25 years later. In any event, the values of the 80–85°N latitude band, 150 and 200 hPa level diagnostic variables, along with the variables for a lot of other locations, are marked as missing in the processed control data temperature changes matrix, by being given an undefined data marker value of ‑32,768°C. Such use of an undefined data marker value is common practice for external data, although not being an IDL acolyte I was initially uncertain why it was employed within IDL. It turns out that, when the code was written, IDL did not have a NaN value available.

Rogue undefined data maker values
Variables in the processed control data are then selected using a missing values mask derived from the actual processed observational data, which should eliminate all the control data variables with ‘undefined data’ marker values . Unfortunately, it turns out that the two 80–85°N latitude band control data variables mentioned, unlike all the other control data variables marked as missing by having ‑32,768°C values, aren’t in fact marked as missing in the processed observational data. So they get selected from the control data along with the valid data. I think that the reason why those points aren’t marked as missing in the observational data could possibly be linked to what looks to me like a simple coding error in a function called ‘vltmean’, but I’m not sure.  (All the relevant data and IDL code can be downloaded as large (2 GB) file archive GRL06_reproduce.tgz here.)

So, the result is that the ‑32,768°C control data marker values for the 80–85°N latitude band, 150 and 200 hPa pressure levels get multiplied by the cosine of 82.5° (the mid-point of the latitude band) and then by pressure-level weighting factors of respectively 0.05 and 0.075, to give those variables weighted values of ‑213.854°C and ‑320.781°C for all rows of the final, weighted, control data matrix.  Here’s an extract, at the point it is used to compute the whitening transformation, from the first row of the weighted control data matrix, CT1WGT, resulting from running the F06 IDL code, with the rogue data highlighted:

-0.0013 -213.8540   0.0127    0.0094     0.0051     0.0047     0.0056     0.0024     0.0005          0.0011
0.0038       0.0115     0.0125     0.0058     0.0060    0.0046      0.0030    0.0016     0.0000        -0.0002
0.0036       0.0077    0.0069     0.0001   -0.0047    -0.0060    -0.0067    -0.0050    -0.0030    -320.7810

I should say that I have been able to find this out thanks to considerable help from Jonathan Jones, who has run various modified versions of the F06 IDL code for me. These output relevant intermediate data vectors and matrices as text files that I can read into R and then manipulate.

Why does the rogue data have any effect?
Now, the control data contamination with missing data marker values doesn’t look good, but why should it have any effect? A constant value in all rows of a column of a matrix gives rise to zero entries in its covariance matrix, and a corresponding eigenvalue of zero, which – since the eigenvalues are ordered from largest to smallest – will result in that eigenfunction being excluded upon truncation.

But that didn’t happen. The reason is as follows. F06 used existing Detection and Attribution (D&A) code in order to carry out the whitening transformation – an IDL program module called ‘detect’.  That appears to be a predecessor of the standard module ‘gendetect’, Version 2.1, available at The Optimal Detection Package Webpage, which I think will have been used for a good number of published Detection and Attribution studies. Now, neither detect nor gendetect v2.1 actually carries out an eigendecomposition of the weighted control data covariance matrix.  Instead, they both compute the SVD of the weighted control data matrix itself. If all the columns of that matrix had been centred, then the SVD eigenvalues would be the square roots of the weighted control data sample covariance matrix eigenvalues, and the right singular vectors of the SVD decomposition would match that covariance matrix’s eigenvectors.

However, although all the other control data columns are pretty well centred (their means are within ± 10% or so of their – small – standard deviations), the two columns corresponding to the constant rogue data values are nothing like zero-mean. Therefore, the first eigenfunction  almost entirely represents a huge constant value for those two variables, and has an enormous eigenvalue. The various mean values of other variables, tiny by comparison, will also be represented in the first eigenfunction. The reciprocal of the first eigenvalue is virtually zero, so that eigenfunction contributes nothing to the r2 values. The most important non-constant pattern, which would have been the first eigenfunction of the covariance matrix decomposition, thus becomes the second SVD eigenfunction. There is virtually perfect equivalence between the eigenvalues and eigenvectors, and thus the whitening factors derived from them, of the SVD EOFs 2–14 and (after taking the square root of the eigenvalues) those of the covariance matrix eigendecomposition EOFs 1–13. So, although F06 ostensibly uses kua=14, it effectively uses kua =13. Mystery solved!

Concluding thoughts
It’s not clear to me whether, in F06′s case, the combination of arbitrary marker values incorrectly getting through the mask and then dominating the SVD had any further effects. But it is certainly rather worrying that this could happen. Is it possible that, without centring, the means of a control data matrix could give rise to a rogue EOF that did affect the r2 values, or otherwise materially distort the EOFs, in the absence of any masking error? If that did occur, might the results of some D&A studies be unreliable? Very probably not in either case. However, it does show how careful one needs to be with coding, and importance of making code as well as data available for investigation.

There was actually a good reason for use of the SVD function (from the related PV-wave language, not from IDL itself) rather than the IDL PCA function, which despite its name does actually compute the covariance matrix. Temperature changes in an unforced control run have a zero mean expectation, provided drifting runs are excluded. Therefore, deducting the sample mean is likely to result in a less accurate estimate of the control data covariance matrix than not doing so (by using the SVD eigendecomposition, or otherwise). However, the downside is that if undefined data marker values are used, and something goes wrong with the masking, the eigendecomposition will be heavily impacted. Version 3.1 of gendetect v3.1 does use the PCA function, so the comments in this post are not of any possible relevance to recent studies that use the v3.1 Optimal Detection Package code. However, I suspect that Version 2.1 is still in use by some researchers.

So, the lesson I draw is that use within computer code of large numbers as marker values for undefined data is risky, and that an SVD, rather than a eigendecomposition of the covariance matrix, should only be used to obtain eigenvectors and eigenvalues if great care is taken to ensure that all variables are in fact zero mean (in population, expectation terms).

Finally, readers may like to note that I had a recent post at Climate Etc, here, about another data misprocessing issue concerning the F06 upper air data. That misprocessing appears to account for the strange extended shoulder between 4°C and 6°C  in F06′s climate sensitivity PDF, shown below.

Nic Lewis