There has been some recent discussion of the Briffa bodge – an early technique to hide the decline. I had drafted a post on the topic and its handling by the Muir Russell “inquiry” in early July 2010, but did not publish the post at the time. In today’s post, I’ve slightly updated my July 2010 draft.
The term “bodge” was used for the first time in a comment (not a post) on November 8, 2009 by me here less than two weeks before Climategate). I had noticed the term “Briffa bodge” in a preprint of Briffa and Melvin
2008 2011 (see here), where it was used to describe a “very artificial correction” to Briffa’s widely used Tornetrask chronology as follows:
Briffa et al. (1992) ‘corrected’ this apparent anomaly by fitting a line through the residuals of actual minus estimated ring widths, derived from a regression using the density data over the period 501–1750 as the predictor variable, and then removing the recent apparent decline in the density chronology by adding the fitted straight line values (with the sign reversed) to the chronology data for 1750–1980. This ‘correction’ has been termed the ‘Briffa bodge’ (Stahle, personal communication)!
Bodging of the Tornetrask chronology had been discussed in much earlier CA posts – e.g. in March 2005 here and again here.
The term “bodge” also occurs in Climategate correspondence, as pointed out by Jeff Id on December 1, 2009 here.
In July 1999, Vaganov et al (Nature 1999) had attempted to explain the divergence problem in terms of later snowfall (an explanation that would seem to require caution in respect to the interpretation of earlier periods.) On July 14, 1999, Ed Cook wrote Briffa as follows:
What is your take on the Vagonov et al. paper concerning the influence of snowfall and melt timing on tree growth in Siberia? Frankly, I can’t believe it was published as is. It is amazinglly thin on details. Isn’t Sob the same site as your Polar Urals site? If so, why is the Sob response window so radically shorter then the ones you identified in your Nature paper for both density and ring width? I notice that they used Berezovo instead of Salekhard, which is much closer according to the map. Is that
because daily data were only available for the Berezovo? Also, there is no evidence for a decline or loss of temperature response in your data in the post-1950s (I assume that you didn’t apply a bodge here). This fully contradicts their claims, although I do admit that such an effect might be happening in some places.
See here for the response.
I raised the Briffa bodge as an issue in my submission the Briffa bodge to the Parliamentary Committee and Muir Russell as an example of “data manipulation”.
Although Muir Russell expressed disinterest in opining on the proxy issues that dominated the Climategate dossier, they reluctantly expressed an opinion on Briffa’s adjustment of the Tornetrask chronology, agreeing that the bodge was indeed “ad hoc”, but found (without giving any evidence) that there was nothing “unusual about this type of procedure”. While I presume that this reassurance was intended to comfort his audience, I wonder whether readers should in fact be comforted by this observation.
Muir Russell condemned critics for even questioning the Briffa bodge, stating that it was “unreasonable that this issue, pertaining to a publication in 1992, should continue to be misrepresented widely to imply some sort of wrongdoing or sloppy science”.
I’ll review the long backstory today.
Early CA Commentary on the Briffa Bodge
Briffa’s ad hoc adjustment of the important Tornetrask chronology was discussed in one of the earliest CA posts here (though the term “bodge” was not then used). I reported:
Briffa makes an ad hoc “adjustment” to the MXD chronology which has a dramatic impact on the relation of 20th century and medieval levels of the chronology, which then affects all downstream multiproxy studies…
The density chronology (Fig. 5b) shows a low-frequency decline over the last century which appears anomalous in comparison with both the RW data and the instrumental data over the nineteenth and twentieth centuries. These facts suggest that the density coefficients in the regression equation may be biased as would be the case if the density decline were not climate related (CO2 increases and/or the potential effects of increasing nitrogen input from remote sources, known to have occurred over the present century, may be implicated here.)
We examined the magnitude and timing of the recent MXD decline by regressing the 10-year smoothed RCS density curve against the equivalent RCS ring-width curve over the period 500-1750. The regression equation explains just under 35% of the MXD variance. Using this equation, we estimated MXD values for 501-1980. The residual MXD data (actual – estimated) are plotted in Fig 7. [this corresponds very closely to my 2nd panel] A systematic decline is apparent after 1750. By fitting a straight line through these residuals (1750-1980) and adding the straight line values (with the sign reversed) to the RCS density curve, the anomalous post-1750 decline was removed (cf. Fig. 7). This ‘corrected’ RCS curve was then used along with the RCS ring width curve in a final reconstruction of April-August temperature. The calibration of this reconstruction now explains 55 per cent of the instrumental variance (cf. Table 2 [where the 'uncorrected' reconstruction shows calibrated variance (R2) of 50.3%]. The improvement supports our contention that the anomalous recent density trend was not climate related.”
Obviously the very slight improvement in r^2 (unadjusted) arising from the bodge would not prove the validity of the bodge to any reasonable statistician. (I note in passing that Briffa et al 2001 also used a supposed microscopic improvement in unadjusted r^2 to adopt a reconstruction variation using principal components.)
CA reader Per (a scientist) commented in 2005 as follows:
Looking at this in broad daylight, I have to say that I am astonished. It is evident that you need the strongest possible justification for the data manipulation that was undertaken, and looking at the result of the data manipulation, it seems that it couldn’t be possible to justify such a manipulation under any circumstances. On the face of it, it seems that the meaning of the data has been turned upside down by this “manipulation” ! This must surely be a major source of embarrassment for all those authors who have used this dataset…
In fact, the climate community was totally unembarrassed by the bodge. The bodged Tornetrask chronology was used in the first generation of multiproxy reconstructions (still in the spaghetti graphs): Jones et al 1998, Mann et al 1998-99, Crowley and Lowery 2000. Tornetrask continued to be used in various versions in pretty much every reconstruction, but this is outside the scope of this post.)
McIntyre Submission, Feb 2010
The terms of reference for the Muir Russell “inquiry” had required them to examine possible “data manipulation”. In my submission (Feb 2010), I included the Briffa bodge of the Tornetrask chronology as a potential example of “data manipulation”.
In my introduction, I observed that some forms of data manipulation were so embedded that reviewers and specialists in the field either no longer noticed or were unoffended by the practices:
some forms of data manipulation and withholding are so embedded that the practitioners and peer reviewers in the specialty seem either to no longer notice or are unoffended by the practices. Specialists have fiercely resisted efforts by outside statisticians questioning these practices – the resistance being evident in the Climategate letters.
The bodge in Briffa et al 1992 was really the first example of “hide the decline”. I took care to observe that the bodge had been properly disclosed in the original 1992 paper. However, I noted that the presence of the bodge was not disclosed in the downstream multiproxy reconstructions (e.g. Jones et al 1998) nor included in the calculation of confidence intervals. Here is the relevant section of my submission:
One of the underlying problems in trying to use tree ring width/density chronologies for temperature reconstructions is a decline in 20th century values at many sites – Briffa’s 1992 density (MXD) chronology for the influential Tornetrask site is shown at left below. The MXD chronology had a very high correlation to temperature, but went down in the 20th century relative to what it was “expected” to do and relative to the ring width (RW) chronology (which had a lower correlation to temperature.) So Briffa “adjusted” the MXD chronology, by a linear increase to the latter values (middle), thereby reducing the medieval-modern differential. This adjustment was described in private as the “Briffa bodge” (Melvin and Briffa 2008).
Figure 1. Tornetrask from Briffa (1992). Left – MXD chronology. Middle – “Briffa bodge” ; right – Briffa 1992 “adjusted”.
In my submission, I pointed out that comments in Climategate programs about “fudge factors” and “artificial corrections” might indicate other use of bodges. In Tim Osborn’s submission to the Parliamentary Committee, he stated that such bodges were not used in his publications (while remaining silent on the use of bodges by other CRU scientists). In fact, CRU publications later than May 1999 – the date of publication of Briffa and Osborn (Science 1999) and Jones et al (Rev Geophys 1999) – do not appear to use bodges (despite code comments). Rather than bodging, they used Keith’s Science Trick (the simple deletion of data) to hide the decline.
CRU Submission to Muir Russell, March 1, 2010
CRU’s submission to Muir Russell, CRU asserted that none of their tree ring chronologies affecting the Medieval Warm Period (i.e. Tornetrask, Yamal, Polar Urals, Taymir) were affected by the divergence problem:
divergence does not affect any of our tree-ring-based temperature reconstructions that extend back to medieval time, the divergence phenomenon does not undermine the validity of our current estimates of the degree of warmth during the Medieval Warm Period. p 3)
Given that the Briffa bodge was designed to “correct” the decline of the Tornetrask MXD chronology (a mainstay medieval proxy), it seems hard to support CRU’s claim that divergence had not “affected” their Tornetrask chronology, particularly when CRU itself admitted that an “ad hoc” adjustment had been made to the Tornetrask chronology (as I had pointed out):
In one earlier publication (Briffa et al. 1992a) describing an analysis of ring-width and MXD in northern Sweden, we discuss a related observation of ‘divergence’ where a local, ring width chronology tracks the local rise in measured late 20th century temperatures, but a parallel chronology constructed from density measurements from some of the samples used to produce the ring-width chronology, exhibits a recent decline relative to the ring-width (and local temperature) data. In that paper it was assumed that the density decline was a recent phenomenon and a clear ad hoc correction, based on a comparison between the ring-width and density data, was applied before the ‘corrected’ chronology was used to reconstruct past temperature. No attempt was made to disguise this correction.
The issue wasn’t whether the adjustment had been disclosed in Briffa et al 1992; I had clearly stated that it had been disclosed in my submission. The issue was whether the bodge constituted “data manipulation” within the terms of reference of the Muir Russell “inquiry”, whether allowance for the bodge had been properly made in multiproxy reconstructions and whether it had been adequately disclosed in the multiproxy reconstructions.
CRU’s argument was that the bodge had been vindicated by later work, in effect saying that Briffa et al 1992 had used erroneous statistical methodology in the development of the original Tornetrask chronology and that the MXD decline at Tornetrask was an artifact of this erroneous methodology:
We have subsequently established that the reason for the relative density decline in this case was a bias in the RCS curve used to remove the influence of tree ageing from the original density measurements.
The bias came about because the standardisation curve (that quantifies the expectation of MXD as a function of ring age) was calculated without removing the parallel influence of climate on the growth of old age trees (see Section 1.1 for a discussion of standardisation and later discussion in this section for details of the bias issue). When a more appropriate application of standardisation is used, the agreement between these ring-width and density chronologies is markedly better, providing support for the efficacy of the original “correction” applied to the MXD data in our 1992 paper (for details of the later analysis see Briffa and Melvin, 2010).
Using the Alice-in-Wonderland logic that is all too prevalent in climate science, CRU then argued that their original reconstruction hadn’t been “affected by divergence” after all:
Because our original correction to the MXD chronology was subsequently shown to be justified and because the cause of the problem was shown to affect only the recent end of the chronology, we contend that our reconstruction of northern Swedish temperatures that used it (Briffa et al. 1992a) can justifiably be considered to be not significantly affected by divergence.
Alice-in-Wonderland logic – even if the bodged chronology were to be somewhat vindicated on other grounds (something that was not established in published academic literature at the time), it is untrue that the original chrnology was “unaffected” by divergence. Indeed, elsewhere, they inconsistently admitted that divergence had, in fact, been “observed” in the Tornetrask chronology:
An observation of divergence in a maximum-density chronology for northern Sweden (Briffa et al. 1992a) was later shown to be caused by the method of chronology production. The ad hoc correction applied to the original work was, therefore, appropriate and the temperature reconstruction for northern Sweden that made use of the corrected chronology is not invalidated by divergence.
And elsewhere, they argued only that improved statistical methods only “mitigated” the impact of divergence:
Though as yet unpublished, CRU has also indicated the possible role of standardisation as a cause of the apparent change in climate sensitivity of growth indices from near-tree-line conifers in North America (D’Arrigo et al. 2004) as described in Section 1.2. The signal-free approach to tree-ring standardisation has been put forward as a possible way of mitigating one manifestation of divergence that arises in the use of traditional and RCS-based standardisation techniques (Melvin 2004, Melvin and Briffa 2008, Briffa and Melvin 2010).
Briffa and Melvin,
2010 2011 of the Muir Russell report is Briffa, K. R., and T. M. Melvin. 2011. A closer look at Regional Curve Standardisation of tree-ring records: justification of the need, a warning of some pitfalls, and suggested improvements in its application in M. K. Hughes, H. F. Diaz, and T. W. Swetnam, editors. Dendroclimatology: Progress and Prospects. Springer Verlag. A preprint is online here.
I might add that I entirely support close examination of tree ring standardization methods of the type that Melvin is carrying out. The statistical issues are not easy ones.
Melvin proposes the concept of “signal-free” stamdardization. I think (though I’m not sure of this) that this relates to some prior discussion at CA in which I noted that dendro chronology calculations have analogues in linear mixed effects models e.g Pinheiro-Bates nlme package in R. I’ve shown that a “conventional” chronology can be obtained using the nlsList function and an RCS chronology using the nls function. The linear mixed effects function is the nlme function. My surmise is that Melvin’s “signal-free” method is a home-made method that has points in common with an nlme calculation. (This was something that I had hoped to present when I had been invited to present a paper at the World Dendro Conference last June; unfortunately this invitation was withdrawn.)
Given that the decline occurs at many sites (Briffa and Melvin focus on Tornetrask), I would be surprised if the decline is simply an artifact of tree ring standardization methodology. However, the potential of systemic sampling problems cannot be precluded. If the decline is simply an artifact of erroneous statistical methodology used by dendrochronologists, this deserves to be known. Given that similar methods are widely used in dendrochronology, if Melvin is right, it will require reappraisal of virtually every tree ring chronology in present use – a point that CRU failed to make.
The April 2010 Interview
After the announcement of the panel in February 2010, the Muir Russell “inquiry” carried out only one interview with Briffa (on April 9, collectively with Jones and Melvin.) The interview was attended by only two panelists, with Geoffrey Boulton, who been employed for 18 years by the University of East Anglia, taking the lead. There isn’t any transcript of the interview. The cursory minutes state on this issue:
The panel members had read the relevant paper (Briffa 1992) and the CRU submission in respect of the adjustment made to the Tornetrask series. They nevertheless wanted to cover the arguments again as an opening to the meeting. What was the scientific reasoning that justified the adjustments to the most recent period, and which has been described as a “bodge” in one submission?
Boulton’s notes simply recapitulate the CRU submission:
The TRW and MXD had proved to be a good proxy for high frequency temperature fluctuations, but the MXD series fell relative to the TRW for low frequency after 1750. The MXD had been adjusted to match the TRW based upon the assumption that the TRW was correct. It had later been found that the effect was due to a bias in the standardization procedure used and hence the ad-hoc adjustment had been good. It had not been used to argue for a temperature increase over the period. The manipulation had not been hidden, but had been clearly described in the paper.
CRU Further Comments, June 2010
CRU was afforded an opportunity to reply to Boulton’s meeting notes. (Critics were not offered a similar opportunity to rebut.)
And, on June 16, 2010, only a couple of weeks before the publication of the Muir Russell report, they did so here .
Their response states that the Review Team had requested “some additional context and support for the UEA response”, which they provided as follows:
The following comment is consistent with the “Summary of salient points…” but provides some additional context and support for the UEA response, as requested by the Review Team.
Professor Briffa was asked about the Tornetrask series created in Briffa et al. (1992). What was the scientific reasoning that justified the adjustments to the most recent period, and which has been described as a “bodge” in Briffa and Melvin (2010)? Briffa explained that the paper used two independent sets of measurements from the Tornetrask trees, maximum latewood density (MXD) and tree-ring width (TRW). The inter-annual variability of both is significantly correlated with variability in local temperature. TRW changes are associated with growing season temperatures mostly in midsummer (particularly July) while MXD appears to respond to the changes during a longer season (April to September). Indices of both (TRW and MXD) correspond well with the high-frequency changes in temperature. Prior to 1750 the low-frequency signals were similar in both series but after 1750 indices of the MXD series fell relative to the TRW series. Because of the much higher replication in TRW, Briffa presumed the TRW were correct and made a linear adjustment to the MXD series to force the MXD series to agree with the TRW series. He referred to this as the “bodge” in later work (Briffa and Melvin, 2010, a copy of which was supplied to the Review Team in the March 1, 2010 submission) which used improved processing methods that showed that the difference between TRW and MXD chronologies had resulted in part because the standardization technique used in the original paper (Briffa et al. 1992) to remove sample-age bias in the MXD data was itself biased. A brief presentation was made to the Review Team to illustrate this. This later work confirmed that it had been correct to adjust the MXD series in the earlier work and thus justified the original “bodge”. The manipulation had not been hidden, but had been clearly described in Briffa et al. (1992). Briffa directed the Review Team to our previous written submission of 1st March 2010 (page 7 in Section 1.2).
The following comment summarises material that was covered during the meeting but that was not included in the “Summary of salient points…”.
Melvin and Briffa also gave a detailed explanation of why the latest published Tornetrask density chronology (Grudd 2008) also advocated by McIntyre in his submission to this review is potentially in error because it applies a biased ‘standardisation’ approach to updated density measurements. This evidence was also shown to the Review Team in a brief presentation. This evidence is not published yet.
Although Muir Russell had stated that they would archive all submissions, they did not archive CRU’s “brief presentation” purporting to show that Melvin’s new statistical approach to standardization had vindicated the Briffa bodge.
Muir Russell Report, July 2010
The Muir Russell Report stated (7.3.4) they did not “focus upon disagreements over comparisons of results using individual tree series” even though issues regarding proxy reconstructions dominated the Climategate dossier. They reluctantly commented on Tornetrask only as the “subject of much misunderstanding” as follows:
Nevertheless we comment briefly upon Yamal as it has received so much attention and the Tornetrask series as it is subject of much misunderstanding.
As elsewhere, they did not provide any citations or references to actual examples of “misunderstanding”. Given that my submission was the only one on this topic, I presume that this accusation is made against me.
In the original press conference, Muir Russell had stated that they did not expect the public to accept “ex cathedra” pronouncements, but, in relation to the Briffa bodge (as elsewhere), that’s what they did, providing only the following declaration:
32. Finding on ― “Bodging” in respect of Tornetrask. The term ―bodging‖ has been used, including by Briffa himself, to refer to a procedure he adopted in 1992 . The ‘bodge‘ refers to the upward adjustment of the low-frequency behaviour of the density signal after 1750, to make it agree with the width signal. This ad hoc process was based on the conjecture that the width signal was correct. There is nothing whatsoever underhand or unusual with this type of procedure, and it was fully described in the paper. The interpretation of the results is simply subject to this caveat. The conjecture was later validated [14 -Briffa KR and Melvin TM, 2010 in press] when it was shown to be an effect due to the standardisation technique adopted in 1992. Briffa referred to it as a ‘bodge’ in a private e-mail in the way that many researchers might have done when corresponding with colleagues. We find it unreasonable that this issue, pertaining to a publication in 1992, should continue to be misrepresented widely to imply some sort of wrongdoing or sloppy science.
As all too common in the execrable Muir Russell “inquiry”, key assertions are made out of thin air.
They said that there was nothing “unusual” about Briffa’s ad hoc adjustment of the Tornetrask chronology. Puh-leeze. Where are other examples of this “statistical” methodology? Where is the discussion of the statistical assumptions and protocols? There aren’t any. Even in the statistically challenged world of paleoclimate, the Briffa “bodge” wasn’t a “usual” practice – as evidenced by the Cook email and the Stahle pers. comm. noted above.
Nor was Muir Russell justified in criticizing critics of the bodge as being “unreasonable”. Even if Briffa and Melvin
2010 2011 saves the Tornetrask chronology without damaging other chronologies – something that is far from demonstrated, Briffa and Melvin 2010 2011 was unpublished at the time of the Muir Russell report. And without a rational justification, critics cannot be blamed for regarding the bodge as “sloppy science” or worse.
And even if Briffa and Melvin
2010 2011 ultimately vindicates the Briffa bodge on other grounds, the bodge itself remains, at a minimum, “sloppy science”. Sloppy science that helped conceal the divergence problem and thereby delay analysis and reconciliation of the divergence problem.