The Smug Loop

Apr 17, 2011 – 12:34 PM

Both Andy Revkin Climate, Communication and the ‘Nerd Loop’ and Randy Olson, in a linked blog post, bemoan the state of “climate communication”, criticizing what they call the “nerd loop”. While I agree with Olson and Revkin that there is much to criticize in the climate communications community, I don’t think that either of them properly diagnose the problem. Continue reading →

By Stephen McIntyre | Posted in Uncategorized | Tagged | Comments (212)

Yamal and Oxburgh’s “Blinder Well Played”

Apr 11, 2011 – 11:09 AM

Please read the preceding post on Yamal background before today’s post discussing the handling of Yamal/Polar Urals by the Oxburgh “Inquiry”.

The Oxburgh and Muir Russell are particularly disquieting when one closely examines their handling of Yamal and Polar Urals, the issues that were most strongly highlighted in my own submission and that were most in controversy on the eve of Climategate.

Today, I’ll examine the findings of Kerry Emanuel and his fellow Oxburgh panelists on Yamal and Polar Urals. ( Emanuel was recently in the news in connection with his testimony to Congress, testimony in which he provided almost total disinformation about hide-the-decline – see CA post here.) I’ll conclude that their findings on these issues cannot be characterized as anything other than pure fantasy – a fantasy that was appetizing to a credulous climate science community, but fantasy nonentheless.

While the Oxburgh panel did not mention Yamal and Polar Urals by name, they made the following findings about CRU’s handling of tree ring chronologies, of which Yamal and Polar Urals were among the most prominent:

4. Chronologies (transposed composites of raw tree data) are always work in progress. They are subject to change when additional trees are added; new ways of data cleaning may arise (e.g. homogeneity adjustments), new measurement methods are used (e.g. of measuring ring density), new statistical methods for treating the data may be developed (e.g. new ways of allowing for biological growth trends).

5. This is illustrated by the way CRU check chronologies against each other; this has led to corrections in chronologies produced by others. CRU is to be commended for continuously updating and reinterpreting their earlier chronologies

[my bold]

CA readers obviously agree that chronologies are subject to change when additional trees are added. That was precisely the point of contention in respect to (a) the “Polar Urals Update” and (b) the regional chronology combining Yamal, Polar Urals and shorter chronologies, both of which were discussed at length at CA.

The issue in question was CRU’s failure to report (adverse) updated versions of the Polar Urals chronology and the regional chronology. On what basis could Emanuel and the Oxburgh panelists find that CRU deserved “commendation” for updating these chronologies, when it was their failure to report adverse updates that was at issue?

The “commendation” even flew in the face of CRU’s own evidence.

In written evidence to Muir Russell, CRU strenuously denied that they had failed to report adverse results merely because they were adverse. Their defence was that (strange as it might appear to third parties) they had never calculated the updated chronologies showing the adverse results. In respect to Polar Urals, CRU denied that they had ever reanalysed the data:

We had never undertaken any reanalysis of the Polar Urals temperature reconstruction subsequent to its publication in 1995.

As noted in my previous post, their evidence in respect to the regional chronology is inconsistent, on the one hand saying that they had never ‘considered” the incorporation of shorter chronologies:

However, we simply did not consider these data [Khadyta River] at the time, focussing only on the data used in the companion study by Hantemirov and Shiyatov and supplied to us by them.

and on the other hand saying that an “integrated” regional chronology had been an objective for Briffa et al 2008, but they “could not be completed in time”:

we had intended to explore an integrated Polar Urals/Yamal larch series but it was felt that this work could not be completed in time and Briffa made the decision to reprocess the Yamal ring-width data to hand, using improved standardization techniques, and include this series in the submitted paper.

Despite two “inquiries”, there has been no clarification of precisely what was done or not, as neither “inquiry” addressed the apparent inconsistency between the emails and their written evidence.

But there is one thing that can be said for certain: CRU did not report or apply either an updated Polar Urals chronology or their regional chronology combining Yamal, Polar Urals and shorter regional chronologies.

From that stating point, Oxburgh and/or Muir Russell could have evaluated whether CRU’s failure to report and apply the updated chronologies were consistent with “acceptable scientific practice” or not.

If such failures were consistent with “acceptable scientific practice” within the field, then, in my opinion, the “inquiries” should have urged that standards in the field be raised to ones that were consistent with what the public expected and was entitled to, given the large policy issues being faced. Or at least no lower than those applying to mining promotions. But that is different story.

However, given these facts, it was not permissible for the Oxburgh panel to make a finding (let alone a “commendation”) that CRU had “continuously updated and reinterpreted” the Polar Urals chronology and/or the regional chronology combining Yamal, Polar Urals and shorter chronologies in the region, an untrue finding that pre-empted consideration of the actual issues of why CRU hadn’t reported apparently adverse updates.

The Oxburgh panel was held out as not merely professional, but selected for eminence by the UK Royal Society. A distinguishing characteristic of “professionals” is an obligation of due diligence. Mistakes happen, but, by exercising due diligence, professionals protect themselves against accusations of negligence in the event of a mistake or error.

The Oxburgh panel flouted both usual inquiry standards and recommendations from the Parliamentary Committee. Among other lapses, the Oxburgh panel did not take submissions, did not interview CRU critics or take any transcripts of their interviews. In my opinion, because of both their inadequate due diligence and cavalier procedure, their incorrect findings about CRU chronology practices, applying to Polar Urals and the regional chronologies, were not merely a mistake, but a negligent mistake, perhaps even recklessly negligent.

It is disquieting, to say the least, that UK Chief Scientist John Beddington should describe such negligence as a “blinder well played”.

By Stephen McIntyre | Posted in Uncategorized | Tagged , , , | Comments (63)

Yamal and Hide-the-Decline

Apr 9, 2011 – 3:28 PM

In The Climate Files, Fred Pearce wrote:

When I phoned Jones on the day the emails were published online and asked him what he thought was behind it, he said” It’s about Yamal, I think”.

Pearce continued (p 53):

The word turns up in 100 separate emails, more than ‘hockey stick’ or any other totem of the climate wars. The emails began with it back in 1996 and they ended with it.

Despite Jones’ premonition and its importance both in the Climategate dossier and the controversies immediately preceding Climategate, Yamal and Polar Urals received negligible attention from the “inquiries”, neither site even being mentioned by Kerry Emanuel and his fellow Oxburgh panellists.

I recently submitted an FOI request for a regional chronology combining Yamal, Polar Urals and “other shorter” chronologies referred to in an April 2006 email – a chronology that Kerry Emanuel and the “inquiries” failed to examine. The University of East Anglia, which seems to have been emboldened by the Climategate experience, not only refused to provide the chronology, but refused even to provide a list of the sites that they used to construct the regional chronology.

This refusal prompted me to re-appraise Yamal and its role in the Climategate dossier.

Yamal at the Start and End of the Emails
Although the climate science community has represented Climategate as being about the CRU temperature record, the temperature record is only mentioned in a couple of emails.

The dossier is not about CRUTEM, it’s about the Hockey Stick. And within that debate, the dossier was seemingly constructed with particular attention to Yamal. Pearce’s observation about the Climategate dossier beginning with Yamal is literally true. The very first email (1. 0826209667.txt) is about Yamal – an opening scene wittily described by Michael H. Kelly (not the Michael Kelly of the Oxburgh panel) in an overlooked account of the emails shortly after they became public:

Like an Aristophanes satire, like Hamlet, it opens with two slaves, spear-carriers, little people. Footsoldiers of history, two researchers in a corrupt and impoverished mid-90s Russia schlep through the tundra to take core samples from trees at the behest of the bigger fish in far-off East Anglia. Stepan and Rashit don’t even have their own e-mail address and like characters in some absurdist comedy must pass jointly under the name of Tatiana M. Dedkova. Conscientious and obliging, they strike a human note all through this drama. Their talk is of mundane material concerns, the smallness of funds, the expense of helicopters, the scramble for grants. They are the ones who get their hands dirty, and their vicissitudes periodically revived my interest during the slower stretches of the tale, those otherwise devoted to abstruse details of committee work and other longueurs. ‘We also collected many wood samples from living and dead larches of various ages. But we were bited by many thousands of mosquitos especially small ones.’ They are perhaps the only likeable characters on the establishment side, apart from the exasperated and appalled IT man Harry in the separate ‘Harry_read_me’ document, and I cheered up whenever they appeared.

The relatively unexplored data accompanying the emails is mostly tree ring data and contains interesting tree ring data not otherwise available, some of which I’ve used in today’s post (see below).

And, as Pearce observed, the closing scenes of the Climategate dossier showed CRU’s reaction to a series of posts at Climate Audit on Yamal in October 2009, which, together with the nearby Polar Ural site, had been a longstanding issue at Climate Audit.

The controversy in October 2009 was actually the second major CA dispute involving tree ring chronologies from NW Siberia. The earlier criticism was of CRU’s failure to publish an amendment to the prominent Polar Urals chronology (Briffa et al Nature 1995) to show the impact of measurement data that became available subsequent to the original publication (the availability of new data and its value in firming up the Polar Urals chronology timing is referred to in a 1999 email). As reported here, the new data showed a prominent MWP, contradicting Briffa et al 1995 on a cold 11th century.

CRU’s failure to report the update and the seemingly opportunistic adoption of the hockey-stick shaped Yamal chronology in its place had been the topic of many Climate Audit posts, some of which attracted CRU notice (as evidenced by the emails). I unsuccessfully raised the issue in AR4 review comments. CRU’s written evidence to Muir Russell defended their failure to publish or use an updated Polar Urals chronology on the simple fact that they had never updated or reanalysed their Polar Urals chronology subsequent to its publication:

We had never undertaken any reanalysis of the Polar Urals temperature reconstruction subsequent to its publication in 1995.

Given the shortage of 1000-year chronologies, one would have thought that additional data for the long Polar Urals chronology would have been eagerly awaited by its proponents, but seemingly not.

On the other hand, as Mosher has frequently observed, it is almost characteristic for Briffa to make totally contradictory statements within the same article and their evidence to Muir Russell is no exception. Having clearly said that they never reanalysed Polar Urals, CRU then alluded to reanalysis of the Polar Urals record. I’ll try to analyse this evidence on a future occasion; in the meantime, interested readers should consult the submission themselves.

The October 2009 Controversy

The Yamal issue in October 2009 (on the eve of Climategate) arose out of Briffa et al 2008 (Phil Trans B), the data for which had just become available in late September 2009 through the intercession of the editors of Phil Trans B – a journal not specializing in climate and thus not prepared to abet Team data obstruction.

Briffa et al 2008 purported to show “regional” chronologies for three northern Eurasian areas. One regional chronology combined data over a large area including both Finland and Tornetrask, Sweden. A second regional chronology (Avam-Taymir) substantially expanded the Taymir chronology of Briffa 2000 through the addition of sites from Sidorova 2007 (mentioned in the text) and the Schweingruber network (Briffa et al Nature 1998.) The effect of the additions was to somewhat mitigate the decline in the prior chronology. Both networks were very large in their modern portion – each had well over 100 cores. However, the network labeled as a “regional chronology” for Yamal did not expand from the Briffa 2000 network, even though this network had only a fraction of the sample in the other networks – only 10 cores in 1990 and 5 cores in 1995.

The precedent of the addition of Schweingruber sites at Taimyr raised the question why Schweingruber sites weren’t added to the Yamal network to make a truly regional chronology along the lines of the other two. Aside from nearby Polar Urals, there was even a Schweingruber site at the very location of a large proportion of the subfossil samples used in the Yamal network (Khadyta River, Yamal). I raised this issue at CA in a series of posts (see here). The matter was covered even in mainstream press. Ross stated the issue sharply in a National Post op ed in October 2009 as follows:

Combining data from different samples would not have been an unusual step. Briffa added data from another Schweingruber site to a different composite, from the Taimyr Peninsula. The additional data were gathered more than 400 km away from the primary site. And in that case the primary site had three or four times as many cores to begin with as the Yamal site. Why did he not fill out the Yamal data with the readily-available data from his own coauthor? Why did Briffa seek out additional data for the already well-represented Taimyr site and not for the inadequate Yamal site?

This time, CRU responded with online posts here on October 1, 2009 and here on October 27, 2009. The latter post came only three weeks before Climategate and hasn’t been fully analysed to date.

In the October 27 article, Briffa conceded that the Schweingruber Khadyta River dataset did meet CRU’s criteria for inclusion in a regional chronology (thus contradicting NASA blogger Gavin Schmidt who had sneered at the idea in an earlier real climate post here.) Briffa’s excuse was similar to the one later provided in connection with their failure to report on the updated Polar Urals data. They explained their failure to include the data in a regional chronology as due only to the fact that “we simply did not consider these data at the time”:

Our current practice when selecting data to incorporate in a regional chronology, is to include data exhibiting high levels of common high-frequency variability (i.e. on the basis of high inter-site correlations, where these are calculated using high-pass filtered data). Judged according to this criterion it is entirely appropriate to include the data from the KHAD site (used in McIntyre’s sensitivity test) when constructing a regional chronology for the area. However, we simply did not consider these data at the time, focussing only on the data used in the companion study by Hantemirov and Shiyatov and supplied to us by them.

In evidence to Muir Russell published after the inquiry, they expanded slightly on this “explanation”, re-iterating that they never “considered” the Schweingruber data, adding that the “purpose of the work” reported in Briffa et al 2008 was merely to “reprocess the existing dataset of Hantemirov and Shiyatov”:

McKitrick is implying that we considered and deliberately excluded data from our Yamal chronology. The data that he is referring to were never considered at the time because the purpose of the work reported in Briffa (2000) and Briffa et al. (2008) was to reprocess the existing dataset of Hantemirov and Shiyatov (2002).

This “purpose” is nowhere stated in the actual text of Briffa et al 2008. A reviewer or editor, aware that this was the actual “purpose” of the article, would surely have asked the authors to, for example, describe the differences between the new method and previous methods and to show the actual impact of the new method on the Yamal dataset, none of which is described in the article. Any reviewer, editor or reader of the actual text of Briffa et al 2008 cannot help but presume that its “purpose” was not to show the reprocessing of the Hantemirov and Shiyatov 2002 data, but to present regional chronologies expanding the network of Briffa 2000 though the combination of sites.

In CRU’s evidence to Muir Russell, Briffa, with his characteristic inconsistency, contradicted his earlier statement that they had not “considered” the possibility of additional data. Not only did he contradict this earlier statement, he stated that the original intention and objective of the Briffa et al 2008 article was to produce an “integrated Polar Urals/Yamal series” (along the lines that Climate Audit had wondered about), and that this objective was not pursued only because “it was felt that this work could not be completed in time”:

Some historical context for our 2008 paper might shed some light on this issue. Some time ago we began work on a multi-institution paper intended to describe the sensitivities in producing tree-ring-based climate reconstructions to the methods of chronology construction and subsequent climate calibration, illustrated using the examples of various tree-ring chronologies across northern Eurasia. When we later received a request to submit a paper to a planned themed issue of the Philosophical Transactions of the Royal Society about ‘The boreal forest and global change’, Briffa and colleagues decided to use some of the material to hand in preparing a draft. It was intended that this should describe 3 continuous 2000-year ring-width series, each originally planned to represent the integration of a large-regional data set of subfossil and living tree data. The focus was to be on representing large-regional growth signals and initial comparisons with equivalent regional temperature data. The western, ‘Fennoscandia’, series would incorporate near tree-line pine data from northern Sweden and Finland; the Avam-Taimyr series would integrate larch data from near the Taimyr peninsula tree-line region.

Between these we had intended to explore an integrated Polar Urals/Yamal larch series but it was felt that this work could not be completed in time and Briffa made the decision to reprocess the Yamal ring-width data to hand, using improved standardization techniques, and include this series in the submitted paper.

Hide the Decline

I’ve referred here to Schweingrubers sits. These are of course familiar to readers because the Schweingruber network was the data used in the Briffa MXD reconstruction of hide-the-decline notoriety. As shown below, Yamal is in the heart of hide-the-decline country in northern Siberia. The figure below shows the location of Yamal on the location map ( Figure 1) of Briffa et al 1998 (Nature), a figure that compared 1975-1985 tree ring values to 1935-1945 values (MXD left; ring width right). The graphic on the left also shows the location of Schweingruber sites – plentifully represented in Russia, to say the least. Briffa et al 1998 Figure 2 showed MXD and ring width chronologies from 1880-1992, with the third row in the panel showing a pronounced decline for West Siberia


Figure 1 Spatial patterns of relative tree-growth decline. a, The location (circles) of tree-ring chronologies and the division (black lines) into regional averages. b, The locations (black lines) of the grid-box temperatures used for comparison with the tree growth series. The coloured contours show where the ring density (a) and ring width (b) are enhanced (positive) or suppressed (negative) relative to summer temperature during the period 1975–85 compared to the period 1935–45.

In most scientific endeavours, one presumes that specialists would wonder why Yamal had a huge hockey stick, when it was located in a zone with dozens of sites showing a decline. However, no such reflection took place among paleoclimatologists. None seem to have wondered about the validity of the Yamal proxy. And even though Briffa’s chronology was never being formally published (it was shown only passim in Briffa 2000, which did not even report core counts), its use spread rapidly among paleoclimatologists – I sarcastically termed the Yamal chronology as “cocaine” for paleoclimatologists (bristlecones were, of course, heroin) and, from time to time, reflected on how to insert musical accompaniment from, respectively, Eric Clapton and the Velvet Underground whenever either proxy appeared.

Not only was the Yamal chronology widely adopted, the evidence of a decline in a network of nearly 400 sites (with hundreds of cores, not 10) came to be discounted. In Kerry Emanuel’s recent testimony, he stated that the evidence from the very large Schweingruber network was “provably false”, a position that has not been contradicted by any dendroclimatologist.

The Vaganov Network
As noted above, in their online response of October 2009, CRU conceded that it was entirely legitimate to include the Khadyta River and other sites highly correlated to Yamal in their regional chronology, explaining their inclusion practice as follows:

Our current practice when selecting data to incorporate in a regional chronology, is to include data exhibiting high levels of common high-frequency variability (i.e. on the basis of high inter-site correlations, where these are calculated using high-pass filtered data). Judged according to this criterion it is entirely appropriate to include the data from the KHAD site (used in McIntyre’s sensitivity test) when constructing a regional chronology for the area.

Although they said that they had not previously “considered” the use of additional data, they now said that the best approach to developing a chronology was “making use of all the data to hand”:

So what is the “best” indication of relative ring-width changes in this Yamal region? One approach is to judge this by making use of all the data to hand.

To that end, they stated that they had “taken the opportunity to acquire and incorporate additional data from the 3 original sites” [POR, YAD, JAH]. (The Climategate documents showed that they had actually had this data since 1996 or so). They produced another chronology including this data with the Khadyta River (KHAD) data that I had previously identified, arguing that this new chronology was similar to the original Briffa 2000 and Briffa 2008 chronologies and that therefore their previous failure to report a regional chronology didn’t matter, a viewpoint quickly accepted by the climate science community.

But there are some important loose ends.

The CRU (October 2009) position only permits a “limited hangout” (to borrow the Nixonian phrase.) Khadyta River is only one of dozens of Schweingruber sites, a number of which are close to Yamal. In addition, Eugene Vaganov, a prominent Russian dendrochronologist, had also made a large network (61 sites) in the early 1990s overlapping the Schweingruber network in Russia and many of his sites are also near Yamal. This data was never archived, but became available in the Climategate documents.

The figure below shows a detail from the Briffa et al 1998 location map on which the Schweingruber sites are shown in orange, the Vaganov sites in red, Yamal in limegreen. I’ve also added a box showing 10 degrees east-west of the Yamal sites and 65-70 degrees latitude that I’ve used to extract subsets for comparison. Within this box, there are 20 Vaganov sites and 15 Schweingruber sites, all of which (as shown below) appear to meet the relevant standards for inclusion in a regional chronology.

Figure 2. Excerpt from Briffa et al (Nature 1998) Figure 1 showing Western Siberian location.

The next figure shows compares core counts in the Vaganov and Briffa 2009 networks. The Vaganov network has 400-500 cores over most of the 20th century. In contrast, the living tree portion of the Yamal “network” in Briffa (2000) and Briffa et al 2008 had only 17 cores (10 in 1990; 5 – in 1995). The somewhat expanded Limited Hangout network of Briffa 2009 still contained only 11-12% of the number of cores of the Vaganov network and thus falls far short os using “all” the data. (It didn’t even use the Polar Urals data.)


Figure 3. Core counts for Vaganov and Briffa networks.

The effect of using “all the data to hand” is potentially quite dramatic. The graphic below compares the Briffa 2009 chronology (red) to the average of site RCS chronologies for the 20 Vaganov sites in the 10-degree box. As you can see, there is considerable correlation between the two chronologies, though the Briffa version is spikier than the much larger Vaganov network. The discrepancy becomes very pronounced from the 1970s on – the Vagnov network shows the characteristic “decline” in the late 20th century that also characterized the large Schweingruber network, while the Briffa Limited Hangout network surges to new records.


Figure 4. Comparison of Briffa 2009 chronology and regional chronology from Vaganov data.

The difference between the two results arises because the difference arising from the closing values of the POR and YAD sites in a small network. If they are added into the 20-site Vaganov network, their influence is diluted, but when there are only two other sites (JAH and KHAD), they still yield high closing values.

I’ve observed on other occasions that the Yamal hockeystick is typically very important in multiproxy reconstructions that don’t use bristlecones. In addition to Briffa 2000 and Briffa et al 2008, it is used in Briffa and Osborn 1999; Mann and Jones 2003; Bradley, Hughes and Diaz 2003; Jones and Mann 2004; Moberg et al 2005; D’Arrigo et al 2006; Osborn and Briffa 2006; Hegerl et al 2007; Mann et al 2008 (blended into the Tornetrask version) and Kaufman et al 2009.

Climategate
The Climategate emails contain much information about Yamal and Polar Urals, none of which is referred to by any of the various “inquiries”. I won’t survey all the emails in this note but will touch on some of the most important.

CRU awareness both of the existence of the updated Polar Urals measurement data (Schweingruber’s polurula dataset) and the close relationship between Polar Urals and Yamal is shown in email 1136918726.txt on Jan 10, 2006 from Wigley to Briffa. (This is also the email in which Wigley wonders whether a “reindeer crapped next to one of the trees”.)

Keith,

Thanx for this. Interesting. However, I do not think your response is very good. Further, there are grammatical and text errors, and (shocking!!) you have spelled McKitrick wrong. This is a sure way to piss them off.

They claim that three cores do not cross-date for TRW. They also say (without results) that the same applies to MXD (these results may be in their Supp. Mat. — I presume you checked this).

So, all you need say is …

(1) TRW was not the only data used for cross-dating.
(2) When MXD is used there are clear t-value peaks, contrary to their claim. You can show your Fig. 4 to prove this.
(3) The 3-core-composite cross-dates with other (well-dated) chronologies (Yamal and Polurula), confirming the MXD-based dating. You can show your Fig. 5 to prove this.

You could say all this in very few words — not many more than I have used above. As it is, your verbosity will leave any reader lost.

There are some problems still. I note that 1032 is not cold in Yamal. Seems odd. Is it cold in *all* of the three chronologies at issue? Or did a reindeer crap next to one of the trees?

Also, there seems to be a one-year offset in the 1020s in your Fig. 6.

I hope this is useful. I really think you have to do (and can do) a better job in combatting the two Ms. If this stuff gets into Nature, you still have a chance to improve it. Personally, I think it would be good for it to appear since, with an improved response, you can make MM look like ignorant idiots.

Tom

This email had been prompted by our submission of short comment criticizing crossdating in the Polar Urals data (Briffa et al 1995), an important element in the medieval-modern comparison of Jones et al 1998, (which I had moved on to after MBH98). In their response, they referred to the close relationship with Yamal (permitting the use of the Yamal chronology for crossdating Polar Urals), but notably did not mention the updated Polar Urals measurements, which we had not referred to in our submission.

Subsequent to the publication of Briffa et al. (1995), completely independent ring-width data have become available for the Yamal area adjacent to Polar Urals (Hantemirov & Shiyatov 2002). Comparisons between ring-width data for each of the early Polar Urals cores under discussion and the Yamal chronology (freely available to MM on my website) all confirm our original dating.

Their response referred to various figures, none of which were provided to us. Nor were the statements in their response correct. Comparisons to the additional Polar Urals measurement data and Yamal confirm our original observation of misdating of the 11th century cores used in Briffa et al 1995 and applied in the Jones et al 1998 reconstruction that is still in use.

Perhaps the most important email concerning Yamal was in April 2006 ( 684. 1146252894.txt ). It was one of only two emails that I referred to in the running text of my own submission to the Parliamentary and Muir Russell committees.

Osborn had emailed Philip Brohan of the UK Met Office (cc Briffa) describing CRU’s three regional chronologies (a single series for each) for the three regions later used in Briffa et al – SCAND, URALS and TAIMY – with the URALS series the defined as “Yamal and Polar Urals long chronologies, plus other shorter ones”. The “other shorter ones” were presumably nearby Schweingruber chronologies – not just Khadyta River. The email:

Date: Fri, 28 Apr 2006
To: philip.brohan
From: Tim Osborn
Subject: Re: Standardisation uncertainty for tree-ring series
Cc: Keith Briffa,simon.tett
Hi Philip,
we have three “groups” of trees:

“SCAND” (which includes the Tornetrask and Finland multi-millennial chronologies, but also some shorter chronologies from the same region). These trees fall mainly within the 3 boxes centred at: 17.5E, 67.5N;22.5E, 67.5N; 27.5E, 67.5N
“URALS” (which includes the Yamal and Polar Urals long chronologies, plus other shorter ones). These fall mainly within these 3 boxes: 52.5E, 67.5N; 62.5E, 62.5N (note this is the only one not at 67.5N); 67.5E, 67.5N

“TAIMYR” (which includes the Taimyr long chronology, plus other shorter ones). These fall mainly within these 4 boxes: 87.5E, 67.5N; 102.5E, 67.5N; 112.5E, 67.5N; 122.5E, 67.5N
We do some analysis at the group scale, and for this we take the JJA temperatures from each box and average to the group scale to obtain a single series from each of SCAND, URALS and TAIMY.

We do some analysis at the overall scale, and for this we take these three group temperature series and average them to get an overall NW Eurasia temperature for boxes with tree chronologies in them.

We did also try using a wider average for the region, including all LAND temperatures from grid boxes within a rectangular region from 12.5E to 127.5E and from 57.5N to 72.5N, but I don’t think it correlated so well against the tree-ring width data (I can’t remember the exact correlations), so we didn’t pursue that.

Does that give you enough information to be going on with? I’d recommend using CRUTEM3 rather than HadCRUT3, because the correlations seem to deteriorate with the inclusion of SST data in some cases — though of course you can look into this yourself.
Cheers
Tim

In March 2007 (780. 1172776463.txt), Osborn again referred to a regional chronology combining Yamal and Polar Urals as being used in a PPT presentation. Osborn noted that revisions to the chronology had made “quite a big change”, thereby getting rid of what had previously been a “higher peak near 1000 AD“:

Here is the old version for you to compare with… the only noticeable difference is for the URALS/YAMAL region, which previously had a higher peak near 1000 AD. Although that was quite a big change, once you average it with the other two series, the overall mean series shows very little difference.
Cheers, Tim

I find the evidence of these emails is hard to reconcile with CRU statements that they had never “considered” updating the Polar Urals chronology, that they had never “considered” adding other chronologies to Yamal to make a regional network and that they had “lacked time” to make a combined Polar Urals/Yamal chronology.

In order for the “inquiries” to properly carry out their obligations, they should, among other things, have obtained the regional chronology referred to in email 1146252894.txt and provided an explanation of why this chronology never appeared. They should also have resolved the apparent contradictions between CRU’s evidence to Muir Russell (and their online statement before Climategate) was that they never “considered” the addition of Schweingruber and other data to make a regional chronology with the evidence of the emails that they had already made such a calculation in April 2006.

I’ll review the conclusions of the inquiries and the reasons for the FOI refusal in following posts.

By Stephen McIntyre | Posted in climategate, Uncategorized | Tagged , , , , | Comments (94)

Disinformation from Kerry Emanuel

Mar 31, 2011 – 2:16 PM

In his written and oral evidence at today’s hearing before the House Science Committee, Kerry Emanuel made untrue statements about deletion of data to hide the decline. From Emanuel’s written evidence (oral was similar):

Consider as an example the issues surrounding the email messages stolen from some climate scientists. I know something about this as I served on a panel appointed by the Royal Society of Great Britain, under the direction of Lord Oxburgh, to investigate allegations of scientific misconduct by the scientists working at the Climate Research Unit of the University of East Anglia. Neither we nor several other investigative panels found any evidence of misconduct. To be sure, we confirmed what was by then well known, that a handful of scientists had exercised poor judgment in constructing a figure for a non peer-reviewed publication. Rather than omitting the entire record of a particularly dubious tree-ring-based proxy, the authors of the figure only omitted that part of it that was provably false. If this was a conspiracy to deceive, though, it was exceedingly poorly conceived as anyone with the slightest interest in the subject could (and did) immediately find the whole proxy record in the peer-reviewed literature.

The true scandal here is the enormously successful attempt to elevate this single lapse of judgment on the part of a small number of scientists into a sweeping condemnation of a whole scholarly endeavor

The proxy in question is, of course, the Briffa reconstruction. Emanuel says that the authors “only” deleted the part of the reconstruction that was “provably false”, with the problem limited to a “non peer-reviewed publication”.

The Briffa network was developed from 387 sites anticipated to be temperature proxies because of their latitude or altitude. Emanuel has no basis for describing the Briffa network as “a particularly dubious tree-ring-based proxy”. It is an important large population and there is no evidence that the measurements were taken inaccurately. The NAS panel in 2006 did say that strip bark proxies should be “avoided” in temperature reconstructions. If Emanuel were seriously concerned about the use of “particularly dubious” tree-ring proxies, shouldn’t these be the ones that he should be worried about?

Emanuel’s evidence to the House Committee that the deletion of the decline was limited to a “non peer reviewed article” was also untrue. I presume that he is referring here to Phil Jones “combo trick” in the WMO 1999 report. As CA readers know, Keith’s Science Trick – the omission of part of the data – was systemic in the peer reviewed literature after 1999. Examples include the spaghetti graphs in Briffa and Osborn (Science 1999), Jones et al (Rev Geophys 1999), Briffa et al (JGR 2001) Plate 3, Jones et al 2001 Plate 2A, Briffa et al 2004 Figure 8, Hegerl et al Figure 5b. (CRU conceded most of this in their March 1, 2010 submission to Muir Russell, see page 38). Plus of course the spaghetti graphs in IPCC TAR and IPCC AR4.

Emanuel says that hide-the-decline was a “single lapse of judgement”. More disinformation on his part. The decision to “omit” part of the record was made over and over. It began in 1999, but continued unabated through IPCC AR4.

Worse, the practice was directly challenged by an AR4 reviewer (me). I requested IPCC to show the decline and explain it as best they could. I said that the deletion of the decline in TAR was misleading and asked that they not do so anymore. Briffa refused, merely saying that it would be “inappropriate” to show the decline. This was not a “single lapse of judgement”. It was something that’s gone on for over a decade.

Emanuel says that the “true scandal” is the elevation of hide-the-decline into a “sweeping condemnation of a whole scholarly endeavour”.

In my opinion, Emanuel and other senior members of the climate community bear much of the responsibility for the escalation of the incident beyond the borders of East Anglia and Penn State. If Emanuel and others wanted to stop criticism and suspicion, they should have carried out their inquiries in a systematic way, as inquiries are carried out in other fields.

The Oxburgh inquiry, of which he was a member, should have had written terms of reference, should have interviewed critics as well as CRU, should have had (at least) transcripts of the interviews – among other things.

The Oxburgh “report” was an insult to the public. Emanuel shares the blame for that.

That Emanuel, a member of one of the inquiries, should be unaware that hide-the-decline occurred in peer reviewed literature and IPCC merely proves, if anyone were in doubt, that the inquiries were cavalier and negligent rather than thorough and diligent.

By Stephen McIntyre | Posted in Uncategorized | Tagged | Comments (188)

Webcast of House Committee Hearings

Mar 31, 2011 – 9:29 AM

See here http://science.house.gov/hearing/full-committee-hearing-climate-change
Webcast

Witnesses
Dr. J. Scott Armstrong, Professor, The Wharton School, University of Pennsylvania
Dr. Richard Muller, Professor, University of California, Berkley and Faculty Senior Scientist, Lawrence Berkeley Laboratory
Dr. John Christy, Director, Earth System Science Center, University of Alabama in Huntsville
Mr. Peter Glaser, Partner, Troutman Sanders, LLP.
Dr. David Montgomery, Economist
Dr. Kerry A. Emanuel, Professor, Massachusetts Institute of Technology

Armstrong – I tuned in late.

Muller
10.32 Unadjusted data is similar to three indices. Why? Muller surprised. Comments on Watts: Poor stations don’t rise more than good stations. Compliments McIntyre and Watts. Some of the biases less of a problem than previously thought.

Christy
Bio details. Criticizes IPCC permitting Lead Authors to assess their own work. 1) Hockey Stick case – Lead Author was conflicted. Hide the decline. 2) the McKitrick case. IPCC Lead Authors assessed their own work. 3) EPA Endangerment finding – EPA overstated agreement between models and data. U.S. should not rely on IPCC process. “Climate science needs adult supervision”. Needs second opinion fro, non-activist scientists.

Glaser
Commented on EPA process. 1) Did not consider societal benefits from fossil fuels. Health has not deteriorated, the opposite. 2) EPA had already decided on finding before comments. 3) EPA relied almost entirely on third party assessment, most heavily on IPCC. Failure to do own assessment is violation of US guidelines. EPA did not examine IQA of IPCC sources. Climategate showed that EPA should have permitted comment on EPA reliance on EPA. Written testimony contrasts EPA procedure on GHG with other policy cases.

Emanuel
19th century history. Arrhenius’ sensitivity estimate 5-6 deg C. Scientific basis is “solid”. Attribution of last half temperature increase to CO2 is solid. Many academies have issued warnings. DOD issued warning. Scientists are conservative and underestimate risks (Fukushima earthquake risk). Assessments of risk are uncertain. Uncertainties unlikely to decline in next decade. No basis for confidence that effects benign. Served on Oxburgh panel. Showed “poor judgement” but no evidence of intent to deceive. Mavericks usually wrong (HIV-AIDS). Politicians should not make mascots of mavericks. Used phrase twice. Appeal to forefathers.

Montgomery
Failures of economic analysis. Global phenomenon. Reducing GHG emissions will have a cost. Deeper the costs, the greater the costs. Green job claims are wrong as they ignore job losses elsewhere. Regulations do not help US competitiveness. Costs of policies underestimated and benefits exaggerated. EPA estimates unreliable. Free lunch assumptions. Unilateral action by US will not accomplish anything. Need India and China as well.

Questions
Hall
Glaser gave example of EPA responding to comments in boiler rules. Process flaws in Endangerment are not mere technicalities. Process was undermined. EPA had determined its conduct in advance.

To Montgomery – repercussions of removing coal from generation mix while electricity growth of 21% by 2035. Carbon capture is speculative technology. EPA regs would increase electricity costs by 40-50% in ten years. Will Russia, China, India and others participate?

Johnston
Is temperature rising and greenhouse gas partly to blame? Christy – Muller – GHG by themselves exert warming. The question is one of degree. Is it high end? How do we work with other countries? If it is low end, then time to implement more long term solutions that some people object to. Emanuel – projections range from benign to catastrophic.

Do we have answers now? Muller – No “conspiracy”, but scientists work as advocates and fears that they lose their impartiality. Don’t trust the public enough. Bad effect is that public loses trust. Christy – tests model output and doesn’t match data. Emanuel – some areas show model estimates too conservative. IPCC is not a research organization, but a “communications exercise” between scientists and public.

Sensenbrenner.
Asks about Oxburgh. Five questions.
Did the panel have any written terms of reference?
Public hearings?
Transcript of hearings.
“No, I don’t believe so”.

Scope of panel. Any breach of scientific integrity.
Describes advocacy of Oxburgh. Isn’t that a conflict of interest? Papers we read, interviews we conducted showed great integrity. No sunshine on the process. Didn’t interview critics, no transcript of interviews.

Miller
Something from Armstrong’s website on election forecasting. Offering personal opinions on matters on which he has appeared as a lawyer. Unethical for lawyers to express personal opinion as opposed to client. Is there any area on which his personal opinion differs from opinion of clients? Asks about how much he has been paid. Attorney-client privilege. Odd that Glaser asked to appear. In a court case, multiple parties.

Bartlett
long statement about energy. Montgomery – oil, climate change and fossil fuels different issues. Peak oil a different issue.

McNair??
Asks Christy about openness of IPCC process. Asks Christy about code. Says important that code be available. Asks about “fraudulently” inclusion of data in reports. “biased… overconfident”.

Asks Montgomery – oil companies spent massive amounts against climate science. Testified as expert witness on unrelated cases for Exxon.

Rep worked in wind energy field. Buying wind turbines from Germany.

Brown
Would you trust data from hide the decline..?

Quotes from harry code. “hopeless data base”. Christy – panels did not address issues. Did reviews address Climategate issues? No. Is independent review of allegations warranted? Asks for report different from IPCC cadre.

MIT received $100 million from Koch. Should this be dismissed?

Edwards
Seven independent exonerations of Climategate (waved). Asks whether funded by oil, coal or energy industry.

She represents NASA Goddard and NOAA. Needs to invest in climate research.

Harris
Is there evidence for confidence that it will be risky? No.
Difference between forecast and risk assessment – house on fire.

Glaser – Glaser’s point that public health has improved. Good things not taken into consideration.

To Montgomery – Going to borrow money from Chinese to buy German or Chinese windmills. Then increase electricity to place US at competitive disadvantage. Yes.

Intervenor
About letter from Watts about Muller testimony.

Clark
Pain and suffering of decline of market share in autos. Represents Detroit. Concerns not heeded. No jobs in auto plants. Concerned about investment in green technology.

???
Asks Christy about great global cooling scare. From mining district in Minnesota.

Tuned out for a while.

Some closing remarks about Climategate. Emanuel said of hide-the-decline that they should have taken whole proxy out. No intent to deceive anyone. Christy – icon itself based on tree rings as not very good. Muller – had to show your dirty laundry at Berkeley. If you don’t show something , you are most likely to fool yourself. Glaser – Climategate about many things. Hockey stick is about to the public. About large pattern of activity. Review panels – English investigated. Asked EPA to investigate and allow public to comment. None of the review panels operated according to US procedures. No interviews, no dissenting points of view. Review panels were critical of scientists. Surprised that climatologists did not invole disinterested statisticians. Operating in culture of secrecy.

By Stephen McIntyre | Posted in Uncategorized | Comments (20)

Muir Russell and the Briffa Bodge

Mar 30, 2011 – 1:25 PM

There has been some recent discussion of the Briffa bodge – an early technique to hide the decline. I had drafted a post on the topic and its handling by the Muir Russell “inquiry” in early July 2010, but did not publish the post at the time. In today’s post, I’ve slightly updated my July 2010 draft.

The term “bodge” was used for the first time in a comment (not a post) on November 8, 2009 by me here less than two weeks before Climategate). I had noticed the term “Briffa bodge” in a preprint of Briffa and Melvin 2008 2011 (see here), where it was used to describe a “very artificial correction” to Briffa’s widely used Tornetrask chronology as follows:

Briffa et al. (1992) ‘corrected’ this apparent anomaly by fitting a line through the residuals of actual minus estimated ring widths, derived from a regression using the density data over the period 501–1750 as the predictor variable, and then removing the recent apparent decline in the density chronology by adding the fitted straight line values (with the sign reversed) to the chronology data for 1750–1980. This ‘correction’ has been termed the ‘Briffa bodge’ (Stahle, personal communication)!

Bodging of the Tornetrask chronology had been discussed in much earlier CA posts – e.g. in March 2005 here and again here.

The term “bodge” also occurs in Climategate correspondence, as pointed out by Jeff Id on December 1, 2009 here.

In July 1999, Vaganov et al (Nature 1999) had attempted to explain the divergence problem in terms of later snowfall (an explanation that would seem to require caution in respect to the interpretation of earlier periods.) On July 14, 1999, Ed Cook wrote Briffa as follows:

Hi Keith,
What is your take on the Vagonov et al. paper concerning the influence of snowfall and melt timing on tree growth in Siberia? Frankly, I can’t believe it was published as is. It is amazinglly thin on details. Isn’t Sob the same site as your Polar Urals site? If so, why is the Sob response window so radically shorter then the ones you identified in your Nature paper for both density and ring width? I notice that they used Berezovo instead of Salekhard, which is much closer according to the map. Is that
because daily data were only available for the Berezovo? Also, there is no evidence for a decline or loss of temperature response in your data in the post-1950s (I assume that you didn’t apply a bodge here). This fully contradicts their claims, although I do admit that such an effect might be happening in some places.

Cheers,
Ed

See here for the response.

I raised the Briffa bodge as an issue in my submission the Briffa bodge to the Parliamentary Committee and Muir Russell as an example of “data manipulation”.

Although Muir Russell expressed disinterest in opining on the proxy issues that dominated the Climategate dossier, they reluctantly expressed an opinion on Briffa’s adjustment of the Tornetrask chronology, agreeing that the bodge was indeed “ad hoc”, but found (without giving any evidence) that there was nothing “unusual about this type of procedure”. While I presume that this reassurance was intended to comfort his audience, I wonder whether readers should in fact be comforted by this observation.

Muir Russell condemned critics for even questioning the Briffa bodge, stating that it was “unreasonable that this issue, pertaining to a publication in 1992, should continue to be misrepresented widely to imply some sort of wrongdoing or sloppy science”.

I’ll review the long backstory today.

Continue reading →

By Stephen McIntyre | Posted in Uncategorized | Tagged , , , , | Comments (97)

Provenance of the Briffa File in the Jones 1998 Archive

Mar 29, 2011 – 2:40 PM

Recently I noticed that there was an (otherwise digitally unavailable) version of the Briffa reconstruction in a second (undocumented) sheet attached (the worksheet tab denoted “Science3”) to the Jones et al 1998 archive at NCDC (see here). Using this version, I was able to replicate graphics that had hitherto been impenetrable. Although the spreadsheet itself is undocumented, it is possible to demonstrate that it is a version of the Briffa reconstruction using the age-banded method later described in Briffa et al 2001 (but without the questionable use of principal components in the latter publication).

There are four other versions of the Briffa reconstruction available digitally. The archive for Briffa et al (Nature 1998) – see here – contains two slightly different normalized versions (NHD1, NHD2) and a temperature reconstruction NHLMT which is a linear transformation of NHD1). These reconstructions were made using Hugershoff standardization. The archive for Briffa et al (JGR 2001) – see here – contains an age-banded version constructed with inverse regression on principal components after deletion of post-1960 values. A version of this data before the deletion is in the Climategate emails here (Oct 5, 1999).

Briffa and Osborn 1999
The version in the Jones et al 1998 spreadsheet was used in the Briffa and Osborn 1999 graphic using a gaussian smooth after deletion of pre-1550 and post-1960 values. The comparison is shown in a blink-graphic below; the match is so precise that it precludes any other version being the source of the Briffa and Osborn graphic.

Figure 1. Blink graph showing the provenance of the Briffa version in Briffa and Osborn 1999. Click on figure to see.

Matching the spreadsheet version (Science3) to Briffa and Osborn 1999 (Science) provides important information on the spreadsheet version. The caption to Briffa and Osborn 1999 stated that the Briffa version in the illustration was from Briffa et al 1998 (393 – volcanic), “processed to retain low-frequency signals”.

Comparison of NH temperature reconstructions, all recalibrated with linear regression against the 1881-1960 mean April-September instrumental temperatures averaged over land areas north of 20ºN. All series have been smoothed with a 50-year Gaussian-weighted filter and are anomalies from the 1961-90 mean…. northern NH tree-ring densities [1550-1960, from (3 – [Briffa et al 1998(393)]), processed to retain low-frequency signals] are in pale blue …

The Appendix to Briffa et al 2001 stated that the method used in Briffa and Osborn 1999 to “retain low-frequency signals” was age-banded standardization (described in the text of Briffa et al 2001):

One curve was produced by performing the age-banding procedure on all chronologies in the data set and by using an unweighted mean of all banded series from all locations. This is similar to the curve from 1650-1960 [sic – should be 1550-1960] presented by Briffa and Osborn 1999 (although we have since made very minor modifications to the age-banding procedure and the input data set). All other curves in Figure 4 were obtained by prior averaging of the age-banded density series into the nine subregions (as defined by Figure 1).

IPCC TAR First Order Draft
The Briffa version in the Science3 tab of the spreadsheet was also used in the spaghetti graph of the IPCC TAR First Order Draft, produced in the wake of the IPCC Lead Authors meeting in Arusha, Tanzania, only two months before the notorious trick email. This helps to further clarify both the provenance of the Briffa version in the Jones et al 1998 spreadsheet and the interesting Climategate emails between the Arusha meeting and the trick email.

The blink graph below demonstrates the use of the Briffa version from the Jones et al 1998 spreadsheet – this time truncated to 1625- 1950 or so. The emulation used Butterworth smoothing (in Mann style). (The caption is inaccurate on this and a couple of other points.)


Figure 2. Blink graph of IPCC TAR FOD Figure 2-25 excerpt illustrating provenance of Briffa reconstruction from the version in the Jones et al 1998 spreadsheet. Click on figure to see.

The original caption stated:

Fig.2.25: Comparison of warm-season (Jones et al, 1998) and annual mean (Mann et al, 1998;1999a) multiproxy-based and warm season tree-ring based (Briffa et al, 1998) millennial Northern Hemisphere temperature reconstructions. The recent instrumental annual mean Northern Hemiphere temperature record is shown for comparison. Also shown is an extratropical sampling of the Mann et al (1998) temperature pattern reconstructions more directly comparable in its latitudinal sampling emphasis to the Jones et al series. The self-consistently estimated two standard error limits (shaded region) for the smoothed Mann et al (1999a) series are shown. The horizontal dashed (zero) line denotes the 1961 1990 reference period mean temperature. All the series were smoothed with a 50 year Hamming-weights lowpass filter, with boundary constraints imposed by padding the series with its mean values during the first and last 25 years.

The graphic itself shows that it was prepared by Ian Macadam on Sept 27, 1999 – only a few days after the fevered exchange among Mann, Briffa, Jones and Coordinating Lead Authors Chris Folland of the UK Met Office and Tom Karl of NOAA. Ian Macadam was a UK Met Office employee, presumably reporting to Folland, who turns up in a few Climategate emails. Mann mentioned him on Sep 22, 1999, a few days before the FOD graphic as follows:

I am perfectly amenable to keeping Keith’s series in the plot, and can ask Ian Macadam (Chris [Folland]?) to add it to the plot he has been preparing (nobody liked my own color/plotting conventions so I’ve given up doing this myself). The key thing is making sure the series are vertically aligned in a reasonable way. I had been using the entire 20th century, but in the case of Keith’s, we need to align the first half of the 20th century w/ the corresponding mean values of the other series, due to the late 20th century decline.

It’s interesting that the very first incident of hide-the-decline in IPCC literature was in a graphic prepared by the UK Met Office.

The following day (Sep 23 – 139. 0938108054.txt), in an email addressed to Jones (also sent to Folland, Briffa and Karl), Mann notes that Phil Jones had sent him the Briffa version used in the FOD spaghetti graph – one that differed from the version in the published article – something Mann described as “tenuous”:

Thanks for your comments Phil, ….

I am definitely using the version of the Briffa et al series you sent in which Keith had restandardized to retain *more* low-frequency variability relative to the one shown by Briffa et al (1998). So already, the reconstruction I’m using is one-step removed from the published series (as far as I know!) and that makes our use of even this series a bit tenuous in my mind, but I’m happy to do it and let the reviewers tell us if they see any problem.

Mann also refers here (for the first time) to yet another forthcoming version. This version – age-banding with principal components – was sent to him by Osborn on Oct 5, 1999, too late for inclusion in the FOD, but used in the final graphic. It was eventually published in Briffa et al 2001. Mann:

If I understand you correctly, there is yet a new version of this series that is two steps removed from Briffa et al (1998)? Frankly, at this stage I think we have to go w/ what we have (please see Ian Macadam’s plot when it is available–I think the story it tells isn’t all that bad, actually) for the time being. Things as you say will change following review anyways.

Please check out the data here ASAP:
ftp://eclogite.geo.umass.edu/pub/mann/IPCC/MILLENNIUM/
This directory has all the series, aligned as I described to have
a 1961-90 base climatology (or in the case of your series, a pseudo 1961-90 base climatology achieved by actually matching the mean of your series and the instrumental record over the interval 1931-60 interval). These are the data that Ian Macadam is hopefully presently plotting up, and I don’t think the discrepancies between the different series are as bad as we percieved earlier (other than the late 19th century where
you are somewhat on the warm side relative to the rest). Please confirm ASAP that we have the right version of the series (note, these have all been 40 year lowpassed)…

Perhaps one of the reasons why the “discrepancies between the different series” were not “as bad as we percieved earlier” was that the FOD graphic deleted Briffa values prior to 1625. If the full Briffa version were used, it would have, of course, yielded the same discrepancy as the deleted portions of the Briffa and Osborn 1999 spaghetti graph.

This subsequent curve used almost exactly the same data, but the result was much closer to the Mann/Jones reconstructions, as noted in Phil Jones’ reply (141. 0938121656.txt):

One important aspect Keith will address is whether you’re using the latest Briffa et al curve. We know you’re not but the one with the greater low frequency and therefore much better chance of looking much better with the other two series, isn’t yet published. We know it looks better in plots we have here.

Unfortunately, Mann’s archive at the University of Massachusetts has been deleted, including the data in ftp://eclogite.geo.umass.edu/pub/mann/IPCC/MILLENNIUM/.

While the information on the provenance of the Briffa version in the Jones et al 1998 archive has to be pieced together, the net result is, in my opinion, a convincing demonstration that the Briffa version in the Science3 tab of the NOAA spreadsheet is an age-banded version constructed from a network virtually identical to the Briffa 2001 network, but without using principal components, and that, after truncation, this version was used in Briffa and Osborn 1999 and (later that year) in the IPCC TAR FOD spaghetti graph.

By Stephen McIntyre | Posted in Uncategorized | Tagged , , | Comments (13)

Keith’s Science Trick, Mike’s Nature Trick and Phil’s Combo

Mar 29, 2011 – 9:36 AM

In a recent post commenting on Rich Muller’s lecture of March 19, 2011 (here) – of which the Climategate portion is more or less the same as his Oct 14, 2010 lecture (online here), John Cook observes:

It’s clear that “Mike’s Nature trick” is quite separate to Keith Briffa’s “hide the decline”. Muller has taken different sections of Phil Jone’s emails and morphed them into a single phrase. To understand how this is a misleading characterisation, it’s helpful to examine exactly what “Mike’s Nature trick” and “hide the decline” refer to.

I agree that Mike’s Nature trick is different from what can perhaps call “Keith’s Science trick”. Jones’ WMO diagram is an unholy combination of the two tricks. The taxonomy of the various tricks has been the subject of many CA posts.

Mike’s Nature Trick
Mike’s Nature Trick was originally diagnosed by CA reader UC here and expounded in greater length (with Matlab code here ). It consists of the following elements:

1. A digital splice of proxy data up to 1980 with instrumental data to 1995 (MBH98), lengthened to 1998 (MBH99).
2. Smoothing with a Butterworth filter of 50 years in MBH98 (MBH99- 40 years) after padding with the mean instrumental value in the calibration period (0) for 100 years.
3. Discarding all values of the smooth after the end of the proxy period.

The splicing of instrumental data with proxy data prior to smoothing has been established by UC beyond any doubt (as will be reviewed below.)

Nonetheless, both Mann and Gavin Schmidt have vehemently denied such a splice.
Mann and Schmidt have denied the use of the method – see UC’s chronology here.

Some of their vehemence undoubtedly arises from Mann’s original and notorious denial of digital splicing of proxy and instrumental records as follows:

No researchers in this field have ever, to our knowledge, “grafted the thermometer record onto” any reconstruction. It is somewhat disappointing to find this specious claim (which we usually find originating from industry-funded climate disinformation websites) appearing in this forum. Most proxy reconstructions end somewhere around 1980, for the reasons discussed above. Often, as in the comparisons we show on this site, the instrumental record (which extends to present) is shown along with the reconstructions, and clearly distinguished from them (e.g. highlighted in red as here).

In the wake of Climategate, Gavin Schmidt staunchly denied digital splicing e.g. here A reader had written in, accurately observing that Mann had spliced instrumental values prior to smoothing:

I don’t think the Nature trick has been described adequately in your post. It isn’t just graphing the instrument record for comparison, but graphing it to ‘hide the decline.’ They didn’t just cut off the proxy value and add on the instrument record from 1961 on; they used the instrument temperature values to calculate smoothed average value for earlier years as well. That is the ‘trick,’ to let the instrument record replace actual values of the data that are lower than you want.

Gavin responded with the disinformation that has become standard:

[Response: This has nothing to do with Mann’s Nature article. The 50-year smooth in figure 5b is only of the reconstruction, not the instrumental data. – gavin]

First, here is the relevant figure from MBH99. Note the closing portion of the smoothed reconstruction with a sort of upside-down S-shape at the end.

Next, on the left is a blowup of the latter part of the above graph.

The overprinted yellow line is a virtually exact emulation of the MBH smooth series: this was obtained by splicing instrumental data from 1981 to 1998 with proxy data up to 1980, followed by truncation after smoothing. (Here to about 1953.) On the right is a graphic also showing the smooth following Mann’s style without splicing instrumental data after 1980.

Left – Excerpt from MBH99 Figure 3 showing emulation of Mannian smooth with spliced instrumental record. Right – also showing (black) the smooth without an instrumental splice.

The trick is clearer in MBH99. The graphic in MBH98 (Nature) is much muddier and doesn’t show the trick as clearly. First, here is the (rather muddy) graphic from MBH98 showing the smooth:

Next, on the left is a blowup of the latter part of the above graph. The overprinted yellow line is an exact emulation of the MBH smooth obtained by splicing instrumental data after 1980 with proxy data up to 1980, followed by truncation after smoothing. (Here to about 1953.) If Mann had not spliced instrumental data after 1980, the smoothed series (following his methodology) would have looked like the version on the right – the orange line showing the result without the instrumental splice.

Although Mann and others have regularly described his “Nature” trick as nothing more than plotting both instrumental and reconstruction data in the same graphic, the “trick” was more than that: it was, as shown above, the splicing of instrumental data with proxy data prior to smoothing. On one occasion however, Mann implictly conceded the Climate Audit exegesis of his Nature trick, stating in an inline comment at realclimate as follows:

In some earlier work though (Mann et al, 1999), the boundary condition for the smoothed curve (at 1980) was determined by padding with the mean of the subsequent data (taken from the instrumental record).

From the precision of the emulation, it appears certain to me that the padding was by the instrumental data (rather than the mean of the subsequent data), but either method involves padding with instrumental data.

Far more serious than “Mike’s Nature trick” is Keith’s Science Trick – the deletion of adverse data (a trick that Mann supported as a Lead Author in IPCC TAR.)
[Note – this section was somewhat rewritten on Sep 5, 2012 adding the figure excerpts in response to a request for clarification of the various tricks.]

Keith’s Science Trick
“Keith’s Science Trick” is first used in May 1999 in Briffa and Osborn (Science 1999) and Jones et al (Rev Geophys 1999). It is nothing more than the deletion of data to hide the decline. The deletion of adverse data to hide the decline was first reported at CA in 2005 here in connection with IPCC TAR spaghetti graph.

In the wake of Climategate, prior incidents of hiding the decline were discussed, including recent analyses of Briffa and Osborn (Science 1999) e.g. here. Following is a graphic showing Keith’s Science Trick – the deletion of adverse data – a practice continued in subsequent spaghetti graphs, including those in IPCC TAR and IPCC AR4.


Figure 2. Annotated version of Briffa and Osborn Science 1999 Figure 1 – see recent posts for derivation.

The Briffa Bodge
The Climategate documents included programs by Tim Osborn and Ian Harris, describing “fudge factors” or “very artificial corrections for decline” e.g. http://di2.nu/foia/harris-tree/briffa_sep98_e.pro. In a submission to the UK Parliamentary Committee, Tim Osborn denied that he had used ‘artificial corrections for [the] decline” in his published articles. While Osborn himself may not have used “artificial corrections”, Osborn carefully did not comment on whether other CRU scientists had used such “artificial corrections”. Briffa’s Tornetrask reconstruction (prominent in several multiproxy reconstructions) had received precisely this sort of adjustment – called the “Briffa bodge”.

In the narrowest sense, Osborn did not use the Briffa bodge (or similar artificial correction) to hide the decline in his articles published after September 1998 (the date of the annotation in the program briffa_sep98_e.pro.) Instead, CRU used Keith’s Science Trick – the deletion of data – to hide the decline.

Phil’s Combo Trick
Phil’s Combo Trick, as others e.g. Hu McCulloch have observed, is a combination of Keith’s Science Trick and Mike’s Nature Trick, leading to a particularly grotesque result.

Values of the Briffa reconstruction were deleted after 1960 (Keith’s Science Trick) and spliced with instrumental data prior to smoothing (Mike’s Nature Trick). But, unlike Mann, Jones didn’t peel the smooth back to the end date of the proxy data.

Cook
Cook’s article has a considerable amount of disinformation about proxy reconstructions, not all of which will be canvassed in this note.

Cook observes:

There is no “decline” in Mann’s reconstructions. As we shall examine shortly, the source of “the decline” come from temperature reconstructions calculated from tree-ring density at high northern latitudes (Briffa 1998). There are very few of these in Mann’s proxy data and hence his reconstructions never required removal of any declining tree-ring density.

As is well known, the long portion of the MBH98-99 reconstruction is little more than an alter ego for bristlecone chronologies and did not “require” removal of the decline. However, Mann et al 2008 contained many (over 90) Briffa MXD series. They deleted actual data after 1960 and replaced it with RegEM data – with the spliced data being used to supposedly demonstrate the statistical significance of the proxy population.

I disagree with Cook’s assertion that the decline was properly disclosed in spaghetti graphs in either IPCC or academic literature. The decline itself was indeed reported in early Briffa articles – indeed, I used these articles to first detect the deletion of adverse data in IPCC TAR – see here.

However, the adverse data was deleted in all spaghetti graphs comparing reconstructions in academic literature and in IPCC reports to policy-makers and the public. This practice should have been disowned by the climate community long ago.

I agree with Cook’s closing paragraph:

Thus it’s clear that “Mike’s Nature trick” has nothing to do with Briffa’s “decline”. There is no “decline” in Mann et al 1998 and Mann et al 1999. To conflate two separate techniques via the phrase “Mike’s Nature trick to hide the decline” is adding to the glut of ‘Climategate’ misinformation.

Yes, indeed. Mike’s Nature Trick and Keith’s Science Trick are two different tricks. Neither practice should be condoned. If I were judging the two as a sort of reverse beauty contest, I would rank Keith’s Science Trick as the most unacceptable.

By Stephen McIntyre | Posted in Uncategorized | Tagged , , | Comments (69)

Hide the Decline: Sciencemag # 3

Mar 23, 2011 – 5:57 AM

The day before yesterday, I reported that Briffa and Osborn (Science 1999) had not just deleted the post=1960 decline (see also CA here), but had deleted the pre-1550 portion as well – the deletions contributing to an unwarranted rhetorical impression of consistency between the reconstructions, an impression that was capitalized upon in the commentary in the running text of Briffa and Osborn 1999.


Figure 1. Annotated version of Briffa and Osborn 1999 Figure 1. See here and here for derivation.

The dossier of computer programs in the Climategate documents is far from complete – programs from Tim Osborn and Ian Harris are in the dossier, but not programs from either Keith Briffa or Phil Jones.

In the directory osborn-tree6, the program science99_fig1.pro both by its name and contents appears to be the program that was used to produce the figure in Briffa and Osborn(Science 1999) – see here for example, though the program in the Climategate zip file is not dated until Feb 16, 2000, about 9 months after the publication of the article.

But there’s an apparent curious inconsistency between the program in Osborn’s archive and the figure as published in Science (Update Mar 23 pm – reader PaulM has now traced the implementation of the deletion of pre-1550 back further).. Here’s the section of code in which the Briffa reconstruction is retrieved:

2: begin ; Age-banded MXD
alltit=”Age-banded density NH growing-season reconstruction”
; Period to consider
perst=1402
peren=1960
fac=0.0 ; do not smooth it any further!!!
; restore,filename=’../treeharry/densadj_all(330).idlsave’
; timey=x
; ; CONVERSION FACTORS FOR AGE-BANDED MXD, BY REGRESSION ON INSTR.
; ts=densadj*0.156525 ; converts it from density to temperature anom
timey=newagetime
ts=newagets
kl=where((timey ge perst) and (timey le peren),nyr)
timey=timey(kl)
ts=ts(kl)
; ts=ts(kl)-0.140369 ; to convert it oC wrt 1961-90
end

The start period for the reconstruction in the code is 1402 (the start of the magenta portion), rather than 1550 – the start of the Briffa version in the actual graphic.

It’s therefore evident that they had, at one time, plotted the Science 1999 spaghetti graph showing data before 1550, but elected to delete the pre-1550 data as well as the post-1960 data. I presume that there is another version of this program (with the corresponding line reading perst=1550) that was used to generate the figure in the Science article. It’s odd that it isn’t in the Osborn archive.

Update Mar 23 pm: – PaluM observes that the deletion of the pre-1550 portion can be accomplished through the parameter yrmxd, which proves to come from either the directory bandtempNHsm50_calmultipcr.idlsave or bandtempNHsm50_calmultipcr_NSIBhug.idlsave.

There is a parameter timey that is used to shorten the interval in the line

kl=where((timey ge perst) and (timey le peren),nyr)

This parameter timey is set to newagetime which is set to yrmxd in the second line of the code.

So to get the figure truncated to 1550, or any other start date you like, you can just run this code as it is with yrmxd set to whatever number you want, depending on how much data you want to hide from the reader. It is not really a code inconsistency.

The truncated time period newagetime is obtained in the lines:

[24] ;restore,filename=’bandtempNHsm50_calmultipcr.idlsave’
[25] ;restore,filename=’bandtempNHsm50_calmultipcr_NSIBhug.idlsave’
[26] ; Gets: nyr,nhtit,yrmxd,prednh,fullnh,predse

The directory bandtempNHsm50_calmultipcr_NSIBhug.idlsave results from a bodge that is mentioned in one of the programs – the age-banded NSIB series did not accord with expectations in some respect and, in some applications, the Hugershoff version was substituted for the age-banded version – as presumably in the directory referred to here.

By Stephen McIntyre | Posted in Uncategorized | Tagged , , | Comments (217)

Hide the Decline – the Other Deletion

Mar 21, 2011 – 12:19 PM

I recently re-visited an article in Science (Briffa and Osborn 1999), that, together with Jones et al 1999 (Rev Geophys), were the first bites of the poison apple of hide-the-decline. I observed that key conclusions in Briffa and Osborn 1999 depended on the rhetorical effect of deleting the decline from their spaghetti graph. Continue reading →

By Stephen McIntyre | Posted in Uncategorized | Comments (89)