The second image below is, in my opinion, one of the most disquieting images ever presented at Climate Audit.
Two posts ago, I observed that the number of cores used in the most recent portion of the Yamal archive at CRU was implausibly low. There were only 10 cores in 1990 versus 65 cores in 1990 in the Polar Urals archive and 110 cores in the Avam-Taimyr archive. These cores were picked from a larger population – measurements from the larger population remain unavailable.
One post ago, I observed that Briffa had supplemented the Taimyr data set (which had a pronounced 20th century divergence problem) not just with the Sidorova et al 2007 data from Avam referenced in Briffa et al 2008, but with a Schweingruber data set from Balschaya Kamenka (russ124w), also located over 400 km from Taimyr.
Given this precedent, I examined the ITRDB data set for potential measurement data from Yamal that could be used to supplement the obviously deficient recent portion of the CRU archive (along the lines of Brifffa’s supplementing the Taimyr data set.) Hantemirov and Shiyatov 2002 describe the Yamal location as follows:
The systematic collection of subfossil wood samples was begun, in 1982, in the basins of the Khadytayakha, Yadayakhodyyakha and Tanlovayakha rivers in southern Yamal in the region located between 67°00 and 67°50 N and 68°30 and 71°00 E (Figure 1). These rivers flow from the north to the south; hence, no driftwood can be brought from the adjacent southern territories At the present time, the upper reaches of these rivers are devoid of trees; larch and spruce-birch-larch thin forests are located mainly in valley bottoms in the middle and lower reaches.
Sure enough, there was a Schweingruber series that fell squarely within the Yamal area – indeed on the first named Khadyta River – russ035w located at 67 12N 69 50Eurl . This data set had 34 cores, nearly 3 times more than the 12 cores selected into the CRU archive. Regardless of the principles for the selection of the 12 CRU cores, one would certainly hope to obtain a similar-looking RCS chronology using the Schweingruber population for living trees in lieu of the selection by CRU (or whoever).
As a sensitivity test, I constructed a variation on the CRU data set, removing the 12 selected cores and replacing them with the 34 cores from the Schweingruber Yamal sample. As shown below, this resulted in a substantial expansion of the data set in the 19th and 20th centuries and a modest decline in the 18th century. (Hantemirov and Shiyatov 2002 had reported a selection of long cores of 200-400 years; while the CRU archive does not appear to be the precisely the same as the unavailable Hantemirov and Shiyatov 2002 archive, it does appear to be related. This pattern of change indicates that the age of the CRU cores is systematically higher than the age of the Schweingruber cores.)
Figure 1. Comparison of core count. Black – variation with Schweingruber instead of CRU; red- archived version with 12 picked cores.
The next graphic compares the RCS chronologies from the two slightly different data sets: red – the RCS chronology calculated from the CRU archive (with the 12 picked cores); black – the RCS chronology calculated using the Schweingruber Yamal sample of living trees instead of the 12 picked trees used in the CRU archive [leaving the rest of the data set unchanged i.e. all the subfossil data prior to the 19th century]. The difference is breathtaking.
Figure 2. A comparison of Yamal RCS chronologies. red – as archived with 12 picked cores; black – including Schweingruber’s Khadyta River, Yamal (russ035w) archive and excluding 12 picked cores. Both smoothed with 21-year gaussian smooth. y-axis is in dimensionless chronology units centered on 1 (as are subsequent graphs (but represent age-adjusted ring width). [Amended Sep 28 6 pm. Replaces url]
Finally, here is another graphic showing the same two RCS chronologies, but adding in an RCS chronology on the merged data set obtained by appending the Schweingruber population to the CRU archive – this time retaining the 12 cores. Unsurprisingly this is in between the other two versions, but most importantly it has no HS.
Figure 3. Also showing merged version up to 1990. (After 1990, there is only the few CRU cores and it tracks the CRU version.) [Amended Sep 28 6 pm. Replaces url ]
I hardly know where to begin in terms of commentary on this difference.
The Yamal chronology has always been an exception to the large-scale “Divergence Problem” that characterizes northern forests. However, using the Schweingruber population instead of the 12 picked cores, this chronology also has a “divergence problem” – not just between ring widths and temperature, but between the two versions.
Perhaps there’s some reason why Schweingruber’s Khadyta River, Yamal larch sample should not be included with the Yamal subfossil data. But given the use of a similar Schweingruber data set in combination with the Taimyr data (in a case where it’s much further away), it’s very hard to think up a valid reason for excluding Khadyta River, while including the Taimyr supplement.
Perhaps the difference between the two versions is related to different aging patterns in the Schweingruber population as compared to the CRU population. The CRU population consists, on average, of older trees than the Schweingruber population. It is highly possible and even probable that the CRU selection is derived from a prior selection of old trees described in Hantemirov and Shiyatov 2002 as follows:
In one approach to constructing a mean chronology, 224 individual series of subfossil larches were selected. These were the longest and most sensitive series, where sensitivity is measured by the magnitude of interannual variability. These data were supplemented by the addition of 17 ring-width series, from 200–400 year old living larches.
The subfossil collection does not have the same bias towards older trees. Perhaps the biased selection of older trees an unintentional bias, when combined with the RCS method. This bias would not have similarly affected the “corridor method” used by Hantemirov and Shiyatov themselves, since this method which did not preserve centennial-scale variability and Hantemirov and Shiyatov would not have been concerned about potential bias introduced by how their cores were selected on a RCS chronology method that they themselves were not using.
Briffa’s own caveats on RCS methodology warn against inhomogeneities, but, notwithstanding these warnings, his initial use of this subset in Briffa 2000 may well have been done without fully thinking through the very limited size and potential unrepresentativeness of the 12 cores. Briffa 2000 presented this chronology in passing and it was never properly published in any journal article. However, as CA readers know, the resulting Yamal chronology with its enormous HS blade was like crack cocaine for paleoclimatologists and got used in virtually every subsequent study, including, most recently, Kaufman et al 2009.
As CA readers also know, until recently, CRU staunchly refused to provide the measurement data used in Briffa’s Yamal reconstruction. Science(mag) acquiesced in this refusal in connection with Osborn and Briffa 2006. While the Yamal chronology was used in a Science article, it originated with Briffa 2000 and Science(mag) took the position that the previous journal (which had a different data policy) had jurisdiction. Briffa used the chronology Briffa et al (Phil Trans B, 2008) and the Phil Trans editors finally seized the nettle, requiring Briffa to archive the data. As noted before, Briffa asked for an extension and, when I checked earlier this year, the Yamal measurement data remained unarchived. A few days ago, I noticed that the Yamal data was finally placed online. With the information finally available, this analysis has only taken a few days. [Update: the Yamal measurement data used in Briffa 2000 proved to be identical to the measurement data used in Hantemirov and Shiyatov 2002, which I had obtained from Hantemirov in 2004, though this had not been stated anywhere. This was the very first look at the Taimyr dataset and it was, in fact, examining the Taimyr procedure that prompted this post.]
If the non-robustness observed here prove out (and I’ve provided a generating script), this will have an important impact on many multiproxy studies that have relied on this study. Studies illustrated in the IPCC AR4 spaghetti graph, Wikipedia spaghetti graph or NAS Panel spaghetti graph (consult them for bibliographic refs) that use the Yamal proxy include: Briffa 2000; Mann and Jones 2003; Jones and Mann 2004; Moberg et al 2005; D’Arrigo et al 2006; Osborn and Briffa 2006; Hegerl et al 2007, plus more recently Briffa et al 2008, Kaufman et al 2009. (Note that spaghetti graph studies not included in the above list all employ strip bark bristlecone pines – some use both.)
Update: Sep 30: Here’s a blow-up of Figure 3 above, from 1850 on. Legend as in Figure 3. The “combined” information is shown to 1990, since post-1990 is, as noted above, limited to the CRU version and, obviously, reverts back to the CRU.
Climate Audit 
415 Comments
######################## ##DOWNLOAD DATA AND FUNCTIONS ############################# source("http://www.climateaudit.info/scripts/utilities.txt") # source("http://www.climateaudit.info/scripts/tree/utilities.treering.txt") f=function(x) filter.combine.pad(x,truncated.gauss.weights(21))[,2] #utility smooth function #Hantemirov at NCDC loc="ftp://ftp.ncdc.noaa.gov/pub/data/paleo/treering/reconstructions/asia/russia/yamal_2002.txt" hant=read.table(loc,skip=57,fill=TRUE,header=TRUE) # Year Recon Chron Samples dim(hant) #4064 4 hant=window( ts(hant[,2:4],start=hant[1,1]),start= -202 ) #minimum in measurement data #Briffa Chronology from CRU loc="http://www.cru.uea.ac.uk/cru/people/melvin/PhilTrans2008/Column.prn" briffa=read.table(loc,skip=1,fill=TRUE) name0=scan(loc,n=8,what="") name0=outer(name0,c("","count"),function(x,y) paste(x,y,sep=".") ) n=nchar(name0[,1]) name0[,1]=substr(name0[,1],1,n-1) names(briffa)=c("year", c(t(name0) ) ) briffa[briffa== -9999]=NA briffa=briffa[,c("year","Yamal.RCS","Yamal.RCS.count")] briffa=briffa[!is.na(briffa[,2]),] briffa=window(ts(briffa[,2:3],start=briffa[1,1]),start=-202) yamal.crn=briffa[,1]/1000 #Yamal measurement data # loc="http://www.cru.uea.ac.uk/cru/people/melvin/PhilTrans2008/YamalADring.raw" # download.file(loc,"temp.dat") # tree=make.rwl_new("temp.dat") # tree$id=factor(tree$id) #252 # tree=agef(tree) #save(tree,file="d:/climate/data/yamal/yamal_cru.rwl.tab") loc="http://www.climateaudit.info/data/tree/rwl/yamal_cru.rwl.tab" #tree download.file(loc,"temp",mode="wb") load("temp") range(tree$year) #-202 1996 yamal=tree dim(yamal) # [1] 40892 4 #yamal$rw=yamal$rw/10 # Sep 28 mean(yamal$rw,na.rm=T) # 61.52668 ################# ## INFO COLLATION ######################### Info=data.frame(id=as.character(levels(tree$id)) ) Info$start= tapply(tree$year,tree$id,min) Info$end= tapply(tree$year,tree$id,max) count=tapply(!is.na(tree$rw),tree$year,sum) Info$max= Info$end-Info$start+1 Info$id=as.character(Info$id) n=nchar(Info$id);temp=n>3 Info$core=""; Info$core[temp]=substr(Info$id[temp],n[temp],n[temp]); Info$test=Info$id; Info$test[temp]=substr(Info$id[temp],1,n[temp]-1) Info$id=gsub("_","L",Info$id) #guess x=substr(Info$id,1,3) temp=!is.na(match(x,c("JAH","POR","YAD")));sum(temp)#12 Info$site=NA; Info$site[temp]=x[temp] Info$site[!temp]=substr(Info$id[!temp],1,1); Info$n=1;Info$n[temp]=3 Info$tree=as.numeric(substr(Info$test,Info$n+1,nchar(Info$test))) #Info[order(Info$n,Info$tree),] # Info[order(Info$n,Info$end),] (index3= Info$id[Info$n==3]) #this identified the 12 trees from 1988 on #"JAH141" "JAH162" "POR011" "POR031" "POR051" "POR081" "POR111" "YAD041" "YAD061" "YAD071" YAD081" "YAD121" ################ # ANALYSIS 1: COMPARE COUNTS: http://www.climateaudit.org/?p=7142 ###################### #1. clip the count image from Hantemirov 2002 CA/pdf/tree/hantemirov.2002.holocene.pdf #2. Compare three versions #plot count in CRU archive count.yamal=countf(yamal) year=c(time(count.yamal));N=length(count.yamal) par(mar=c(3,3,2,1)) plot(year,count.yamal,col="grey80",type="l",ylim=c(0,45),yaxs="i") polygon(xy.coords(x=c( year,rev(year)),y=c(count.yamal,rep(0,N))),col="grey80",border=1) title("Yamal Count: CRU Archive") #plot count in H and S version par(mar=c(3,3,2,1)) year=c(time(hant));N=nrow(hant) plot(c(time(hant)),hant[,"Samples"],col="grey80",type="l",ylim=c(0,45),yaxs="i") polygon(xy.coords(x=c( year,rev(year)),y=c(hant[,"Samples"],rep(0,N))),col="grey80",border=1) title("Yamal Count: Hantemirov NCDC") #clip other illustration from HAntemirov and Shiyatov 2002 ################ # ANALYSIS 2: COMPARE CHRONOLOGIES ###################### chron=ts.union(hant=hant[,2],briffa=yamal.crn) Yamal=data.frame(year=c(time(chron)),chron) fm=lm(briffa~hant,data=Yamal[Yamal$year<1800,]);summary(fm) # Multiple R-squared: 0.8164 #correlation 0.908 ################# ## 3. COMPARE COUNT TO URALS ################### download.file("http://www.climateaudit.info/data/esper/pol.rwl.tab","temp.rwl",mode="wb");load("temp.rwl") dim(tree) #157 count.urals=countf(tree) #plot comparison plot(c(time(count.urals)),count.urals,type="l",xlab="",ylab="",xlim=c(600,2005)) lines(c(time(count.yamal)),count.yamal,col=2,lty=3) legend("topleft",fill=2:1,legend=c("Yamal","Polar Urals")) title("Core Counts") ################### ##4. SCHWEINGRUBER russ035 IMPACT #################### #calculate base case RCS emulation using CRU archive of Yamal chron.yamal=RCS.chronology(yamal,method="nls") #takes a little time #show that emulation of CRU archived chronology is accurate par(mar=c(3,3,2,1)) delta=mean(yamal.crn)-mean(chron.yamal$series);delta # -0.04820995 ts.plot(f(yamal.crn)) lines(f(chron.yamal$series)+delta,col=2) legend("topleft",fill=1:2,legend=c("Archived","Emulated") ) title("Yamal RCS Chronology (CRU)") #close match #download russ035w data - identified as being at Yamal download.file("http://www.climateaudit.info/data/tree/russ035w.rwl.tab","temp.rwl",mode="wb"); load("temp.rwl");dim(tree) # 3872 4 # collated from ftp://ftp.ncdc.noaa.gov/pub/data/paleo/treering/measurements/asia/russ035w.rwl russ035=tree info.russ035=data.frame(id=as.character(levels(russ035$id))) #34 cores mean(russ035$rw,na.rm=T) # 70.1419 #make dataset without picks and with Schweingruber temp=!is.na(match(yamal$id,index3)) #the 12 picked series tree=rbind(yamal[!temp,],russ035) length(unique(tree$id)) # [1] 274 #calculate RCS chronology including Schweingruber and excluding Briffa picks: chron.var1 chron.var1=RCS.chronology(tree,method="nls") #Briffa plus Schweingruber tree=rbind(yamal,russ035) chron.var2=RCS.chronology(tree,method="nls") chron.var2=window(chron.var2$series,end=1990) #at count #plot showing counts without 12 picks and with Schweingruber count.r035var1=countf(tree) count.pick=countf(yamal[temp,]) ts.plot(count.r035var1); lines(count.yamal,col=2,lty=3) legend("topleft",fill=1:2, legend=c("Schweingruber Variation","CRU Archive")) title("Yamal Core Counts") #count.comparison.gif: shows more samples in 19th and 20th centuries and a few less in 18th century #shown in CA post #plot the comparison of the two chronologies:VERY IMPORTANT ts.plot(f(chron.var1$series),ylim=c(0,2.8)) lines(f(chron.yamal$series),col=2) legend("topleft",fill=1:2, legend=c("Schweingruber Variation","From Archive")) title("Yamal RCS Chronologies") #yamal/rcs_chronologies.gif; in CA post http://www.climateaudit.org/?p=7168 #plot the comparison of the three chronologies 1850-2000: ts.plot(chron.var1$series,ylim=c(0,3.5),xlim=c(1850,2000)) lines(yamal.crn,col=2) lines(chron.var2,col=3) #lines(chron.yamal$series),col=2) #lines(f(chron.var2),col=3) legend("topleft",fill=1:3, legend=c("Schweingruber Variation","From Archive","Little Bit Pregnant")) title("Yamal RCS Chronologies") #yamal/rcs_chronologies.gif; in CA post http://www.climateaudit.org/?p=7168 #X=ts.union(year=-202:1990,crn1=chron.var1$series,crn2=chron.var2) #write.csv(X,file="d:/climate/data/yamal/september2009.csv")Woah! Excellent agreement in the past with a missing hockey stick in the calibration range.
Very nice work, both in getting the release of the data and quickly pointing out how special it is. I’ll snip the rest while I think about this.
Steve, wow.
Please would you help us slow but willing followers by labelling the axes a bit more clearly – especially if these hot pics are going to be quoted. I mean, isn’t the y-axis in fig.2 a measure of temperature divergence from the zero line “mean”, or at least a simple multiple thereof?
Steve : y-axis is in dimensionless chronology units centered on 1, standard in the industry, as are subsequent graphs (but represent age-adjusted ring width). Temperature doesn’t enter into the data set until these ring width chronologies are statistically related to temperature. The ring width chronologies are the building blocks.
“breathtaking” is one word for it. “Micky-taking” might be another.
Steve, an absolutely magnificant piece of forensic data analysis.
Surely this must be published? And not just on this blog.
I bet right now “The Team” and Realclimate are already working out ways to
discredit this.
Re: Don Keiller (#6),
For the benefit of both supporters and critics, this unblocks a number of analyses that have been stalemated for several years. I’ve already been in contact with Ross on this and we will proceed forthwith.
It looks like they will have to do this without Briffa for (hopefully no longer than) a little while. On Friday, Sep 18, Hu received the following automated email:
CA wishes Briffa a speedy recovery and hopes that he is soon able to lend his considerable energies to future Team enterprises.
Here’s a re-cap of this saga that should make clear the stunning importance of what Steve has found. One point of terminology: a tree ring record from a site is called a chronology, and is made up of tree ring records from individual trees at that site. Multiple tree ring series are combined using standard statistical algorithms that involve detrending and averaging (these methods are not at issue in this thread). A good chronology–good enough for research that is–should have at least 10 trees in it, and typically has much more.
.
1. In a 1995 Nature paper by Briffa, Schweingruber et al., they reported that 1032 was the coldest year of the millennium – right in the middle of the Medieval Warm Period. But the reconstruction depended on 3 short tree ring cores from the Polar Urals whose dating was very problematic. http://www.climateaudit.org/?p=877.
2. In the 1990s, Schweingruber obtained new Polar Urals data with more securely-dated cores for the MWP. Neither Briffa nor Schweingruber published a new Polar Urals chronology using this data. An updated chronology with this data would have yielded a very different picture, namely a warm medieval era and no anomalous 20th century. Rather than using the updated Polar Urals series, Briffa calculated a new chronology from Yamal – one which had an enormous hockey stick shape. After its publication, in virtually every study, Hockey Team members dropped Polar Urals altogether and substituted Briffa’s Yamal series in its place.
http://www.climateaudit.org/?p=528. PS: The exception to this pattern was Esper et al (Science) 2002, which used the combined Polar Urals data. But Esper refused to provide his data. Steve got it in 2006 after extensive quasi-litigation with Science (over 30 email requests and demands).
3. Subsequently, countless studies appeared from the Team that not only used the Yamal data in place of the Polar Urals, but where Yamal had a critical impact on the relative ranking of the 20th century versus the medieval era.
http://www.climateaudit.org/?p=3099
4. Meanwhile Briffa repeatedly refused to release the Yamal measurement data used inhis calculation despite multiple uses of this series at journals that claimed to require data archiving. E.g. http://www.climateaudit.org/?p=542
5. Then one day Briffa et al. published a paper in 2008 using the Yamal series, again without archiving it. However they published in a Phil Tran Royal Soc journal which has strict data sharing rules. Steve got on the case. http://www.climateaudit.org/?p=3266
6. A short time ago, with the help of the journal editors, the data was pried loose and appeared at the CRU web site. http://www.climateaudit.org/?p=7142
7. It turns out that the late 20th century in the Yamal series has only 10 tree ring chronologies after 1990 (5 after 1995), making it too thin a sample to use (according to conventional rules). But the real problem wasn’t that there were only 5-10 late 20th century cores- there must have been a lot more. They were only using a subset of 10 cores as of 1990, but there was no reason to use a small subset. (Had these been randomly selected, this would be a thin sample, but perhaps passable. But it appears that they weren’t randomly selected.)
http://www.climateaudit.org/?p=7142
8. Faced with a sample in the Taymir chronology that likely had 3-4 times as many series as the Yamal chronology, Briffa added in data from other researchers’ samples taken at the Avam site, some 400 km away. He also used data from the Schweingruber sampling program circa 1990, also taken about 400 km from Taymir. Regardless of the merits or otherwise of pooling samples from such disparate locations, this establishes a precedent where Briffa added a Schweingruber site to provide additional samples. This, incidentally, ramped up the hockey-stickness of the (now Avam-) Taymir chronology.
http://www.climateaudit.org/?p=7158
9. Steve thus looked for data from other samples at or near the Yamal site that could have been used to increase the sample size in the Briffa Yamal chronology. He quickly discovered a large set of 34 Schweingruber samples from living trees. Using these instead of the 12 trees in the Briffa (CRU) group that extend to the present yields Figure 2, showing a complete divergence in the 20th century. Thus the Schweingruber data completely contradicts the CRU series. Bear in mind the close collaboration of Schweingruber and Briffa all this time, and their habit of using one another’s data as needed.
10. Combining the CRU and Schweingruber data yields the green line in the 3rd figure above. While it doesn’t go down at the end, neither does it go up, and it yields a medieval era warmer than the present, on the standard interpretation. Thus the key ingredient in a lot of the studies that have been invoked to support the Hockey Stick, namely the Briffa Yamal series (red line above) depends on the influence of a thin subsample of post-1990 chronologies and the exclusion of the (much larger) collection of readily-available Schweingruber data for the same area.
Re: Ross McKitrick (#10),
Thank You, Ross McKitrick for the recapitulation.
It does bring an understanding and appreciation of this moment to the larger readership of this site.
Re: Ross McKitrick (#10),
Thanks a lot for this fine recap. Just one small point:
You say that “Subsequently, countless studies appeared from the Team that not only used the Yamal data in place of the Polar Urals…”
Wouldn’t the number actually be more like 10? Or how many are there?
Steve: Let’s see: Briffa 2000, Mann and Jones 2003; Bradley, Hughes and Diaz 2003; Jones and Mann 2004; Moberg et al 2005; D’Arrigo et al 2006; Osborn and Briffa 2006; Hegerl et al 2007; Briffa et al 2008; Kaufman et al 2009. It’s not in Mann et al 2008 (bristlecones and upside-down Tiljander instead); Esper 2002 (two strip bark foxtail series); Juckes 2007 ( foxtails; plus the original and obsolete Briffa 1995 Polar Urals series: 1032 coldest of the millennium). Any that I’ve forgotten to mention?
Dialing back.
.
Still dialing.
.
One wonders what accounts for the uptick in the cherry-picked samples. Obviously if NAS were to rule, they would say “don’t use them”. But what is it about them? It’s not like they have stripbark as an external cue to avoid them.
.
That divergence is huge.
What is the first year of divergence? 1976?
Re: bender (#14),
Actually Steve, I think I did catch your error first. That’s why I asked above in #14:
You said somewhere it was in the early part of the 20th century and that did not ring true for me from my scan of the raw data.
I imagine that now we know exactly which grouping of 12 CRU cores causes the large uptick, that someone could run 12 series – each one removing a particular core.
Nice work Steve.
What a trainwreck. Somebody should get off the ice and back to the bench.
Very nice work, and persistence, Steve. I understand the significance, and trust that you and Ross will be published quickly and broadly.
Since the team will be struggling to get the injured player back on the bench I think it would be instructive to try to play for their side while they try to clear their heads.
Moshpit suits up for the team.
You have a population of tree core samples. On the assumption that some trees at some times perform as tree momemeters it makes perfect sense to throw out cores that suffer from a divergence problem. Look, when we take cores we do so in the hopes that we are selecting the trees that have been good treemometers over time. There are some selection criteria applied prior to coring, but these criteria obviously are not perfect. Some trees will be selected that have issues that prevent them from being well calibrated treemometers. But the science of dendrology tell us that some trees, some species in some locations under some conditions will function as treemometers. If we merely throw all the samples into the analysis meatgrinder those well calibrated instruments will be lost in a sea of noise. So, we have to select sub samples. And that selection can only be driven by looking at how well the uptick of the 20th century is represented.
Granted this will play havoc with the calculation of Confidence intervals. But the only way to recover the signal is by selecting cores that calibrate well. That entails throwing out cores with divergence problems. The divergence is something that we need to understand more fully, but we should not let our current lack of understanding in that area stand in the way of using the best trees to reconstruct the climate signal. Very simply, the working assumption of core selection is the belief that some cores represent temperature better than others. That belief is not in question. Of course, finding a divergence problem should give us pause. The relationship between temperature and ring growth is complex, but it exists nevertheless.
The best way of understanding it is by segregating those cores that exhibit a good correlation with modern temperatures from those that don’t. As noted above selecting cores according to their modern period correlation statistics will complicate the matter of confidence intervals. And to some it will look like cherry picking. If we want the best estimate of past temperatures it only makes sense to limit our sample to those cores that correlate well with the instrument period. That should be obvious. In the end we may have to walk away from strong claims about the confidence intervals around the estimates around the MWP, but even if we take that path the core of AGW beliefs remain intact.
ok. What’s your best argument for the team position on the core selection problem
Re: steven mosher (#21),
I just promised to dial back and then Mosher you go and do this. Actually I think your rendition of the team position is in line with what I have read in the past. And you did it without imposing the b word as in bizarre.
Re: steven mosher (#21), or, more briefly: We assume (“know”) that temperature has a (unprecedented) large recent uptick. We select trees that demonstrate this large recent uptick in ring width and, voilà, we get a hockey stick.
Re: steven mosher (#22),
Steven, surely this is the logical problem with the argument you suggest.
You need to show that some trees in a given area are thermometers for their lifetimes. However what you have proven by the correlation with mercury in modern times is that a few are thermometers for the mercury period and many are not. You cannot conclude from that, that the ones that are for the mercury period are for other periods, unless you exclude the hypothesis that trees move in and out of being thermometers. If this is so, then could you extend your mercury measurements back, you’d end up with a quite different set of trees for the years 1000-1500, for example.
Your logical problem is caused by what you really found with the correlation. You found some correlation, which you explain by an hypothesis about some trees being thermometers for their whole lives, but you also found a large chunk of non-correlation, and this has to be explained too. It is possible they are just bad trees, but there has to be some characteristic to point to to make them bad. Otherwise you have not excluded the possibility that no tree is a thermometer for its whole life, and the ones that correlate only do so for this period.
They could move in and out of correlation by life cycle, by weather variation, by state of climate. It could be lots of things, but as long as you cannot give any explanation, and prove that the trees which are modern thermometers also were ancient thermometers, you have no basis for excluding any.
If Steve’s summary above is correct, neither Esper nor Briffa has supplied any reason for thinking this. The killer is what happens when you add all of the proxies you can find, as in the charts above. The logical problem is how to exclude on a proven basis, and excluding on the basis of non-correlation is not going to do it for you. The argument you supply assumes the only thing you have to explain is inclusion: it isn’t.
Basically until Briffa has given some proof that some trees are thermometers for their whole lives, he has no basis for exclusion or exclusion of any samples. He’d have to throw them all in, and hope that noise would cancel out.
I’ve been trying to think of a medical, or quality control analogy, but its hard. Maybe we are running an inspection sampling program on goods in, and we are worried about integrity of the castings in the centre of the part. We do a 100% sample, and we find something that correlates in about 10% of the sample. There is no obvious distinguishing mark for the ones that correlate and the ones that don’t. Surely this, absent any distinguishing mark, means that we cannot use this test to screen all the ones in the warehouse? Or is this not a valid analogy? It seems to be, absent any distinguishing mark.
Re: steven mosher (#22),
“The best way of understanding it is by segregating those cores that exhibit a good correlation with modern temperatures from those that don’t.”
Mr. Mosher, just when do you believe that you have reduced the sample size significantly enough as to invalidate the study? How can a sample size of 10 represent a good statistical study of any kind, scientific or not? Particularly on a world with a surface area of 5.1 * 10^8 Km^2?
Re: steven mosher (#14),
snip – blog rules prohibit complaining about policy.
I am sure you will be double checking the work so any criticism of it will not get off the track and then publishing it. My comments are dialed back, but not my anticipation of seeing this in publication.
Are there any precedents or conventions for selecting proxy sub-samples on the basis of insturmental correlation and excluding others? Jeff mentioned the Jacoby quote at tAV.
Well, won’t the team argue that the Schweingruber series be excluded because it doesn’t calibrate well to recent temperatures? So, for whatever reason, they are not good thermometers?
One of the quotes that some of you are possibly trying to recall is Esper discussed here :
Re: Steve McIntyre (#30),
At first glance, the problem I have with this is the lack of site documentation. It seems to me that if there IS justification for selection of proxy sub-sets, that each site (selected and excluded) needs to be documented to the nth degree for obvious reasons. It would appear from Steve’s work that the uptick is the exception, and divergence the rule. The reasons for an uptick at an individual site could be due to a local externality such as a change in soil nutrient flux.
Steve,
Absolutely priceless, I know it was wrong of me but when I saw Fig 2, I just laughed out loud. I have got tears in my eyes now. The nerve of it all, are they cheeky or are they cheeky.
Anyway I have a dreadful cold and you have made my day.
I guess the authors will have an explanation. Once the data was archived they must have guessed that they will need one by the time you found it. Hopefully it will be a good one.
Alex
I think one of the things that is disquieting to you, Steve, about that second image is that try as you might, you can’t help publishing information on motive and intent. That image screams its message about intellectual integrity.
=======================================
Here is a plot of the ring widths of the 10 trees in use in Yamal 1990. The majority of these trees (like the Graybill bristlecones) have a prolonged growth pulse (for whatever reason) starting in the 19th century. When a one-size fits all age profile is applied to these particular tries, the relatively vigorous growth becomes monster growth – 8 sigma anomalies in some of them.
One of the problems with dendro chronologies that is not talked about very much is the extraordinary variability from tree to tree. Thus slight changes in sampling protocol or selection can result in huge changes in the average. Doug Larson at Guelph in thinking about cedars on cliffs mentioned the concept of cedars “mining” for nutrients – every so often running across a rich “vein” and thus having a growth pulse. I’m not in a position to try to figure out botanical issues, but the big growth recovery of these trees in the 19th century begs for some process-based explanation. Additionally, in looking at many of these series over the years, it also seems to me that “alpha” trees, ones that live a very long time, sometimes seem to be relatively unaffected by aging and thus rote application of an age-adjustment curve isn’t appropriate. If the sampling process of present subconsciously prefers these alpha trees (as it seems), then it would yield biased results.
But the results here do not depend on such speculation. The results are what they are.
Re: Steve McIntyre (#35), For relatively long-lived trees, a common way to get a growth pulse is for neighboring trees to die. This opens up living space for the remaining trees, which show a growth pulse.
Re: Steve McIntyre (#35), If a tree is a “good” thermometer, then the trees calibrated to temperature in the 20th century should agree in earlier years. There is not much hint of agreement in the linked post graphs.
Re: Steve McIntyre (#35),
This gives the pronounced HS? I bet if you removed YAD061 and POR081 you would wipe out much of the upslope.
Question: How do they calibrate the shorter samples? And how are the different scales resolved? The YAD081 scale is so much larger than any others, that it would dominate a sample like POR111, so that any resulting straight merge would look basically like YAD081.
Also, if divergence is the rule, and uptick the exception then how can it just be assumed that the predominant signal – that of divergence – is merely absent in the uptick proxies leaving a clear temp signal?
Folks, PLEASE do not divert this thread into general complaining about tree rings.
If your comment does not include the words “Briffa” and “Yamal”, hold your peace or find another thread elsewhere. Also please do not try to propose your own bright ideas on how things should be done.
I’m obviously not grasping the huge impact of this post. I thought Rob Wilson told us years ago that they do only retain those chronologies that show good correlation during the calibration period (?). Like I said, I’m sure I’m missing something very important. If the samples diverge in the modern warm area then they are not reliable thermometers…so their MWP/CWP ratio is meaningless, right?
Steve: This is not simply a matter of how the chronology is selected, but how the Yamal chronology itself was constructed. This is not an incidental chronology, but one which impacts many multiproxy studies, including most recently Kaufman. A lot of studies depend on this chronology.
Brilliant work. Especially as you state that: “With the information finally available, this analysis has only taken a few days.”
Do we know when exactly the Yamal data (Briffa version) were archived? How long have they been sitting there waiitng for Steve to uncover them?
Steve: It’s the Team and there’s no change notice. I guess someone could ask Melvin. If he doesn’t answer, we could send an FOI to CRU :). I believe they know how to deal with FOI requests. I know that I checked earlier this year and they weren’t up. So the max is 6 months. I trawl these things from time to time and I’m semi-sure that I’ve checked in the last 3 months. My guess is that the top end is a couple of months and the bottom end a couple of weeks.
Re: bender (#55),
The data appeared before 9th Sept 2009.
Re: Tim Channon (#55),
Based on the Google cache, it appeared on or before Sep 9, 2009 as it was cached that day. The host page was cached on Sep 8, 2009 and contained a link (which might not have been up yet.) Another page http://www.cru.uea.ac.uk/cru/people/melvin/PhilTrans2008/Column.prn was cached on Aug 10, 2009 and wasn’t re-cached on Sep 9, 2009.
I don’t know how Google cache works, but it picks up pages here almost instantaneously. This would argue for a date around Sep 8-9.
Also it’s not unheard of for people to alert me to data that I’m interested in and no one did so in this case. This is slight evidence for a latish date as well.
I wouldn’t be totally surprised if it were archived around Sep 8-9 (nor would I be surprised if it were archived a few months earlier either.
There’s not much point speculating on this. If anyone is exceptionally interested, they can email Melvin and ask him.
Waiting for the “It doesn’t matter” post on RC in 3…2…1…
Steve: Don’t wait. It’s not going to happen. There’s no incentive for Gavin to protect Briffa.
Re: MattN (#32), unfortunately, the RC lying fest has begun. “It doesn’t matter” is actually the theme. Once again, the RealClimateers rush in (like smart sounding fools?)to undo the truth in ther effort to assure th throbbing alarmist masses of their correctness. I do notice, however, that Gavin and his cronies don’t have the backbone to confront you directly here by posting. snip
“Over 30 email requests and demands” (Ross McKitrick, #10): I am stunned. The nub of the problem–I’m saying nothing new here of course–is the nonarchiving of data, including any data NOT used in the calculations of the resultant paper. I am appreciative that S. McIntyre goes on and on about this: Someone has to have the integrity to do so; ultimately, everyone would be helped. The ball is always in the court of the journal editors, as SM avers, and the release of Briffa’s data here is a clear case in point: The Phil Trans editors did the right thing, in rather severe contradistinction to the editors at Nature and Science, who sat by.
Ya know, it would be easy enough for journal editors to charge one peer reviewer per submission to do nothing but check to see that the paper’s authors had archived every single bit of data. That would be his/her sole job. Can’t be too difficult or time-consuming.
Steve: A first step would be even simpler and wouldn’t even require that much work. Ask the authors to submit a form warranting that they’ve archived the data. Most authors would archive the data if they have to submit such a form. I’d do that before asking reviewers to nag.
Steve just doesn’t like cherry pie.
clearly, the statistics is not going to be at issue. Whether a joint schweingruber/yamal series has a critical effect on various reconstructions is also going to be solved quickly.
However, I am not clear from the above if there are biological reasons as to why the two series should not be unified. Is there any relevant detail from the paper ? I appreciate that the Avam-Taimyr joint reconstruction makes a case that it is possible to do a geographically related chronology, but two wrongs do not make a right.
full marks to the editors at Phil Trans.
per
Re: per (#59),
per, I’ve read the paper very carefully and I don’t see any reason why the Schweingruber data should not be incorporated into the Yamal data set. It is squarely within the geographic specification of the Hantemirov and Shiyatov study and is the correct species. The Yamal data set is far smaller than the other data sets as well in the modern portion. In addition to the AVam-Taymir merger, also note the Tornetrask-Finland merger in the third Briffa et al 2008 data set.
Also keep in mind the implausibly small size of the current portion of the Yamal archive. It would be one thing if they had only sampled 10 trees and this is what they got. But they selected 10 trees out of a larger population. Because the selection yields such different results from a nearby population sample, there is a compelling prima facie argument that they’ve made biased picks. This is rebuttable. I would welcome hearing the argument on the other side. I’ve notified one dendro of the issue and requested him to assist in the interpretation of the new data (but am not very hopeful that he will speak up.)
Re: Steve McIntyre (#62),
The Torneträsk-Finland merger is definitely questionable. The climate is much more continental in northern Finland and tree growth there might well be moisture-limited which is unlikely in the quite maritime Torneträsk area.
Re: tty (#82),
About finish tree ring temperature dependency; see eg:
“Analyses showed that the correlations between ring widths and mid-summer (July) temperatures did
not vary signifi cantly as a function of time. Early (June) and late summer (August) mean temperatures
were secondary to mid-summer temperatures in controlling the radial growth”
What do they expect, the bottom line is trees aren’t good thermometers. They are a lot better at being trees.
So after that minor point I wonder, has the Schweingruber series used here ever been employed as a temperature proxy elsewhere or is the team simply going to claim that these are water trees and Steve doesn’t know which sticks to use for thermal divining rods?
Re: Jeff Id (#60),
Jeff, good question. However, it’s one of the series (in its MXD version) in the Briffa MXD network that you recall from Mann et al 2008, but which has a longer history. (The Briffa MXD reconstruction was the series that Briffa truncated in the IPCC 2001 spaghetti diagram).
Go to http://www.cru.uea.ac.uk/~timo/datapages/mxdtrw.htm which provides links to lists of sites in the various Briffa studies. (It took me years to make them make this page.) The webpage says:
The series in question is:
878A Khadyta-River KHADYTLA LASI 69.95 67.13 1782 1990
It’s listed as being used in the following Briffa studies (typically in the MXD version, rather than RW) – the lists below including Briffa et al 2001 and similar studies.
http://www.cru.uea.ac.uk/~timo/datapages/b98sens_site.txt
http://www.cru.uea.ac.uk/~timo/datapages/b98volc_site.txt
http://www.cru.uea.ac.uk/~timo/datapages/b02sens127_site.txt 127 sites
http://www.cru.uea.ac.uk/~timo/datapages/b01abd_site.txt
http://www.cru.uea.ac.uk/~timo/datapages/b02sens341_site.txt
In addition, its MXD version was used in Rutherford et al 2005 (as part of the Briffa MXD network), yielding the gridded MXD series that you enjoyed so much in Mann et al 2008. It would probably have contributed to Mann et al 2008 gridded series 933 (67.5N 67.5 E).
So it will be tough for them to argue that they haven’t used Khadyta River as a temperature proxy.
Caution is warranted. The absence of detailed meta data remains a problem. There is a reason for the upticks, there is a reason for the downticks. That reason might be known but not revealed (e.g. insect damage to some areas, not to others, to give but a guess example). A credible explanation is needed for the upticks just as much as the downticks before an appearance by CRU before the Judge would be sought by this bush lawyer.
This is not to detract from the seriousness of Steve’s findings presented so far. It is superb and persistent forensic work. (It seems that the most fertile ground to audit is that where the resistance is strongest). Remember as well that the correlation of ring properties with temperature has its own noise and I for one am very unsure about the validity of the temperature record. Like many, I’m working on it.
Given my druthers, I’d be trying to crack the CRU temperature problem as well as the dendro problem, because in combination they might tell a stronger story than each alone. There are many other studies that will have to be reexamined because of the emerging dendro problem; there are many also reliant on the temperature record being correct.
Re: Geoff Sherrington (#65),
I agree. It’s hard to believe that the dendros are so silent on the divergence issue. I think they have ideas, but no solid proof yet. It’s hard to believe that no one at LTRR knows what caused the uptick in Graybill’s samples.
.
Did Briffa, in his undergraduate training, take courses in tree physiology and statistics? It’s hard to believe he didn’t know what he was doing in selecting the samples he did.
Steve: it’s hard to say how the Briffa sample reconciles in detail to the subset that Hantemirov and Shiyatov chose for their corridor chronology. Details on this may come out over time.
LOL.
Steve,
Just to make sure the differences you find are not a consequence of how you standardized the raw measurements and generated the RCS tree-ring chronologies (I guess you do that using R), you may want to try and present the results using the “official” dendro software ARSTAN freely available (together with many other great dendro-climate related routines) at the lamont’s tree ring lab website.
Re: giano (#68),
There is no “official” software for RCS.
For a long long time, despite the prevalence of Briffa’s RCS chronologies in Team reconstructions, I was unable to locate any benchmark situations where both a measurement data file was available AND an archived RCS chronology. Either one or the other would be available. In fact, this might also be the first occasion on which both have been available at the same time.
In mathematical terms, RCS is actually simpler than “conventional” standardization. I’ve emulated the archived RCS chronologies for Avam-Taymir and Yamal very closely using my algorithm from the newly available data and am not worried about this aspect of things.
I’ve placed my RCS software online and if anyone on the Team has any fault with it, I’ll take a look at their software and re-implement the results. But my emulations of available data is so close that it’s hard for me to picture a way in which the results would be sensitive to remaining implementation details.
Re: Steve McIntyre (#74),
OK then, you have that point covered. Like many others I think it will be very interesting to see how this issue evolves.
Are these Yamal larch in an alpine situation, or more tundra/bog?
.
And how far away is the nearset Starbucks?
Steve: Tundra
C’mon, it’s simple. Invert the core count and compare with RCS: divergence explained.
Why do the old divergent trees do that? Maybe it’s some variation of a distance to inlet type of local feature. Obviously Off Topic unless you are Geoff, Bender or Mosh… great points all.
Steve
Did these 12 trees happen to occupy another “sweet spot” where they would be sensitive to the “global temperature field”?
I hardly know what to say about the sharp divergence except “Wow” which doesn’t really cover the range of emotions I feel at the moment. I feel like I’ve been sold a large, very expensive piece of jewelry and then when I’ve taken it to a jewelers to be assayed find that its made of glass and base metals.
That’s all I’m sayin’
Show us a peer-reviewed paper on this subject and you might have something. For right now, all it is is noise.
Re: Sue (#87),
We do not need a peer reviewed paper to know when someone has knowingly or unknowingly tried to sell us a pup. After all look at the many peer reviewed papers which have subsequently been shown to be wrong and not just in so called climate science?
The signal is loud and clear there’s something not right. The “gentle pillow” of believe in the hypothesis of anthropogenic global warming may be of comfort to some?
“That aint working that’s the way you do it raise the temperature by a few degrees”
“Money for nothing and the plaudits for free”
Sorry for the broken link above. Maybe sometime I’ll learn those tags. Also, it would have been much more into the spirits of CA to dig out those series and calculate the correlations.
Not sure is this the location?
Not sure is this the location?
http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=+67.13+++69.95&sll=67.209752,69.830132&sspn=0.062106,0.210457&ie=UTF8&ll=67.124354,69.954414&spn=0.062326,0.210457&t=h&z=12
Click to access ACIA_Ch14_Final.pdf
interesting reading.. especially the parts siberian warming in the mwp
How about getting back to us when you have a peer-reviewed paper to present? Right now, all you’re doing is making conclusions without the requisite review by others to document that you’re actually providing some enlightenment.
Re: Ellie (#92),
You do not have to read it. Also Steve is open to review, here and now. As I understand it, his findings so far, are sufficiently well documented above, for others to verify or disprove. Personally I think he is running a brave strategy, and it is fascinating to watch. He tells us what he is going to do, does it, and tells us what he has found. He says that he believes in openness in science and that is how he seems to operate.
I think it is normal to draw conclusions prior to their being reviewed the only unusual thing here is that he is thinking out loud. If you do not enjoy that aspect you do not have to listen.
Alex
Re: Ellie (#92),
“How about getting back to us when you have a peer-reviewed paper to present?”
Subtext:
“Preferably after Copenhagen. Meantime, please shut up.”
Re: Ellie (#92),
Probably would have had one years ago if Briffa had archived his data when he should have. While you’re here, any thoughts on the non-archiving of the data?
Re: Ellie (#92),
Ah yes, “the requisite review.” That would be the kind that doesn’t involve actually checking any calculations. As for scientific review, Briffa’s refusal to release his data lo these many years means that his chronologies, and all the papers re-using them, neatly avoided it, at least until about 12 hours ago. If you understood the story better you would realize that there is more rigorous review unfolding on this page than occurred during the decade-long process of publishing journal articles that used and re-used the Yamal chronology.
Re: Ellie (#92),
Ellie makes a good point here. For those who need to take their lead from peer reviewed papers and comments, and better yet a review process of peer reviewed papers like that of the IPCC, before even starting a general review and analysis process, Ellie is correct. I think it is also comforting for one with a consensus view to allow the selection process of the consensus to work.
What happens at CA, which no one claims replaces the functions of the peer review, is not for everyone – like an Ellie – and is something more for those scientists and laypersons who are willing to dig into these papers and methods on their own. Can these discussion at CA miss some important details? Yes. Can the peer review process miss important details? Yes. And unfortunately with a considerably longer process in correcting the situation than at a good technically oriented and reasonable blog.
Re: Ellie (#92),
Ellie. Let me peer review your comment.
You state that steve all steve is doing is making conclusions. This is factually wrong. He is plotting data. You will see no conclusions about the reasons for exclusion of this data, no conclusions about what it says about the MWP. He is plotting data. It’s the same exercise he ALWAYS DOES. To those of us with data analysis back grounds his approach is absolutely standard. here are the rings collected. here are the various subsets. here is A versus B, etc. We all are experienced with starting our analysis this way. especially when we use OPD ( other peoples data). So you are wrong. I see no substantive conclusions about the data. The data is the data.
WRT the review by others. The authors of these studies, the collectors of these data have ben invited to review steve’s work. They choose for whatever reason to remain silent. Perhaps they have nothing to say. the data is the data. They excluded data that ran counter to their preconception. That perhaps is defensible. They have chosen not to defend it.
Finally, in this field peer review is the problem, not the solution. Steve has no peers when it comes to his combined talents. ( opps I exceeded my one pat per post limit)
Re: Ellie (#63),
Ellie and the girls aloud crowd. Let me tell you something absolutely certain. I had papers pier reviewed in mine time and you quite obviously have not. I would much prefer the process of pier review to the process on view here. This is frightening but at the same time exilerating, fascinating and totally way beyond anything you understand.
As others have said, this is breathtaking. Congratulations to Steve for outstanding work.
What gives this story such a sharp edge is the background. As I understand it (thanks to Ross McKitrick for the summary), Briffa et al refused to reveal their data until finally they were forced to by the scientific journal (credit to them also). After that it took Steve a matter of days to complete an initial analysis, the result of which is, as I said, breathtaking.
.
Steven Mosher is right to act as Devil’s Advocate. His argument is plausible. After all, it’s standard practice to reject data that is ‘obviously wrong’. But how do you determine what is ‘obviously wrong’? If you do this on the basis of your existing beliefs then you’re on a very slippery slope indeed. You might end up depending on reasoning that is more than a little circle-shaped.
.
It seems to me that any analysis like this should not depend on preconceived ideas. The best approach would be to include as many series as possible, rejecting only those with *known* problems. The remaining series will inevitably suffer from many defects, but I would think there’s no fundamental reason why they should all give errors in the same direction. There should be safety in numbers, so that the average result would tend to iron out the defects.
.
This is why I think the third graph is the most significant, because it includes all the series. And the result is far more plausible, without an exaggerated fall or rise in recent decades.
.
I very much hope that this extraordinary work will culminate in an important scientific paper that will make a huge contribution to climate science. And that it will be noticed by the people that matter.
Chris
From a previous post:
I wonder how the author of that quotation is viewing (and Sydney to a brick he is) this post. Interesting? Vicious? Viscous? Something else?
I’m wondering if it should be named “The WOW graph”.
Peer review of this work is in progress, right now. Anyone who finds fault with the above presentation is free to point out the oversight – either here or at any other location which they feel is more accommodating of their own personal view. Clearly, most of us are in no position to judge if the original ‘journal published’ work is any more correct than the work published on this blog – but open discussion will surely help the experts to form a view. If there is a fault with this analysis, I would expect to read more than ‘it is just noise’.
Is it possible to get hands on the remaining 20th century Yamal tree ring data, from which those 10 were obviously cherry-picked? That would be the final proof.
Congratulations Steve on cracking another nut open.
I notice the flurry of activity here has been met with a dignified silence over at RC.
Right now, peer review is ongoing on this thread. Briffa, Mann et al are kindly invited to participate, but somehow I have the idea that they will not participate and hope that this thunderstorm will pass by and die out. In the meantime you are idnly invited to participate in tis peer review or do you, as usual, prefer to attack the messenger instead of the message?
I have to wonder if we are going to see a slowing of the climate related publications to Philosophical Transactions of the Royal Society. Considering their strict policy on data sharing. It might be something worth monitoring.
Well done Mr. McIntyre! – a great/notable refutal, with skilled investigative analyses.
Chris Wright 95 says His argument is plausible. After all, it’s standard practice to reject data that is ‘obviously wrong’
Maybe I’m just an old fashioned scientist but just when did it become standard practice? I have often wished to reject outliers in data sets but scientifically I can only do that if there is a good rationale for so doing. The fact that the data doesn’t fit a preconceived hypothesis does not provide a good rationale, and is often a reason for rejecting the hypothesis. Lots of environmental data is highly variable but this is frequently not ‘noise’ which could be filtered out but a range of other signals that have yet to be identified.
Re: Arthur Dent (#78),
Steve has demanded that various proxies be removed himself in the past.
Steve: I’ve never suggested that dendros do anything other than archive and use all their data. I’ve definitely suggested that things like the Tiljander sediments, stated by the author to have been affected by local bridge building and farming in the recent period, not be used as temperature proxies – particularly upside down to the original publication. There was “a good rationale” for doing so. I’ve consistently argued that all proxies of a given type be considered as a population, rather than cherrypicking in Team style.
Re: slownewsday (#374),
There’s a bit of difference between avoiding invalid proxies (e.g. entire datasets) vs cherry-picking data points or data series within a single data set.
IOW,
Steve doesn’t like the idea that Mann continues to treat data from Europe as if it were in North America (“the rain in Maine falls mainly on the Seine”). European proxies don’t make sense in a NA reconstruction.
Meanwhile, Briffa excluded much of the data available for the Yamal area.
Re: MrPete (#380),
In light of Dr Briffa’s response, I want to update my own comments in this thread. I’ve commented a few times on Briffa’s selection of certain data sets and the impact of those selections.
It seems to me that his motivation for the selection is immaterial to science.
It also seems to me that if his selection was in fact inappropriate, then in one sense it doesn’t matter too much exactly how he got it wrong. If the result is in fact inappropriate cherry pie, then it doesn’t matter if he intended to make berry pie and accidentally opened a can of cherries… or if he was going for cherries all along.
Right now, it would appear that even Dr Briffa cannot substantiate that he’s made something other than cherry pie. His only claim is that cherry pie was not his intent.
If it is NOT a cherry pie, then fine, there’s plenty of humble pie to go around.
🙂
Another possibly relevant reconstruction is
A 900-YEARS LARCH CHRONOLOGY FOR NORTH-WESTERN SIBERIA ON THE BASES OF ARCHAEOLOGICAL WOOD OF THE UST-VOYKAR
SETTLEMENT, MARINA GURSKAYA, GEOCHRONOMETRIA 28 (2007), pp 67-72.
pdf here
This is just south of Yamal. It shows no hockey stick, and is in agreement with the Yamal reconstrution of Hantemirov. Briffa is not cited 🙂
Steve: Interesting reference. One of the ongoing problems with dendro is that different standardization methods yield very different results and so to compare apples and apples you need to compare the data sets with the same method i.e. you need measurement data for both. As noted before, the Hantemirov chronology is done with a method that removes centennial variation and so I place no weight on comparisons to it. Please see the prior thread for my comment on the Hantemirov chronology. If someone can get the data, this would be interesting to follow up on.
Re: PaulM (#79),
Look at that Vokyar chronology in Fig 4A. It shows a Shweingruber Khadyta-like downtick at the end. (But man does that crossdating diagram Fig. 2 look, ummm, interesting.)
As an applied mathematician, I have dealt with large quanties of data from different sources that had to be combined (for orbit determination and trajectory analysis). We would find data from individual sites that were outliers and individual points of data that were outliers and in all cases we discarded them. Dendrology as being done by the team seems to keep the outliers and throw away the consistent data.
Re: LarryT (#125),
Larry, A treemometer is a rare breed. Yes, you could look at it as an outlier. But remember outliers often teach us a lot about the “normal case” see, abductive reasoning
I tried asking a polite question at RealClimate about their response to this thread. Unsurprisingly the comment was instantly “vaporized” and never made it to a post. I’d like to hear their side of the story but they’ll refuse to acknowledge the issue at all.
Tom P. stated:
OK, but if this is the case (haven’t looked at the temp records for the area), and only these trees respond to current rise in temps, shouldn’t these be the most proven accurate proxies, and the only ones to be used for temp recons for this area?
Re: Sonicfrog (#143),
Yes, and I presume that forms part of the basis for Briffa selecting the Hantemirov & Shiyatov (2002) data for the Yamal reconstruction.
I don’t know if its as good as Bishop Hill would do but I’ve done my best to make a non-technical explanation of what you have found at my blog. Apart from anything else I used google maps to create a map of Yamal, Taymir atc. that may be of interest
Why can’t some trees in a population serve as more “accurate thermometers” than other trees? If one wants to compare temperature in the late 20th century with the MWP, one obviously needs to select trees/thermometers capable of accurately describing late 20th century temperatures, not trees with a “divergence” problem. Following this strategy, one could select and use these more “accurate thermometers” in reconstructing climate. However, one is then required to PROVE that this sub-population of trees IS more accurate by showing that they perform substantially better in a validation period than the whole population – not just in the late 20th century. Is it safe to assume that this was not done?
Re: Frank (#151), Not only is there no out of sample validation of tree-mometers, but IF they are good proxies, the trees you select should as a whole agree with one another going back in time. They do not (see plots above) and no one has ever even tested for this.
snip – editorially there is zero purpose to this sort of generalized comment about AGW and I’ve asked people not to make them. Plus I’ve asked on numerous occasions on this thread that people specifically comment on Briffa and Taimyr. Please be polite enough to respect these requests.
re:
Seems to me this is the beginning of peer review… on Briffa’s papers. Science takes time!
I’ve been re-examining the paper on trail on getting the Esper measurement data (look at the Esper Category in the left frame) and encountered this description of a dendro statistical procedure (post-Hwang Science required Esper to answer). No paleoclimatologists to my knowledge are troubled by this sort of statement.
5
Maybe its a bit unfair to suggest the 10 were cherry-picked. They were selected only because they were right, and obviously unaffected by spurious noise and unexplained influences.
Good stuff Steve.
And may I say: “I was here”. This will be a day to remember.
A good opportunity for those of you who appreciate the sterling work that Steve McIntyre is doing to hit the tip jar!!
Mosher said “why did these trees cease to be treemometers? which is the divergence question.”
I am wondering what would be necessary to explain the folllowing phenomena that appears to be true:
As studies or data are made avaialable, not only are more groups being discarded as not good thermometers, but more individuals are being discarded. As one progresses from 1960 onward, an obvious trend of divergence is established, resulting in fewer groups of proxies meeting the acceptance criteria, with fewer indiviual trees in a group meeting acceptance criteria. The relationship if carried forward indicates that trees with acceptable properties will be statistically extinct in a relatively short time as compared to the lifespan of the tree that is being used as a “good” thermometer. At what point does this trend invalidate the assumption of trees as thermometers? Without senstivity testing, claiming the tree proxies remain good seems problematic and unacceptable.
Don’t want to make this out OT; but what Dr Loehle wrote about divergence and the basic assumptions of Briffa may be appriate. Perhaps Dr. Loehle will chime in. The above could happen if one were looking at a growth limited phenomena that is bell shaped and started throwing away “diverging” data. One would tend to keep trees that were in the accelerating region, AND most importantly, ones near the flat response region, having thrown away the “divergent” ones. Thus when one gathers more proxies, or resamples at a later date, one is more likely to find more “divergence.” This outcome would also be more likely if the optimum growth region was below present conditions. I think this would fit Dr. Loehle’s clipping of the sine curve that he showed in his paper.
Can I suggest that Ross’ excellent review of the history at post #10 somehow be linked at the top of Steve’s original post as a pre-requiste primer? Lots of people who don’t come here often are liable to show up to see what the fuss is about, and that’s an excellent contribution to understanding why it matters that this get reviewed and settled.
I love Steve’s work, but in blogging on these issues sometimes the degree of context that is already assumed to be known by the readers is mountainous. Usually that’s not such a big deal, when it will typically just be the regulars reading –but on a biggie like this, helping occassional or entirely new readers climb the context mountain first seems to me to be not only appropriate, but highly desireable. And Ross did that excellently in his post at #10.
Let me see if I can get this one. Your objection would be true enough if it were proven that particular tree ring series reflect temperature over arbitrary time scales. The problem is that this is not proven anywhere except in a circular manner.
To take the sharpshooter’s fallacy one step further. Suppose that there were thousands of targets pasted to the barn, and the “sharpshooter” was blind, and took a few shots, then said that only targets with the bull’s eye hit were the actual targets he was shooting at? Well, it is pretty sure you could find a couple that meet that description, so you would have a couple “true” targets that prove the the blind man is a “crack shot.”
Now suppose you have a large number of tree series and you just pick out the ones that match the temp records. Does this prove that they are “true” thermometers? Let’s get back to the blind “sharpshooter.” What would your first reaction be? It would be disbelief, and you would ask him to do it again. If the blind sharpshooter hit those targets again with any kind of accuracy, you would have to concede that something is going on there, but if the blind sharpshooter completely misses the newly identified “correct” targets, you would surely believe him to be either a fool, or trying to put one over on you.
Well, Mann and the hockey stick people were given a second shot at their targets to see if they could do it again, and instead of just leaving up the targets they hit with a bullseye before, they again pasted thousands of targets to the wall, and selected the ones that they hit, again! In other words, they are trying to pull the wool over your eyes, and likely think that they are doing it to save the planet, but perhaps, they are just stupid.
This needs to be published in the peer reviewed literature. Are there plans for this?
Tom P, you just described “cherry picking”.
Ross McKitrick, but isn’t that essentially the procedure Rob Wilson described on this blog a couple of years ago? I still don’t think its a legit approach. Given enough ‘samples’ we would certainly find some that could be considered to provide good correlation during a calibration period…which of course in no way indicates that ANY samples are really thermometers (rather some percent will randomly ‘fit’ for some interval). Anyway, I thought they admitted a long time ago that they do regularly disregard the nonconformal samples…as wrong as that may be…I still don’t grasp the full significance of this post.
LarryT –I’m struggling to think what would constitute an outlier location for what is claimed to be a global issue. Surely every location on Sol 3 is just as centric as every other? If you meant “remote”, then the question becomes “remote from whom?”, followed by the reasons that locations with the necessary length of dendro history will by necessity be remote from most observers.
But cherry-picking the data points for friendly outliers is of course another matter.
snip – sorry. I’ve asked people not to discuss dendro from first principles on this thread.
Tom – good post and paper on some of the problems of uncertainty in dendro chronologies here:
http://www.climateaudit.org/?p=3260
@Geo, no 175.
I agree absolutely – I was getting the most important parts of the message from Steve’s intro, but the additional explanation/context provided by Ross is essential for any part-timers here.
This analysis is brilliant, so truly a fantastic job by Steve.
snip – venting
It is obviously wrong and unscientific to graft a tempreture plot to a dedro or multiproxy chronology but it is perfectly appropriate to select proxy data based on whether the data conforms to a modern temprature graph. This is how we prove things in climate science.
The attacks have come – It is a blog, sigh .. A biased blog roll .., what papers have been referenced? Is it peer reviewed? can we have any confidence in this “unscientific piece of blather” which “add up to nothing more than pointless minor statistical gripes that have been repeatedly shown to have no significant influence on the output but nevertheless are zombified into perennial doubts that cycle through the ignorant.”
Re: Richard (#198),
That kind of attack is to be expected. The response should be simple and unadorned.
McIntyre is merely plotting data that has been used or ignored in peer reviewed literature. That data is now open and free, thanks to efforts. His code for displaying the data is also open for review. The plots tell the following story: The peer reviewed literature relies on data that has been cherry picked without any documented justification. There is no peer review required of McIntyre’s display of data. The data has already been reviewed and officially archived. It’s ground truth; Now that truth has been made public. The onus in this matter lies entirely with Briffa and others to explain the following:
1. what method did they use to cherry pick this data?
2. why did they fight the release of this data?
It is not Steve’s responsibility to read Briffa’s mind or construct justifications for his actions. He has done what every auditor does.He checks the books. In this case, Steve is merely pointing out what is in the accounts that have been hidden from view. An auditor doesn’t explain why the loot is there or come up with an explnation, he merely points out: “hey.. your hiding this cash over here.. what’s up with that?”
If anyone doubts Steve’s work, they are free to replicate his work. Briffa or Mann or any other person can go get the data and show that Steve’s charts are wrong. They haven’t. Maybe they can’t.
Tom P, we need to look more closely at the calibration/verification process in the instrumental period and then differences in the reconstruction period to better understand the implications of using an a prior selection criteria.
Ideally we would have a selection criterion prior to the calibration and verification period or we would be cheating on those parts of the process – agreed? We would then use an “average” proxy response to produce our reconstruction model. Some portions of the data would be in better agreement with the instrumental temperature than other portions. Eliminating some of the data at this point would be no different than eliminating some portion of the data that lies in contrast when we carry out the reconstruction to the pre-instrumental time period –for instance a dip in the medieval warming period. We are required to take averages over the entire time period.
Now, if the calibration, and better still the verification, time period correlation is high, our reconstruction will have lower uncertainty with everything else being the same. If however, a reasonable and physical a priori selection process leads to a poor correlation we would have much higher uncertainties in our reconstruction results and with a sufficiently poor instrumental correlation the uncertainties may be so high we would reject the proxy’s capabilities to provide a reasonably accurate reconstruction.
Re: Kenneth Fritsch (#200),
I need to add to my previous post that selecting after the fact for proxy response to instrumental temperature correlation would not even allow one to see the problematic issues that are revealed with an a prior selection process and it certainly voids the intent of the calibration/verification processes.
re: Tom P.
In effect, you seem to saying this:
1. choose the tree ring sets that match the 20th century surface station temp data.
2. Use those sets to compute estimates of pre-20th century temperatures.
This is all ok, except remember that the sets chosen in step #1 are effectively your “training” data, which means you cannot legitmately report any temperatures for the 20th century (because you deliberately chose the sets to match for the 20th century).
In the humble opinion of this layman lurker, what cannot be reconciled in this situation is this:
If Briffa had a perfectly valid reason for using only the 10 series he used and a perfectly valid reason for excluding the other data that Steve has uncovered, then why the refusal to archive the data? Why the ongoing, years-long effort at obstructing access to it?
Dennis – re:1. I disagree; unless one has a causal explanation why the retained data is a valid temp. proxy and the discarded data is invalid, IMO this is equivalent to inventing numbers to support a postulated case.
Re: The 5 POR trees, are they treated as a separate Regional group with their own Regional Curve? If so they are all about the same age which seems to be not what one would want. If not which “similar” group(s) are they combined with?
Also are the data, ring widths, and if so are they in microns? If they are how come do they drop to zero for some years?
Alex
Steve, thank you for your persistence and hard work. This post has been very revealing. While I am reading with great interest the comments regarding criteria for inclusion of cores used in this study, it is the final sample size for the “blade” that I find most alarming. Especially given the reliance on this dendro series in so many papers. Just 12 cores….
Remember Steve… “Sign doesn’t matter.” 🙂
Argggh. One of the perils of real-time calculations is that it isn’t just the Team that makes mistakes. I’ve had to replace the two key figures in the post. I noticed the problem in trying to merge Polar Urals and Yamal data. There is nothing in the text of the post that requires changing. However, I obviously believe that errors “matter” and should be corrected and have done so.
The proprietor does not claim infallibility. One of the reasons for providing scripts and data is to improve error detection and correction – even against myself. In this case, I was working on an amalgamation of the Polar Urals and Yamal data sets and noticed the error myself. I’m happy to report that I caught the error within one day and that I caught it before any one else. I’ve replaced the two figures, leaving a link to the replaced figures; I will notify Anthony that a change is required in the mirror post as well.
Re: Steve McIntyre (#213),
Actually, I went to run your script, but then I realized I didn’t have the rwl conversion function in my library. Otherwise I might have caught it first.
.
People who are commenting will nilly had better read the blog. Simple issues like sharpshooting/cherrypicking have been covered before ad nauseum.
Re: Steve McIntyre (#219),
Nice job, just before I ran your script. What is this insanity about coming clean with errors. Obfuscate man obfuscate!!!! It’s a unique feeling to make a mistake in front of 30,000 people.
I notice now that visually the new combined graphs might actually correlate reasonably well with temperature. Wouldn’t that be interesting.
Re: Tom P (#221), We cannot discuss this on this thread but IMO you are contradicting yourself. I think you missed the main point of VonStorch as well. Perhaps leave the point on tAV and we can beat the heck out of it unmoderated although I’ll be slow to respond tonight.
Way ahead of you…
Please move this discussion over to the new Unthreaded area. This topic is very familiar to readers here and I don’t want this to hijack the important new material here. You are however confusing several different issues. If you are advocating that some trees in a forest are “proxies” for temperature, but others aren’t – and that you can only tell after you peek at the hand, then you aren’t going to be able to reconstruct temperature. This point seems to baffle climate scientists, but every statistician and econometrician understands it instantly. But over to Unthreaded with this. Thanks.
Sorry for the crosspost #217 with #216.
Gary (#108)
Beutiful, but a bit too “difficult”. I was thinking about bhe “wow signal”, the (supposed) radio signal from space, of course, if there ever was one, but come to think of it, i would have called this graph “the magic flute graph” after one of Steve’s fines posts:
http://www.climateaudit.org/?p=2920
In your new Figure 2…. [Amended Sep 28 6 pm. Replaces url] you are referencing the Figure 1 Yamal Core Counts in the “replaces url”
Even as a spectator and reader of this site, I feel I am participating in history.
Aside from Steve’s great work, I’d like to thank those that clarified (for me, and others I presume), the issue. In particular, in addressing Tom P’s points, you’ve really helped me understand the significance.
From my perspective (engineer), my view would be to compare the instrumental record with as many proxies (trees) as possible. If I couldn’t see some sort of correlation between the two, then I’d assume that there IS no correlation, and look for another type of proxy set.
Do you not read?
Please move this discussion over to the new Unthreaded area.
Kenneth #200 et al – shouldn’t your reference be to “a priori” information rather than “a prior”?
Steve, in your new “Figure 2…. [Amended Sep 28 6 pm. Replaces url]” you are referencing the Figure 1 Yamal Core Counts in the “Replaces url”
[Sorry for the duplicate, but I didn’t address my post to you originally]
Suddenly I keep seeing Hockey Sticks…
Another layperson here. Apologies if this is a daft question.
How significant is the Yamal analysis in relation to the Hockey Stick as a whole? Presumably there were other analyses which contributed to the output. Is the main issue here the questionable methodology that Steve has revealed? Or does it produce a significant difference to the whole output?
Re: BeePee (#237),
The divergence now appears restricted to the last two decades. The inclusion of the Schweingruber series would therefore depress the very last datapoints in Biffra’s RCS northwest Eurasian chronology by an index value of 0.1 to 0.3 or so – I think it would still look like a hockeystick.
I read the NLS portion of the script. It doesn’t look perfectly straight forward, but it shows good replication. Is there an older post I can look at to figure this out.
Re: Jeff Id (#240),
Jeff, the RCS.chronology script is very old for me – 2005 vintage and my style is cleaner now. Also I carried a few methods used for emulating some older methods that were used in a couple of old papers, but I’ve taken them out of my working RCS.chronology script and placed them in RCS.chronology.old.
In addition, here is a thread on a mixed effects approach to tree ring standardization http://www.climateaudit.org/?p=2435 , which compares RCS to “conventional” standardization. The two can be viewed as variants of a linear mixed effects model – though people in the field don’t understand this.
Re: Bender #130 and Kenneth Fritsch #136,
I did begin my question with “If”. Bender must have missed that, but thanks to Fritsch for reading and understanding. My thrust was one of sensitivity analysis. So what are the correlations of the additional samples with the modern temperature record? If it was Christmas I would ask for correlations over fractions of the modern temperature record as well.
As Ross McKitrick said in #200, data mining is old hat in econometrics and stock picking schemes. Random correlations occur all too frequently and all too frequently disappear outside of your sample interval.
Readers, please try to understand that there are two levels of communication going on here. This post is certain to attract specialist attention – whether they present themselves here or not.
I’m reporting in real time and am not reporting the entire backstory. A lot of regular readers here know the backstory. If you don’t know the backstory, please ask at Unthreaded. People will talk there as well.
I don’t mean to discourage discussion of what this particular thing means, but editorially I want to maintain some technical focus on this thread and have already asked many times.
Unless you are confident that you are commenting at a specialist level (and here I exclude attempts to debate dendrochronology from first principles or what this all means), please comment at the companion Unthreaded thread, People read that too.
Re: Steve McIntyre (#246),
As I said:
We now know the answer to the second point: the Schweingruber series had been inadvertently miscalculated. The new version does not substantially contradict Briffa.
Are there any grounds left for excluding the Schweingruber series? As it doesn’t contradict the temperature record, I’d say few I could argue for. Interestingly, having included this series we see the complete reconstruction agrees very well with Briffa’s original Yamal reconstruction for the pre- instrument period.
Do you really think that’s just luck?
The only point left open is why the Schweingruber series was not included in Briffa’s original paper. The excellent correlation between the two Yamal datasets strengthens confidence in both before 1850.
Steve: As noted below, you’ve totally misunderstood the sensitivity analysis. There aren’t “two Yamal datasets before 1850” – they are the same dataset up to 1800 or so. Only the modern portion differs – one uses the Schweingruber datset russ035 (which only goes back to 1850) and one uses 12 CRU cores. The other 240 cores are identical between the two versions. Of course, they’re correlated. In reconstruction terms, the differences in the last half 20th century are huge.
Re: Tom P (#249),
Ok, so you are blind. The black and red curves are identical are they?
What is the correlation with climate during the non-divergent period where the chronology match is good? Why not use that to calibrate the temperature response and reconstruct cliamte? Do you know what the consequences would be. I’ll tell you …
Re: Tom P (#249),
The Schweingruber trees (russ035) do not go back any farther than 1782 with fully half of them starting in 1847 or later. Perhaps someone will correct me if I am misconstruing the reconstruction procedure (I haven’t worked through the RCS portions fully yet), but I fail to see how these trees can have much of an impact on the period earlier than than the late 1700’s. I would expect the two reconstructions to be virtually identical in the early part. Why would that “strengthen” your confidence?
Re: RomanM (#174),
You’re right. Inclusion of the Schweingruber series only gives another seventy years of overlap before the instrument record.
Of course is Steve McIntyre, or anyone else, had spotted the earlier bogus discrepancy before the end of the 18th century, a lot of trouble would have been saved!
Steve: I apologize, but the pre-1800 discrepancy was slight and you were inconvenienced for maybe a few hours?? I’ve been trying for over 5 years to get Briffa’s data.
Re: Tom P (#169),
Tom, you’ve TOTALLY misunderstood the sensitivity analysis. The following statement shows this total misunderstanding:
The only POSSIBLE difference between the data sets arises in the 19th and 20th centuries. The subfossil data is EXACTLY the same in both sensitivity cases by the construction of the test.
There are 252 cores in the Yamal data set. In my sensitivity test, I removed the 12 cores still active in 1988 on, and replaced them with 34 Schweingruber cores sampled in 1990. 240 cores remained identical in both data sets, including all the cores before AD1800.
So of course the early portion is identical.
Re: Steve McIntyre (#174),
The greatest apparent divergence in the plots is very recent. It would be good to see a magnification of the plot covering just the last 200 years to as close to present as possible for the two individual and merged series.
Steve: OK.
Re: Tom P (#178),
Here is a comparison of the Briffa chronology of the spaghetti graphs (red) versus the “SChweingruber” variation i.e. using russ035w instead of 12 recent of 252 CRU cores, leaving 240 unchanged. (The red curve here is the archived CRU chronology, which varies slightly from my emulation of the RCS chronology.)
Re: Steve McIntyre (#180),
Thanks for that.
Is there a reason to not to include the 12 CRU as well as the Schweingruber cores to produce the broadest record? I presume for the complete dataset there would be a dip below the red trend line between about 1960 and 1990: the final trend might actually look more like a hockey stick than the original Briffa plot.
Steve: I did that in the main graphic. I thought that you wanted to see the contrast. The more lines of spaghetti in a graphic, the harder it is to see the point being illustrated. I dislike the confusing spaghetti graphs that are so common in the field and try to include a smaller number of things in any given panel. The green is just a blend of the two and is not a HS at all tho the 20th century is “warm” – which makes sense. (see Figure 3.) Recall that this merged data set is no “complete” in that we believe the Schweingruber russ035w to include all relevant samples, but the CRU 12 are selected from a larger population. There’s a case to be made that the Schweingruber population without the CRU 12 might be more representative, since you still have the impact of CRU picking in the merged data set – the effect is diluted by the SChweingruber data set, but is still present. Combined with the large population “divergence”, I’d be mildly inclined to view the Schweingruber variation as a better representation of ring widths than the combo data set, but to avoid arguments will use the combo data set as well. I’m working on a broader data set including Polar Urals as well, and that will probably be my base case.
Re: Tom P (#167),
I don’t believe that you have considered the full implication of your argument on the entire reconstruction.
Let’s suppose that you are right and that there are real treemometers which you can identify by comparison to an observed record. Your statement above indicates that there also trees that are not good proxies. Unless, false proxies are a recent phenomenon, the logical conclusion is that there must be a collection of these distributed throughout the entire time period prior to the start of the temperature record since there is no way to identify and exclude those proxies that are not good (unless you know that of course there was no MWP).
So what effect will this have on the reconstruction? Having only good ones in the modern era, we will see that the temperatures have been warming, but we already knew that. The effect the good proxies will have on the early part of the reconstruction will be merely to center it at a particular level of dimensionless chronology units. It will have little or no effect on the quality of the results prior to the actual time at which these known to be good proxies existed.
When we reconstruct the early portion, we will have a mix of good and bad proxies at most time periods. Steve’s sensitivity test shows that the result of including “bad” proxies is to flatten the reconstruction – even in the merged case, the difference is as large as a full unit, of the same order of magnitude as the range of the entire original reconstruction prior to 1800.
The amount of flattening will depend on the relative proportions of good and bad proxies, but the net result will tend towards a hockey stick shape. Any error bars constructed from the fit will seriously underestimate the bias created by the false proxies. Without knowing how prevalent bad proxies are, there is no way to adjust for this bias.
What if the choice of modern proxies is just opportunistic matching to the temperature record? Well, then you get a … hockey stick. But that’s another thread.
Re: romanm (#199),
If we were discussing individual trees with independent random variation, I would see your point. Any random set selected to match the recent climate record will trend to zero eventually and create a hockey stick. This seems to be quite a few people’s argument, and given the assumption that trees are not thermometers, it is correct.
But the aim is to construct a local tree series of a particular species, preselected to produce a clear proxy on internal quality control alone (clear rings, consistent statistics). The underlying hypothesis is to validate such a series in its entirety as a thermal signature if it correlates to the instrument temperature record. Only then can the series as a whole be used as a proxy on the assumption of some continuing temporal dependence prior to the instrument record. If you don’t accept that, then tree thermometers just won’t work.
Even such a validated series will become a worse thermometer back in time as the red-noise errors that can’t be rejected by the instrument record increasingly dominate any signal. This doesn’t mean a hockey stick, but an increasingly large error growth which can not exclude greater past temperature variability.
There are of course ways of contaminating the series through cherry picking at various points of the process. Steve talks about “12 special picks” of cores by Briffa. But as the original Hantemirov and Shiyatov Yamal reconstruction appears to have virtually the same number of cores as the CRU set for the last four decades, I’m not clear how a selection could be occurring.
It therefore seems reasonable to put the 12 CRU/H&S cores back in with the Schweingruber variation unless there is any valid reason to exclude them.
Re: Tom P (#205),
The commenter is, in fact, blind to the point, and has degenerated into a distraction. He has shown his mind is impervious to logic. Very sad.
Re: Tom P (#205),
Most of this is incomprehensible jibberish. If there is a logical question here, perhaps it can be restated more simply? Five-paragraph ramblings loaded with incorrect presumptions are not going to advance the discussion any.
Re: Tom P (#205),
You don’t seem to understand that the “correlation” is guaranteed by the cherry-picking and not necessarily by the “thermal signature”. The series with the thermal signature do not extend across the entire time interval of the reconstruction (none of them is present before 1550) , so your assumption of a “continuing temporal dependence” is an assumption made for an unknown mixture of good and bad proxies.
Again, you are not looking at the details. The response of “genuine” treemometers to temperature variations will result in deviations from an average noise level which are either closer to zero (a lower limit for a ring width) or (possibly substantial) upward increases due to warming. A bad proxy would be expected to stay closer to the average level with added noise variation. Yes, the error variation would become larger. However, the averaging procedure would have the net effect in both cases of reducing the impact of the warming or cooling and creating a flatter result nearer to the average noise level of non-treemometers. This would create an unaccounted for bias in the estimated results approximating the handle of the stick.
Newbies: post your general questions in the thread called “unthreaded”.
Threads at CA are just that: “threads” – where a topic is followed.
Off-topic commentary is effectively making a mess of Steve’s lab notebook. Nice.
Anyone have issues with my comments, take it to “unthreaded”. I will not respond here. These bone-headed questions have simple answers. UNTHREADED.
We can now clearly see why Steve has repeatedly called for archiving and free access to data used in these studies. Look what he can do with it when he gets it.
We can also see why the Team has resisted archiving the data and making it freely available.
The UNEP graph recently discussed here is an interesting one: it uses both Briffa’s Yamal series and Mann’s infamous PC1.
Tom, if you’re interested in the detail, the script is completely turnkey and R is free – It contains a few goodies not posted here. Copy paste that’s it.
The [Amended Sep 28 6 pm. Replaces url] for Figure 2 still incorrectly links to Figure 1 as reported by James Smyth (#156)
Steve: Link fixed.
For reference, there are many discussions here on the “Divergence Problem”. Schweingruber was a frequent Briffa coauthor and Briffa has spent most of his career publishing data collected by Schweingruber.
Divergence between temperature and density (MXD) is the topic of a number of Briffa articles, but the divergence between temperature and RW is illustrated passim only in Briffa (Schweingruber) et al 1998, discussed at CA here – I urge readers to look at some of the Feb 2006 posts (pre-NAS Panel) with the following key image. This graphic is the only representation in the entire tree ring literature of the average of a large population of tree ring chronologies from sites expected ex-ante to be temperature sensitive. The “Divergence Problem” is something that paleos don’t like to talk about in public, though it is discussed occasionally in technical literature (where cargo cult explanations abound).
Steve,
I think a 200-year close-up of the combined green plot to date would be instructive, as would be a reaveraging of this new set with Briffa’s other two to produce his RCS northwest Eurasian chronology.
I don’t see a major inconsistency between your work here and Biffra’s paper, but please tell me if you beg to differ.
Re: Tom P (#184),
Tom, I don’t mean to be rude, but you’re being obtuse. THe huge Yamal HS blade is gone.
Re: Steve McIntyre (#185),
The green curve I suggested will make this plain, then, as well as the shift in the RCS northwest Eurasian chronology.
Steve: Please don’t inflate your contribution here. You didn’t “suggest the green curve”; it’s been included in my Figure 3 from the outset.
Re: Tom P (#186),
It’s too late tonight.
Please understand that I already have a very good idea of the impact of this on the multiproxy studies because of prior experiments with Polar Urals instead of Yamal – which by itself changes the MWP-modern relationship in important spaghetti graphs. There are only so many calculations per hour that I can do.
If you review past discussions on the “Yamal substitution”, the “reasons” for substituting Yamal for Polar Urals are now shown to bogus. (You’re coming into the middle of a long conversation here. Look at the Feb 2006 archive for example.) The logical data set, as I said above, is the combination of Urals and Yamal and russ035 in a NW RCS, which I was working on earlier today and will finish tomorrow. I’d be inclined to remove the data affected by CRU cherrypicking but will leave it in for now. [Note: Oct 4 – as noted in other posts and comments, it was and is my view that the selection of cores was done by the Russians and not by CRU. In context, there is a statistical issue of the representativeness of combining a population from one site with a screened population from another site. In the present circumstances, the logical thing to do would be to either have the authors of Briffa et al 2008 provide the data on the complete population or to resample the site or both.]
Re: Tom P (#184),
Hey Tom, you got the code and the data, knock yourself out. Steve ain’t your data dog. You go fetch.
Re: steven mosher (#188),
I’m patient enough to let Steve to plot his own data when he’s ready – it’s up to him to quantify the inconsistency with Briffa.
Sad but not too surprising. Good job, Steve.
Everyone: you may want to expand and improve the Wikipedia article on the divergence problem:
http://en.wikipedia.org/wiki/Divergence_Problem
Tom P.
It’s up to Briffa to explain his cherry picking. As for the inconsistency look at figure b in Briffa 2008.
From that other thread here is a Briffa quote you didn’t highlight:
“However, in many tree-ring chronologies, we do not observe the expected rate of ring density increases that would be compatible with observed late 20th century warming……This prevents us from claiming unprecedented hemispheric warming during recent decades on the basis of these tree-ring density data alone”.
And that describes the real situation directly from the horses mouth. Everything else is mere speculation – not “evidence” of any kind.
Actually [speculative and maybe off-topic] it wouldn’t surprise me if it’s more to do with the coring process than the environment. I’d expect to get a far better record from the hole because the core should change shape significantly once released from the external stresses imposed by the rest of the tree. Of course surely dendros check for this, but then again, my experience is that most engineers don’t know that this happens in metals too: And they remain non-believers until they actually measure it. Nature always has to re-establish equilibrium so two separated parts from compressed material are not quite the same as the original. And wood is a lot more active than metal so it would change shape just by taking it down the mountain. Again I presume dendros know this and cater for it, but it still makes it complicated to ensure data quality from any coring operations. Hence I’d expect a large percentage of poor data so I can see the temptation to discard suspect data and truncate the series. Swapping in “good” data based on mere assumptions though remains an indefensible practice. Or so you would have thought. But there always seems to be a Tom P around to try to defend the indefensible.
Tiny point Steve but some notice. Could you change your spelling “Taymir” at least in the titles to “Taimyr” as used in RS Philosophical Transactions?
Steve,
Given the doubt raised by these findings do you believe the responsible position for Jones at the CRU would be full disclosure of data to avoid the doubt ‘spreading’?
Steve,
With respect (and you surely deserve it), given the amount of work you say it took you to find this ‘divergence’ is a further FOI request to the CRU appropriate? Presumably the data has been supplied to ‘academics’ who would have had a chance to do the same review you have done. Perhaps a list of those in possession of the data might permit direct contact with them to allow them the opportunity to provide further scientific explanation of these remarkable findings?
I understand that some readers have walked into the middle of an ongoing conversation here and do not necessarily understand immediately exactly why the falling apart of the Yamal series “matters”. I sympathize with that, but at the same time, the Yamal data was hot off the press and lots of readers know the backstory.
In this case, I had recently observed that the Kaufman reconstruction was in the Yamal “family” of hockey stick reconstructions and had shown a KAufman-style reconstruction replacing the Briffa RCS reconstructions with equally plausible tree ring series from the same area: Grudd’s Tornetrask instead of Briffa’s; Esper’s Polar Urals instead of Briffa’s Yamal and the Yakutia series used in Moberg instead of Briffa’s Avam-Taimyr. All alternatives had acceptable pedigree. I also did some cosmetic things like place the Tiljander series upside-up instead of upside-down. The effect is discussed in a Sept 19 post here. Jeff Id did a lengthier step-by-step commentary here. The difference is shown here:
I will discuss the impact of the Yamal collapse in more detail in other posts. I’ve provided access to scripts and data that enable interested readers to draw diagrams that interest them.
I haven’t decided yet whether I want to dignify the CRU selection of 12 cores by using this data to dilute the results from population averages (e.g. russ035w). At this point, I’m inclined to think that the calculation using russ035w without the 12 special picks is a more defensible sampling procedure and that a “compromise” merged set is still acquiescing in an undocumented and suspect picking process, albeit diluted by information from an actual population, and am reluctant to appear to give any perceived endorsement of such a “compromise”. I’ll think about it, but there’s lots on my plate right now.
Re: Steve McIntyre (#194),
May I pre-empt Tom P?
“Those curves are identical.”
🙂
Yamal itself presents no real surprise. Mother Nature is complicated and unpredictable; I’m not sure what one would expect to see in the tree rings even if AGW were entirely dominant or non-existent. The antics of Briffa, Mann, Moberg, D’Arrigo, Hegerl, Jones, and the rest, also come as no surprise, superficially complicated in their own way but pathetically predictable.
Kudos to Steve!
I have been watching the topic develop with keen interest since yesterday morning and have been astounded both by the data cherry picking uncovered by Steve and the fact that a few posters have tried to justify the selection of 12 data sets which “give the correct answer”. If you are convinced that recent global warming is “unprecedented” you may feel the need to defend the hockey stick, but this manipulation of data is indefensible. If I set out to prove that the average height of students in my class is over 5ft 10 inches, I can’t measure the height of each student and then only select the six footers to prove my point. That’s not an unbiased sample.
Re: EddieO (#196),
But biased sampling is an advantage unique to dendroclimatology. You’re talking mensurational demographics.
🙂
Re: bender (#198),
Hmm, haven’t seen that word since I was in academia! 🙂
Searched at the realclimate site for Yamal = no hits and Schweingruber, last hit December 2004!
I love it!
Great work Steve, we share something in common, we both like to dig. Its very exhilarating when you hit pay dirt, don’t you think?.
Keep up the great work.
I suggested earlier that I could not run the script because I did not have some of the functions. Well, that was just a guess. I just tried it and the script runs just fine. So I don’t know what Tom P is complaining about. You want a graph, graph it yourself. The scripts is, as Jeff Id said, completely turnkey. If only the team went that far – then there would be no blog wars.
EddieO:
Yes, I completely agree.
And I add that the entire world owes a debt to Steve for his above analysis and for his previous analyses of the dendro data.
snip – politics
Steve:
Brilliant! Thank you.
Richard
Re: Richard S Courtney (#202),
The paleoclimate data do not refute the GCMs.
#205, Tom P: You are making a distinction with very little difference. Regardless of whatever your intent is and whatever filtering you do intially on the quality of the trees, you are still performing the correlation filtering that you admit will manufacture a hockey stick. If it will manufacture a hockey stick with random noise, then it will manufacture a hockey stick with tree rings, regardless of whether they are random noise or if they are perfect thermometers. There is no way to tell what the trees are from the procedure you describe… except, perhaps, selecting trees that calibrate to temperature, and then peeking at a different period to validate that they still correlate with temperature. The closest thing to such a test (re-coring trees in the exact same area of previous studies) has shown that the trees miserably fail.
The number of samples is not the issue. The commenter is obfuscating the issue, which is the difference in signal contained in Briffa’s choice versus virtually any other subset that could have been chosen. “How a selection could be occurring” is by comparing the end product to the juiced-up surface instrumental record, and selecting those that maximize the correlation. That’s how.
.
Further obfuscation will be duly noted by many, many witnesses.
Obfuscation. The primary issue is not whether Briffa’s 12 belong in. The primary issue is why did he specifically take them out and make them their own chronology?
What the heck is going on with those 12 is a secondary issue. If you can’t explain the divergence amongst samples you have a serious reconstruction problem. The proxy lacks credibility when “sensitivity” is inexplicably lost. Who’s to say it wasn;t inexplicably lost during the MWP? Which was precisely RomanM’s point. (Well explained, by the way, RomanM.)
Steve M,
I apologize for all these comments, but this thread is lost to trolldom. I can’t leave these drive-bys go unanswered. It sickens me.
Amazing. I wish more people cared.
Just a quick note – various comments suggest that the Schweingruber Khadyta River series might have young trees in it. I should point out that at the ITRDB there is a site roughly half way between Yamal and Taymir called “Kheta River” (69N,84E). There are two separate entries for this site one of which specifically says junge Baume (i.e. young trees)
If he split the old and young trees at Kheta then presumably he’d have done the same as Khadyta
Steve: Not a given. There are many archived Schweingruber sites and the vast majority do make such a segregation.
It’s an interesting point Tom is making. I had often wondered at tAV how someone could make a reasonable argument for scrapping data which didn’t fit a conclusion.
What Roman is saying is that when a portion of reasonable error-free data does not correlate in the calibration range, you still cannot scrap it. Doing so biases the signal to noise ratio in the calibration range to a different value than in pre-calibration. The net result after scaling each accepted series to temp is a demagnification of the historic signal and a near guarantee of unprecedentedness in the calibration range (reduced noise).
The fact that correlation stinks is only related to the quality of the data and more or different data should be collected. After the fact sorting of series with very high noise for a particular signal is mathematically dishonest. In addition, the quality of correlation becomes irrelevant when high percentages of data are scrapped.
Note the big blade on the end of the Briffa Yamal hockey stick. Note also that very high variance was found in the 12 cores in relation to all the other available cores. This suggests a pre-sorting operation of some kind. Note also that it took 10 years for Briffa to release the cores. Finally, consider that methods for sorting and scrapping are standard fare in this ..um…science. There is substantial reason to believe that much of the collected data which wasn’t used has not been released.
The inclusion of more cores or not shouldn’t make such a huge difference in outcome – but it does. I’m quite happy to see them all included, green line above but to insure that we’re not biasing the record, the methods, locations and all remaining data from the original Briffa series need to be disclosed. Otherwise the well documented Schweingruber set is actually superior (I can’t believe I just wrote that).
Good luck with getting that out of Briffa, especially after this post.
I did a post on this at tAV copying Romans comments above.
Bender, I take Tom P’s participation here as a good thing. I consider his POV much inline with what I suspect is that of many climate scientists. Tom is obviously an intelligent man and well informed of the reconstruction literature. His comments allow some of us to see where misconceptions have been allowed to creep into the rationalization of the reconstruction process and particularly the selection process.
I would also guess that we have many readers and even those commenting here at CA that do not entirely understand the concepts, that RomanM has explained best in the preceding posts, and continue to scratch their heads on why someone should not pre-select “temperature sensitive” proxies/samples. Rob Wilson, who appears to be well thought of here at CA, came across to me with these same misconceptions. I also have had the experience with another web site where a very bright economist and statistician held forth explaining these concepts to some very bright and well informed people with regards to investment strategies and to no avail with a number of them.
Re: Kenneth Fritsch (#216),
The question is, what are tree-ring proxies like, financial time series or thermometers?
I have absolutely no problem with in recognising that a model fit to financial data over any limited time is no guarantee of either hindcast or prediction – markets are stochastic processes.
On the other hand, if I calibrate a thermocouple after an experiment, I can derive a temperature from the voltage record. It’s a deterministic process.
A tree-ring reconstruction will lie somewhere between, but ignoring the deterministic properties is to throw away valuable information to help extract the best signal.
Re: Tom P (#227),
Tom P my reference to stock investments was to show that bright and informed people can continue to have misconceptions even after a tutorial by an informed statistician, even though the analogy to tree thermometers is a good one.
I think that the part you are missing about the deterministic view of trees as thermometers is that we need to select trees that are truly thermometers and accurate over the time period of the reconstruction. A reasonably produced thermometer correlates very well with temperatures and this can be shown with an independent study and over the entire range of interest and with a physical model. That obviously is not the case for tree thermometers and even in the calibration and verification periods.
The question becomes: how does one show/select a tree thermometer, or better an average from a series of tree thermometers, that will “work” over the entire period of interest? If one could do in-sample and out-of-sample testing this task would be less difficult but still not bullet proof – a physical model to test would complement this testing very nicely. Please consider at this point the so called divergence problems of trees used as temperature proxies. That problem can be considered merely an out-of-sample test that failed for trees that were evidently selected for performing well in-sample. This result points to the problem of tree proxies selections.
The uncertainty related to a proxy model that is constructed in-sample is related in part to how well trees selected, with a reasonable and physical a priori selection criteria, represent temperatures in the instrumental period. In this testing procedure it should be obvious that you cannot select after the fact for temperature sensitivity without greatly changing and in undeterminable amounts the uncertainty of the resulting calibration, verification and reconstruction results.
If one could posterior provide a physical model for the trees selected acting as thermometers, then that development could pass some credibility muster. For example, I have a thermometer that I used to measure an unknown temperature and then I can later show and test a reasonable physical model explaining the temperature relationship. But thermometers workings are deterministic, while tree thermometers evidently have no connected physical model or that would be used for that selection criteria we talk so much about invoking a priori.
Re: Tom P (#226),
Except that what is happening is that a “noisy” signal is being thrown away for the recent time periods, but it is being *retained* in the early time periods. Instant bias.
If your conclusion is that larch ring widths in northern Siberia should be rejected, that’s fine with me. What is absurd is your apparent position that some Yamal larches are thermometers, while other Yamal larches are not – without any objective way of distinguishing them. The problem of “positive and negative responders” at the level of an individual site is a conundrum in dendrochronology. If your position is that this conundrum needs to be resolved prior to tree ring chronologies being adopted as temperature “proxies”, that too seems like a tenable position.
Re: Steve McIntyre (#220),
One objective way would be to look at the match with the temperature record – that worked and identified your miscalculated series (a fluke?). Another would be to look at the internal statistics of each series.
How do you think Briffa has “selected” his twelve cores? They seem the same as the earlier H&S series.
Re: Tom P (#218),
Nope. Not on a tree by tree basis. The standard practice in the field is take the population. Jacoby, D’Arrigo, whatever other faults they may have, use the entire crossdated population from a site. (They cherry pick sites, but don’t cherry pick trees within a site.) Ed Cook, a fine field dendro, bases a chronology on the entire crossdated population. So does Stahle. So does Peter Brown. Likewise the young dendros worried about divergence: Wilmking, Wilson, Pisaric, Andrew Bunn – their chronologies all use the entire crossdated population.
If you can get a single dendrochronologist to support Briffa’s use of 10 trees in 1990, I’ll be flabbergasted. They will be astonished and appalled at the procedure. The young dendros will be wincing and some of them will probably be bit shell-shocked at this news. It’s very embarrassing for the field. I don’t expect any of them to announce their disappointment (we’ve encountered the silence of the lambs phenomenon before), but make no mistake: no young dendro will stand up for what Briffa did here.
As to whether the selection was previously made by Hantemirov and Shiyatov or not – I discussed this in the previous post and noted that there was evidence that the selection might have been made by them. But as I noted in that post, that doesn’t help Briffa. RCS needs a larger population than corridor/conventional chronologies (which don’t recover centennial variability). This is well-known both in Briffa’s own work and to the younger dendros. Briffa should have been using 60-100 cores in 1990, not 10. End of story.
Re: Steve McIntyre (#219),
Of course on a series basis, as I mentioned above.
As for Briffa’s Yamal cores, the issue seems to be of their recent paucity than any biased selection by Hantemirov and Shiyatov. I see no reason for not including them in the enlarged Yamal reconstruction.
Re: Tom P (#221),
Throwing away data on a series basis suffers the same problems. Since you’re not throwing away any historic series you bias the data in the range you know. It’s really a simple concept. If there is reason to believe there are errors in the data certainly you can throw it out, throwing out trees which don’t match thermometers and accepting all past trees biases the data and overstates your confidence in the result. There is no other purpose for it.
Re: Steve McIntyre (#219),
I just had a first play with R. Here’s the combined dataset plotted against Briffa’s original. Looks like the blade is back again:
I think this is right, but you might want to check.
Re: (#233),
Steve, I’m not sure if you’re looking at this plot, or it’s got stuck in the system. If the latter, here’s the link, plaintext:
http://yfrog.com/28schweingruberandcrudp
Sorry:
Register article: Treemometers: A new scientific scandal
Upon reflection, I think the real scandal here is not the use by the Team of wildly inappropriate statistical methods, the real scandal is the years of obfuscation by the Team. It can no longer be argued by the journals that there is no need to force authors to publish the underlying data and methods. This episode demonstrates the abject failure of the ‘clique-review’ process that the journals have allowed to exist. If there is one action that could come out of this it would be for the journals to now go back to Team members and insist that they archive all data and methods or face disciplinary actions including having their articles withdrawn.
Steve, not to add to your to-do list, but I think you would be in a unique position to write an editorial to one of the tier 1 journals, laying out this sorry episode and arguing for changes in editorial policy.
Let me see if I can get my non-specialist mind around what’s going on here. The key point is that the trees involved in this study have been cited time and time again to “prove” that the modern warming is outside the normal range of climate variation. Doing that inherently means comparing tree growth characteristics from the modern era to similar characteristics from earlier eras like the Medieval Warm Period. Even if you accept all of the assumptions of the studies, and all of the defenses of the likes of Tom P, making that comparison is indefensible.
1) What Steve M. has apparently proven here is that data from a randomly selected group of trees does not pick up the large modern increase in temperatures that thermometers in the area supposedly do.
2) The study involved here gets around that problem by finding a subset of trees that match the thermometer numbers. The rest of the data is rejected. The study is making an unstated assumption that for one reason or another there are “thermometer trees” and “noise trees”. Based on that assumption, unless you get rid of the data from the “noise” trees, it masks the signal from the “thermometer” trees, making the large modern temperature rise invisible. To get around that problem, get rid of the ‘noise” trees.
3) On the surface that sound plausible, though it also assumes that “thermometer” trees remain that throughout there life. However, when you try to compare this modern sample to one from the Medieval Warm Period you’re faced with a problem: You’ve already asserted that some trees from a population are noise and others are thermometers. Steve has already established that including the alleged noise trees can hide a temperature rise as large as the modern one.
4) When you look at the Medieval Warm Period, though, you are looking at both noise and temperature trees. We’ve already demonstrated that including both can hide a temperature rise as large as the modern one because we’ve shown that it did hide the modern one. Since there is no thermometer record to let you pick your trees from the medieval warm period you end up comparing a noisy signal to a clean one and the comparison really tells us nothing about the temperatures in the two periods. Medieval warm period could have been cooler, warmer, or about the same as our current temperature.
Re: Barry R (#227), Slight correction: the “thermometer trees” don’t go back very far in time (living larch don’t live that long) so the MWP is totally from trees with unknown thermometerness.
Steve, as always, outstanding work. Congratulations. It sounds like you and Ross are working up a submission. I also suggest that in addition to whatever Geophysical or Climate journals you choose, that you also submit to Technometrics.
Tree growth is a stochastic process in that random behavior is introduce through random changes in the local environment (the random introduction of a new tree that blocks sun, and random change to soil conditions resulting from a mud slide, the random injection of a blight that slows growth, etc., etc.).
Some press interest in the story.
Re: Bishop Hill (#235), Re: StuartR (#239),
Interesting, but there are unnecessary errors and typos in the article. In general, if you allow for the flaws it’s correct, but it’d be nice if they had an editor and a CA science expert who could correct them where necessary.
One thing which bugs me, but which may be becoming a lost cause is talking about dendrochronology when it’s really dendropaleoclimatology (or something like that) which is meant.
Re: Dave Dardinger (#241),
I cringe at the listing of “CO2” as the first factor confounding dendroclimatological analysis. Precip is first, unquestionably. CO2? Prove it. Trees are worse carbonometers than they are treemometers.
Tom P, just a thought.
Do you suggest the trees’ supposed temperature signal to be calibrated against global instrumental records or local? Maybe OT, sorry..
TomP
Please explain to me: If you develop an algorythm based on a small subset (by eliminating a high percentage of the population), then apply the algorythm to a much larger and more diverse data set (without eliminating any percentage of the population), how can the algorthym generate valid data across the whole set?
It sounds to me like you test all sea creatures for intelligence vs. fin size, filter out everything that is not a Dolphin, then use those numbers as a proxy to relate intelligence to fin size for all sea creatures back to before the whaling industry started. Suddenly we have a “fish getting dumber crisis” because the whale-to-sea creature ratio was been interfered with by humans.
Thanks for the link Bishop, having lurked here reading this page I obviously couldn’t comment until it was ‘real’ and in the media 😉
I like El Reg. The Register is pretty iconoclastic when describing the major players in the IT industry on technical issues, and I think this particular issue crosses over nicely into their territory. They have smart writers who have the confidence to turn around copy in quick time on technical subjects and take the consequences…
I would hope this good work by our host will finally bubble up in what is known as the mainstream media one day, but as others have noted above, its not going to be in the papers for a while. Until then, any rebuttals that appear (or the lack of ) will be interesting.
Rob @ #236
I had the same question. I would assume that you need to use to most local instrument temperature record possible to calibrate the trees’ temp signal but I am not from this field.
Was the temp record used by Briffa specified or provided? Lucy Skywalker looked at the available instrument records in the area of the Yamal penninsula and there does not appear to be an instrumental hockey stick to support the Yamal proxy stick.
http://www.greenworldtrust.org.uk/Science/Scientific/Arctic-Yamal.htm
This seems problematic, no? Do we know what instrumental temp record Briffa used?
Consider the following (stylized) model. Suppose there are collection of tree ring chronologies extending back a number of years with each individual series being in one of two states. Either exactly correlated with temperature for that year. Or completely uncorrelated. As we go back in time, a chronology may switch from being correlated to uncorrelated with probability A and uncorrelated to correlated with probability B. Now, we are interested in saying the most accurate thing we can about past temperatures from a collection of these chronologies but only have benchmark temperatures for the first 5% or so of the years. Now, we could sample our collection and limit ourselves only to the series that correlate in the benchmark period. This subset is likely to be more correlated with temperature in the non-benchmark period (which is good) but is of more limited size than the entire collection (which is bad). However, if probabilities A and B are both low, this may be the optimal strategy. Good trees being unlikely to switch to bad one and bad ones being unlikely to switch to good ones. On the other hand, if probabilities A and B are both high then we may well be better off using the entire collection. Starting out better doesn’t really matter and we have a larger sample. One other casual point in this simplified model. If one choses to reduce to a subset at the start, the probability of correlation decreases (depending on A and B) as one goes back in time. The net effect of which is to “flatten” the scale of any signal that comes from averaging the selected chronologies over time. Hence, saying anything about whether a derived signal in the past is larger or smaller than one in the present depends on correctly understanding this “scale flattening” effect. To the extent this simple model captures the essence of the discussion at hand, it seems that we are simply arguing priors about what A and B really are.
Can I see if I have this correct?
1. We have a number of tree ring samples from different areas
2. Briffa has selected only rings that match valid local temperature records.
3. Briffa assumes that these trees stay in sync with temperature to an early time in their life.
4. Briffa matches older dead(?) trees to those that match local temperatures and says that these must therefore also be in sync with local temp.
5. repeat 4. and until tree rings are available at the required earliest age.
6. Statistics does not allow matching of tree ring temperature proxy to real temperature because this is cherry picking and will always produce a hockey stick
I agree that extending backwards from multiple overlapped records must produce greater deviations from reality.
I agree that a single tree ring record can deviate once away from the matched record.
However, thinking as an engineer trying to find an accurate reading of for example a time series of a voltage supply to a building monitored with inaccurate chart recorders over various lengths of cable (= added noise). if one recorder is known to have been calibrated (to national standards) over recent part of that record, then I would look at the other recorders over this calibrated period and throw out all the outlier readings (they are wrong now, and I do not know if they were ever correct so there is no reason to include them in my determination. Some of the more accurate recorders may have read high before the calibrated range and some may have read low. Some will be recording significant noise compared to the majority and so these could be ignored if sufficient others remain to determine this fact. I would then average the remainder and suggest that this average record is the most likely record of voltage.
As a statistician are you suggesting that all recorder outputs should be averaged including those reading zero and those reading full scale and those whose readings deviate grossly from the mean.
This seems wrong and certainly will give a invalid result. Am I wrong?
But then we need to look at the tree sampling.
Were the trees sampled totally at random – trees in water, trees in bogs, trees scraping by on a solid rock, trees near to death, young trees etc?
Were they at the tree line or sea level?
Were they all the same “make”?
Etc.
I would suggest that the actual sampling was not random. Altitude, health, species, etc. are all non randomly chosen (cherry picked)
If this is the case what is the point suggesting that they should not be further chosen to best represent temperature? what would be the point for example choosing a tree that fell over during its life but continued to grow with diminished root function? What would be the point in chosing a tree with growth limited by water/nutrients. What would be the point in including a tree 100skm further north than the rest? Would your statistical methods require that these be included in the sequence?
Re: thefordprefect (#244),
Some of your premises are not yet demonstrated. There are a couple of different levels of consolidation: at a “site”, multiple cores are taken, usually within fairly close proximity to one another. These are composited into a “site chronology”. Briffa unusually composites samples from areas not at all close to one another in Avam-Taimyr and Tornetrask-Finland. At Yamal, for some reason, he has not composited samples from Polar Urals, which is closer to Yamal than Avam is to Taimyr.
2. Briffa has selected only rings that match valid local temperature records.
[There are two issues: selection of trees at a site e.g. Yamal and selection of sites, e.g. Avam and a nearby Schweingruber site into Taimyr. The procedures are not described. It is not known how Yamal core selection decisions were made and which were made before CRU and which at CRU].
3. Briffa assumes that these trees stay in sync with temperature to an early time in their life.
[Not necessarily. WE don’t know what was done. It is possible that trees with elevated growth rates were preferentially selected, but we don’t know that for sure.]
4. Briffa matches older dead(?) trees to those that match local temperatures and says that these must therefore also be in sync with local temp.
[ no. crossdating is done by pattern matching.]
5. repeat 4. and until tree rings are available at the required earliest age.
6. Statistics does not allow matching of tree ring temperature proxy to real temperature because this is cherry picking and will always produce a hockey stick.
[this is a different issue.]
FP, I don’t have time to provide personal education to everyone trying to get up to speed. I’ve asked people who don’t have specialist viewpoints to comment on Unthreaded.
Re: Steve McIntyre (#250),
In an aircraft the various computers for controlling systems are usually arranged in odd multiples.
There is a reson for this
If there is disagreement with some outputs for safety critical control a vote is taken and the majority takes control. It is a great blessing that staticicians have not got their fingers in this pie.
Staticicians would never throw away invalid readings just average them – 3 computers say we are taking off 2 say we are landing so we’ll just throttle back a bit and put the undercarriage half down!
If data is in error it should not be used. Therfore if the only known valid temperature data is the last 200 years then anything that disagrees with this is in error
Re: thefordprefect (#275),
Misapplication of statistics can lead to the formation of some very profound and unwelcome conclusions.
So what you’re saying is that, because we don’t have temperature data stretching back for 2000 years, we cannot use dendropaleoclimatology to estimate past temperatures because we have no way of knowing which data “is in error”? Enter statistics.
Here is another chronology of larches from the area. No hockey stick. Could be a student of Hantemirov.
Click to access Geo28_10.pdf
Re: MikeN (#245),
This was presented already.
I posted the following comment over at WUWT where Tom P commented adversely on my being “quiet” for a little while.
I am online too much, but I am not online 24/7. I’ve been out playing squash. Surely I’m allowed to be offline occasionally without a poster commenting adversely on this.
While I was out, CA crashed as well. Thus, it was “quiet.
Contrary to Tom’s speculations and misrepresentation of my statements, it is my opinion that there is considerable evidence that the 12 cores are not a complete population i.e. that they have been picked form a larger population. Rather than quote form actual text, Tom puts the following words in my mouth that I did not say:
This is not my view.
The balance of Tom’s argument is:
I disagree. I do not believe that they constitute a complete population of recent cores. As a result, I believe that the archive is suspect. There is every reason to exclude them in order to carry out a sensitivity as I did. The sensitivity study showed very different results. I do not suggest that the sensitivity run be used as an alternative temperature history. Right now, there are far too many questions attached to this data set to propose any solution to the sampling conundrum. It’s only been a couple of days since the lamentable size of the CRU sample became known and it will take a little more time yet to assess things.
Reasons why I “suspect” that a selection was made from a larger population include the following. A field dendro could take 12 cores in an hour. We took a lot more than that at Mt Allegre and a field dendro could be far more efficient. Thus, it seems very unlikely that the entire population of cores from the Yamal program is only 12 cores and on this basis, it is my surmise that a selection was taken from the cores. Standard dendro procedures use all crossdated cores and definitely use more than 10 cores if they are available.
This doesn’t “prove” that a selection was made, but it is reasonable to “suspect” that a selection was made and to ask CRU and their Russian associates to provide a clear statement of their protocols. There’s no urgency to do anything prior to receiving a statement of their sampling protocols. For this purpose, it doesn’t matter a whit whether the selection was made by the Russians or at CRU or a combination. In my first post on this matter – which Tom appears not to have read, I canvass the limited evidence for and against. There is certainly evidence supporting the idea that the 12 cores were among 17 selected by the Russians, but in other parts of the data set, the CRU population is larger than that used in the Hantemirov and Shiyatov chronology. The construction of the CRU data set is not described in any literature; the description in Hantemirov and Shiyatov has something to do with it, but doesn’t yield the CRU data data set. Some sort of reconciliation is required.
In addition, the age distribution of the CRU 12 is very different than the age distribution from the nearby Schweingruber population. In my opinion, the uniformly high age of the CRU12 relative to the Schweingruber population is suggestive of selection – in this respect, perhaps and even probably by the Russians. Again this isnt proof. Maybe they were just lucky 12 straight times and, unlike Schweingruber, they got very long-lived trees with every core. Without documentaiton, no one knows. In any event, this doesn’t help the Briffa situation. If these things are temperature proxies, the results from two different nearby populations should not be so different and protocols need to be established for ensuring that the age distribution of the modern sample is relatively homogeneous with the subfossil samples (and they aren’t.)
[Note – one other very important piece of evidence mentioned in my first post: missing ID numbers at JAH, POR and YAD evidence are very convincing evidence that the archive is incomplete – it is immaterial whether that is at the Russian or CRU end, particularly since Briffa has relevant coauthors.]
The prevailing dendro view is that an RCS chronology requires a much larger population than a “conventional” standardization. Thus, even if the data set had been winnowed down to 10 cores in 1990 and 5 cores at the end, this is an absurdly low population for modern cores, which are relatively easily obtained. Use of such small replication is inconsistent with Briffa’s own methodological statements.
Tom also misses a hugely important context. There is a nearby site (Polar Urals) with an ample supply of modern core. Indeed, at one time, Briffa used Polar Urals to represent this region. My original question was whether there was a valid reason for substituting Yamal for Polar Urals. The microscopic size of the modern record suggests that there was not a valid reason. However, this tiny sample size was not known to third parties until recently due to Briffa’s withholding of data, not just from me, but also to D’Arrigo, Wilson et al.
Until details of the Yamal selection process are known, my sense right now is that one cannot blindly assume – as Tom does – that what we see is a population. Maybe this will prove to be the case, but personally I rather doubt it. A better approach is to use the Polar Urals data set as a building block.
As to Tom’s argument that none of this “matters”, the Yamal data set has a bristlecone-like function in a number of reconstructions. While the differences between the versions may not seem like a lot to Tom, as someone with considerable experience with this data, it is my opinion that the revisions will have a material impact on the medieval-modern difference in the multiproxy studies that do not depend on strip bark bristlecones.
Re: Steve McIntyre (#248),
Anyone well familiar with the backstory knows all of this. I’m glad someone (i.e. Kenneth Fritsch) is getting some benefit from Tom P’s seeming sock-puppet function. My advice to newcomers is to read the blog and get the backstory first hand. Tom P is a lazy dog demanding such spoon-feeding. Yes, it will take a few weeks to read the whole blog. Just do it.
Re: Steve McIntyre (#248),
Whether and how Briffa cherry-picked is immaterially ad hominem. The fact is that the Yamal chronology is (by Briff’a own criteria!) wholly inadequate to carry the weight that it does in the scientific and policy spheres.
.
Tom P: Why do you think the lambs are silent? Where are the defenders of Yamal? Of Briffa? Think about it.
Re: bender (#251),
bender, I agree with your point. I’ve tried to steer a careful line here. If you think otherwise, can you give me particulars as I don’t wish to unintentionally feed views that I don’t hold. It is not my belief that Briffa crudely cherry picked. My guess is that the Russians selected a limited number of 200-400 year trees – that’s what they say – a number that might well have been appropriate for their purpose and that Briffa inherited their selection – a selection which proved to be far from random and which, as you and I agree, falls vastly short of standards in the field for RCS chronology (as opposed to corridor or spline chronologies).
Re: Steve McIntyre (#252),
I do not think otherwise. I wanted to restate in my own words a key point that I thought was embedded in there. If ON TOP of the material deficiency, Briffa cherry-picked, he is going to lose some friends.
Re: bender (#251),
The period in question, (around 2000), is within the temperature record. Could it be that, rather than fraud, they chose not to use a proxy that was clearly at odds with what is known? Is it all possible? Should implications of massive incompetence or fraud, (for fraud is the word being used all over the denialst web sites), that there is a perfectly reasonable explanation?
Re: slownewsday (#265),
That might seem reasonable and logical, but the point of making these reconstructions is to reconstruct temperatures in the past, where temperature is not known, not the present. So, if you take a perfect reconstruction of modern day temperaure but you have to eliminate lots of trees to do it because they don’t reconstruct temperature very well, how do you do the same for the early periods when there isn’t a temperature to match them to? The major problem is that the “perfect” trees only cover recent time periods. They stretch back only a fraction of the entire time period.
.
So, while modern temperatures have been reconstructed using “perfect” treemometers, the past temperatures have been reconstructed using “perfect”, mediocre and poor treemometers, crowding out the “signal” from the “perfect” trees. What you end up with is a perfect reconstruction of modern temperatures and a poor reconstruction of past temperatures. This produces a huge bias towards modern temperatures ska, a hockey stick.
Re: Steve McIntyre (#249),
There may well be an initial selection based on the age of the tree – this would automatically produce the maximum overlap between the individual trees in the series and helps with the RCS approach. This selection would be associated with merely picking the largest trees living in the stand for coring by the Russian workers. I see no evidence of a biased sampling here.
All the more reason for including all the cores when constructing the chronology.
The core-count distribution is in Briffa’s Royal Society 2008 paper and for the last few deacdes matches the core-count distribution in the original Russian 2002 paper.
And yet you are willing to cut out the living trees from this record and replace them with a different set, leaving the rest of the population intact. Why do you think that is a justified step? You can dismiss the Yamal population as a whole because you believe it doesn’t have sufficient documentation, but that is no basis for excluding a particular subset.
Re: Tom P (#268), slownewsday (#265), and thefordprefect (#275),
If you go to an area and sample trees, it is completely invalid to choose the trees that match the temp record *after* seeing the data. It isn’t as if some of the trees have a functioning treemometer while others are broken. We’ve been through this extensively at CA. Select an area, sample trees, and then deal with whatever data you get. Anomalies have to be explained, not trashed.
It’s also unreasonable to assume that one can preselect large trees and obtain nice long cores from every one. It just doesn’t work that way in the field. You’re going to get short cores some of the time! So, where are they? As Steve has noted, cores are missing from the set. (Unfortunately, this is not all that unusual — much has been made here at CA over the years about how much cherry-picking goes on in some dendro labs.)
At first glance, the most reasonable-sounding argument for the biased core selection is Tom P’s suggestion:
…except that this does often produce a known bias that needs to be accounted for. The oldest trees also tend to be the most stressed due to strip bark and other factors. Thus, they tend to have a recent growth pulse due to that stress.
Thus, it only makes sense to compare samples from those long-core trees with samples from much healthier young trees. It also makes sense to obtain multiple cores from a single tree.
As we found when sampling BCP’s on Almagre, two samples a few inches apart can demonstrate the kind of divergence we see here.
Clearly, one can have little confidence in the outcome when the data from a single site varies so widely.
That’s the story here: Briffa used and published only the data that fit his conclusion. Other data from the same area doesn’t match.
This needs serious explanation. At the least, the confidence intervals appear to need a bit of adjustment.
Re: MrPete (#279),
Surely you mean fit the temperature record.
Are you suggesting that when sampling trees YOU made no distinction between wind falls/ fugus damaged/growing in rock/growing in water/lightning damged/etc. YOU simple multiple cored them all?
This of course would change the need for WUWTs surface station project. The siting/equipment quality etc. of the site is irrelevant. Statistics will give the correct result – no need to throw away the data for the thermometers sited next to an airconditioner fan in a broken screen. All data is valid!!!!
Re: thefordprefect (#282),
So how were the dead trees covering 1000AD chosen to fit the temperature record?
Re: thefordprefect (#282),
Ford, you’ve got it the wrong way round. Briffa seems to have chosen the tree data that didn’t fit the temperature record and ignored those that did. As someone else pointed out somewhere, the nearest station with a long records (Salehard) shows no 20th century temperature rise.
Re: PaulM (#291),
A simple eyeball of that graph shows a 20th century temperature rise.
Re: slownewsday (#294),
He is not the only one going a bridge too far.
.
The strongest case against Briffa’s reconstruction is to concede that trees are very weak (but noisy) treemometers, to concede a weak rise in 20th c. temps and a weak rise in tree ring widths, and to deny him from supposing a strong rise in each. This will bring the treemometer calibration slope (and significance level) way down and thus expand the (already huge) region of uncertainty around the medieval reconstruction period. Leaving one to conclude: nothing. Temperatures now are roughly what they were 1000 years ago; the uncertainty is too large to say much more than that.
.
There is no need to go a bridge too far on any of these premises. It’s the conjunction of errors that brings the alarmist hypothesis down, not any one error.
Re: bender (#298),
Nicely put bender 🙂
@Steve: I am just shocked once more. Keep up the great QA/QC work. It almost brings all the fears around bad archiving, selection bias and peer review (that have been heated topics recently) right to the forefront of the argument. You would think with the amount of money being tossed around in the realm of climate science recently that there’d be stringent and rigorous standards to uphold by now. It shouldn’t be just up to you.
Re: bender (#298),
If you cannot rely on proxy data then could you tell me how you conclude this please?
Re: thefordprefect (#306),
premise:
evidence?
Re: bender (#309), “evidence?”
The evidence is there’s no physical theory that will reliably convert any proxy metric into a temperature.
And the “reliably” is in there only to acknowledge that sedimentary or ice core dO18 can be physically related to its depositional temperature in principle, though not in fact.
Re: thefordprefect (#281), SamG (#283),
As others have noted, not necessarily. All one needs is a set of nice “upticks” in the last 150 years or so to eliminate the general divergence.
Obviously you haven’t read the backstory. Check out our metadata and compare to the metadata available from the professionals. Ours includes a reasonably extensive set of environmental context factors. Theirs rarely has anything other than a location…if that.
I’ll comment on a few of your suggestions just for fun:
1) Wind falls. It is rare for anyone to core anything other than a live or recently-live tree, simply because older trees usually rot away.
2) Growing in water. The Finns are a notable exception, having discovered caches of well-preserved trunks under water.
3) Growing in rock. That’s pretty standard fare at altitudes where the long-living trees are found.
4) Lightning damaged. Pretty much all of the strip-bark BCP’s are extensively lightning-damaged. See the photo and panorama collections and discussion here.
BCP’s make up the vast majority of the hockey-stick proxy sources; Yamal was the other “reliable” HS source. Until we amateurs came along, the metadata for Graybill BCP’s was essentially nonexistent. And now Steve is filling in the gaps on Yamal.
Bottom line: when we set out to prove the Starbucks Hypothesis, Steve committed up front to make ALL the data available. That’s what we did. That’s what scientists normally do.
Proper scientific data collection involves retaining all of the original data. How can we know how much “noise” is in your data collection process if we toss what are believed to be outliers in the original data set?
Unfortunately, some dendro’s famously disagree with this. Partly they base this on a pair of huge unfounded assumptions: a) factors affecting a tree’s growth today have always been the same in the past and will always be the same in the future; b) scientists can correctly (ahead of time) select trees to core that will be valid proxies for any given climate factor.
They assume a low uncertainty behind these assumptions. Clearly, much more caution is in order. Especially when it is now proven that they are tossing data that doesn’t fit… after the fact. Very strong evidence that their hypothesis is invalid.
This has all been done to death at CA over the years. My personal favorite analogy: how much would you trust a medical trial that simply tossed the results from test subjects who didn’t respond well to the medication under test?
Unless Briffa et al can come up with a VERY good explanation, with documentation, I think that is exactly what we’re seeing here.
Re: Tom P (#269),
Where Tom P continues to misrepresent Steve’s sensitivity analysis as offering an alternative recon.
… and tries to switch the question to whether or not Briffa’s dozen ought to be included.
That ain’t the issue, Tom P.
Jeff Id has examined both my script and Tom’s script in an excellent post:
Jeff shows that Tom continued the “combined” graphic past the point where Schweingruber data ends (1990). The elevated values in Tom’s “combined” data arise because he is using the very data in question for that portion. That portion is very similar to that derived from Briffa data because it is only derived from Briffa data.
Tom P: What a maroon.
Back on topic,
My layman’s summary of the Yamal affair.
Re: Bishop Hill (#258),
Superb story-telling. Thank you. I would, however, eliminate references to Briffa’s illness, even if you know something we don’t.
One more possible mechanism that could explain the uptick. If the larch are in a peatland setting, increased drainage would have exactly this impact. The Finns are the masters of ditching to increase growth in bog-loving larch and spruce.
Re: bender (#259),
This is a good point, I’ve been awaiting for someone to make it. As I am a Finn, I know what you are saying. That indeed is the case here.
I’m sure you already thought of this, but as far as I know, the Yamal site is covered mainly by permafrost, and as temperature goes up, it melts. Thus the water in the soil tends to gather at these melting ponds, that are seen allover the place. The area is literally covered with them.
I’m not sure if this has been brought up, but it gives a pretty good view of the area right next to Yamal; Holocene changes in treelines and climate from
Ural Mountains to Finnish Lapland: Seija Kultti
Also, here you can find some photos of the area next to Yamal; Tundra project
Re: Jarkko (#271),
no one has mentioned that mechanism yet – water availability from permafrost melting
Thanks for all the effort Steve, and the script works like a charm.
It seems odd that Tom P has now stopped posting after Jeff Id’s assessment of his data analysis. As Jeff was very complimentary I would have thought Tom P would have been encouraged to post more. Perhaps however Tom P is a little chagrined at not realising before posting a rather querulous post at WUWT that producing the hockey stick blade when combining the two data sets is because the “blade” is only from the Briffa data.
Re: Ian (#263),
Tom P is asleep. hehe.
He will claim I tricked him into using steves code. u cant use code without first understanding it.
slownewsday
What is your “perfectly reasonable explanation” for the years long refusal to supply the data requested? Dooesn’t that seem “clearly at odds” with the situation if there is a “perfectly reasonable explanation”?
Where does this put the Global Warmers?
I have just checked the page where the raw data is, and it’s interesting to see the timestamps involved with the different objects on the page. It seems to have happened happened on Sep, 8th. But what intrigues me Steve is the TayBavRing.raw file; it also seems to be new. Might it also represent something special?
Ecotretas
Tue, 08 Sep 2009 10:38:27 GMT – http://www.cru.uea.ac.uk/cru/people/melvin/PhilTrans2008/
Sat, 21 Apr 2007 07:33:04 GMT – http://www.cru.uea.ac.uk/cru/people/melvin/PhilTrans2008/EurasianGridBox.dat
Thu, 24 Jul 2008 10:29:20 GMT – http://www.cru.uea.ac.uk/cru/people/melvin/PhilTrans2008/Column.prn
Mon, 14 Apr 2008 08:01:00 GMT – http://www.cru.uea.ac.uk/cru/people/melvin/PhilTrans2008/RCS_TRW_SSA.xls
Wed, 10 Dec 2008 10:50:18 GMT – http://www.cru.uea.ac.uk/cru/people/melvin/PhilTrans2008/TornFinADring.raw
Tue, 08 Sep 2009 10:31:04 GMT – http://www.cru.uea.ac.uk/cru/people/melvin/PhilTrans2008/YamalADring.raw
Tue, 08 Sep 2009 10:31:08 GMT – http://www.cru.uea.ac.uk/cru/people/melvin/PhilTrans2008/TayBavRing.raw
One of the most disquieting comments ever presented at Climate Audit.
On what basis
On the basis of “rather doubt it” you have set the blogosphere ablaze with calls for criminal prosecution, accusations for fraud, demands for resignation, and impuned an individual’s reputation.
Having read most of this thread, I am struck by what wasn’t speculated upon:
Suppose a data set is just noise and you pick those individuals that rise at the end because they match the thermometer record. Since they are random individuals, that just happen to rise at the end, the rest of their lengths averages out to a (noisy) straight line yielding a hockey stick.
In other words a different way to accomplish what Mann did in the original hockey stick paper – turn noise into a hockey stick.
Thanks
JK
Re: jim karlock (#273),
such an original thought, you ought to publish it
🙂
Withholding data for going on a decade did enough to impugn his own reputation.
I am a biochemist and I have to agree with Jeez that wiitholding data doesn’t enhance one’s reputation and in fact in the circles in which I move you would not be able to get away with it. Incidentallly I wonder if you could take the trouble to reply to my post (266)
Steve McIntyre’s chronology above shows the data before the inclusion of the Schweingruber cores, but not after. I have shown the entire combined chronology.
Here is the period when cores from both series are contributing, namely 1780 to 1990:
http://yfrog.com/03schweingrubercru1780199p
I’ve reduced the truncated Gaussian smooth to three years to prevent the post-1991 CRU archive contributing to this series, hence the increased scatter in the points.
Here’s the entire series up to 1990 plotted on this basis:
http://yfrog.com/9gschweingrubercru0019903p
It still doesn’t look like the blade has been broke
Re: Tom P (#278),
Tom, can you explain the difference between your graphs and the red/green variants in Steve’s figure #2?
He claims the green line is a merged combination of both data sets. You claim yours is the same.
In his, ending at 1990, the merged version ends much lower than the selected-12 version.
In yours, ending at 1990, the merged version ends identical to the selected-12 version.
Why do both of your series end up with an identical high value, even though the majority of the data comes from the other cores?
(For convenience, here’s Tom P’s graph:
Tom, I don’t have your code for these updated graphs, but it would appear that you have simply repeated the same mistake: you’ve included the post-1990 period, in which the ONLY data is the selected-12 cores. Of course those cores match themselves.
The whole issue is the divergence of other data from the same/nearby sites vs the selected data.
Apparently, all you have demonstrated is that the selected data is compatible with itself.
Have I missed something?
Re: MrPete (#282),
The plot is to 1990. A different smoothing was used compared to Steve’s figure 3, three years truncated Gaussian rather than 21, so the graphs will not be identical.
The code changes to the original are trivial:
truncated.gauss.weights(21)
to
truncated.gauss.weights(3)
ts.plot(f(chron.var1$series),ylim=c(0,2.8))
to
ts.plot(f(chron.var1$series),xlim=c(0,1990),ylim=c(0,2.8))
Please produce evidence before making such an accusation. As I have shown, Briffa’s results are not particularly sensitive to the inclusion of the Schweingruber cores.
Steve: Sep 30 10.15 Eastern. I don’t know how familiar you are with R. I’m pretty handy with it by now. The code changes are trivial but don’t accomplish a 1990 cutoff. The smoothing function has endpoint padding (defaulting to the mean.) You’ve used the post-1990 undiluted Briffa data in the endpoint padding because you smoothed first rather than windowing first. You need to include the control window(chron.var1$series,end=1990) BEFORE applying the smooth function f. In addition, contrary to appearances, xlim=c(0,1990) does not place a hard control on the graph display. It will display a number of post-1990 values. You can get a hard end by also using xaxs=”i” which places the right axis dead on 1990. Or alternatively, you can window the function before plotting.
Re: Tom P (#290), just hand-scaling your plot, it appears to go past 1990.
1990 would be 4/5ths of the way between 1950 and 2000 (which would appear to be the right-hand border of the chart). It looks more like 9/10ths. I’m guessing you plotted up to 1995, with 5 years being only Yamal. That would explain the sudden match between red and black for what appears to be 5 annual data points.
Re: Tom P (#290), just hand-scaling your chart, it appears to go beyond 1990.
I don’t know if it’s a programming error, or if it’s over-riding your “ts.plot(f(chron.var1$series),xlim=c(0,1990),ylim=c(0,2.8))” command for some reason, or what. But it’s not stopping the plot at 1990.
Re: Tom P (#290),
The evidence has been produced and your inexpert monkeying with code you don’t understand is only going to get you deeper in trouble while wasting people’s time that could be better spent auditing work that matters. How many errors do you need to make before you give up? A half dozen. A dozen? Have you no shame?
Re: Tom P (#290),
As Bender points out, Re: bender (#287), you seem not to understand the difference between sensitivity testing and actually producing an alternative temperature reconstruction. Steve is trying to see how much a given temperature reconstruction (like Yamal) relies on a particular site or sites. This can most easily be done by removing one or a few sites at a time and seeing the results. In this case removing the small sample of recent trees and replacing them with a reasonable substitute results in totally reversing the HSness of Yamal. This test result is not a replacement for Yamal, it’s an indication that Yamal is probably too sensitive to use in a definitive paleoreconstruction. Unfortunately, Yamal has been used in almost all the paleoreconstructions which don’t rely on the strip bark BPs. This means that, in essence, there are no paleoreconstructions which can be relied on. This in turn means the claim that the recent warming is extraordinary is not proven. snip
Re: Dave Dardinger (#296),
Even that is conceding too much. Briffa’s recon is flawed because any reasonably-sized sample that includes his very few Yamal cores is dominated by the HS in those Yamal cores. Therefore those cores cannot stand alone as a climate proxy. That is not saying they shouldn’t be used as a small part within a much larger sample. Briffa’s trees are SO special that they appear to have been selected precisely for that reason.
Re: Tom P (#290),
This is one of the most disingenuous things I’ve ever seen done by someone who has a technical background. You’ve presented a graph with so much noise that any blade cannot cross statistical significance, provided a magnified Y scale while removing the x axis and I note you refused to acknowledge your mistakes in your last accusations.
What a friggin, joke. Why not show the graph pre-1800 Tom. Why this deliberate twisting of facts? What’s the point of the blatantly false spin Tom.
My opinion of you is dropping rapidly and I find myself squarely in Bender’s camp now.
Aw screw it, I’m just wasting my time.
Re: Jeff Id (#307),
Yes, Jeff Id and Steve M, I think any further dialogue with Tom P is wasting your time. Until Kenneth Fritsch chimes in otherwise, I would forget it. Tom P is guilty of confirmation bias every time he posts. A graph pops up that he likes, and that’s his new defense. The errors are revealed after careful scrutiny. A new graphic & defense emerges. The pattern of incompetence and distortion and denial is established. What value could Ken possibly derive from this? Me, I hate the sight of so much blood and gore.
Re: bender (#310),
I think if you send Tom P away at this point you are letting him off the hook without a finale. It should be understood how he is proceding and why, and not have him leave without clearly getting that information into posts here. Tom P in my mind is handling this issue not unlike some more-timid-than-he climate scientists would.
Re: Kenneth Fritsch (#314),
Define “finale”
Re: bender (#315),
I’ll give an example:
Re: Tom P (#317),
I understand what Tom P has done. The differences between his and Steve M graphical presentations are in the definitions of what one wants to show, not in the application or errors in the R code. Why Tom P wants to show the data the way he does when the intent here is to show what the combined data looks like from two different proxies (a sensitivity test) both separately and together is beyond me.
I do find, without a reasonable explanation, that Tom P’s not showing Schweingruber as a stand alone series in his graphic presented above, disingenuous and misleading.
Re: Kenneth Fritsch (#327),
I made a very minor tweak to the code ten minutes after downloading R to include the 12 cores back in the reconstruction – nothing else in Steve’s script was changed. The appearance of the plot was chosen by Steve.
If you want to present this tweak in a different way, be my guest and get working on R yourself!
Re: Tom P (#328),
Blame-shifting. You did not understand the modified instructions that YOU fed to R. YOU jiggled the code and YOU misinterpreted the output. The output and the misinterpetation were yours alone.
Re: Tom P (#328),
A “minor” “tweak” whose non-trivial consequence you could not predict and did not understand. Your errors are understandable, but not excusable.
Re: Tom P (#328),
It’s not clear to anyone what your “tweak” was intending to show. If you ask a well-posed question, I’m sure someone will be happy to modify the code to produce the graph you want. Especially if the question is relevant.
.
Just stop pestering Steve and Jeff. [And if that’s your objective, just say so.]
Re: Tom P (#328),
Taking on the code yourself, even if you made a mistake, is something to be applauded. It has been applauded in this thread.
You are being criticized, in my opinion quite properly, for two things:
1. You have repeatedly claimed that the purpose of your code is to “include the 12 cores back in the reconstruction”. This implies that Steve’s original analysis did not show the impact of including the 12 cores. It plainly does. This has been pointed out to you repeatedly by Steve and others. yet you continue to act as if Steve somehow omitted this analysis.
2. You have stubbornly insisted that this “very minor tweak to the code ten minutes after downloading R” had some how invalidated Steve’s point, even after multiple problems (such as the [accidental?] inclusion of post 1990 data) had been pointed out to you, and then again refused to acknowledge this.
At this point, absent some acknowledgement of these errors, people are going to start ignoring you; assuming that you are either a troll, or too stubborn to be constructively engaged.
Re: Jason (#338),
Jason,
My other comments appear held up in moderation, but to answer your points together:
Yes, Steve included the twelve cores in a previous plot but I wanted to see what happened to the 21-year smoothed data beyond 1990 when both sets of cores would still be contributing. There is no reason to throw out the data past 1990, even though the number of cores has fallen. A full analysis would weight the data according to its statistics, both the number of cores contributing and the spread in the data. This has yet to be done.
It was not incorrect to include the post 1990 data, but it would have been better to include some confidence measures. As no error bars have been given on any of these reconstructions to date, this omission is not just mine.
But if someone can show that there is a number below which no core series can be considered valid, I am willing to be educated. And the valid time period of the original Yamal series could have been determined in 2002 when the core counts were first published.
Re: Tom P (#349),
Be patient. These calculations are happening in real-time.
Re: Tom P (#349),
Proper plotting of confidence intervals will always take care of that issue. The real problem is not the number, but what subsample that number might be coming from. The confidence intervals on the CRU HS blade will be relatively tight. That doesn’t mean it is a representative sample.
Re: Tom P (#349),
It’s a sensitivity analysis, you cannot conclude that the curve is insensitive by using only the original data? Therefore you cannot include data past 1990 as there is nothing to compare! Why have you persisted in a self contradictory argument.
Nothing makes sense.
Re: Jeff Id (#355),
I agree. But by the same token, sensitivity to an alternative series can only demonstrated over the period of overlap. If the original analysis had been restricted to this period of overlap, then there is no confusion. But plots here have included data both before and afterwards.
I understood that the aim of this audit is to test the robustness of the entire Briffa reconstruction. The sensitivity to the Schweingruber series can be used from 1780 to 1990, and a different approach should be used elsewhere.
Re: Tom P (#359),
So since you now admit that your method is not valid for a sensitivity test — do you now retract this statement based on your incorrect sensitivity analysis at WUWT:
Re: Jeff Id (#371),
No, my statement stands. What I have presented is just a plotting of the full dataset.
But, I admit that unless there is data to contradict the post 1990 record, there will continue to be a very pronounced recent rise irrespective of any previous reconstruction.
However, if there are reasons to completely discount the post 1990 data, I have no reason to think the black line of your plot at
is not a reasonable reflection of the entire Yamal dataset.
One reason for discounting the last part of the dataset has been advanced, broken labelling core sequences. This has been shown to be specious – it would reject the Schweingruber series as well.
But Steve has stated another reason:
I’d like some specific numbers put on this with respect to the post 1990 data. If indeed these numbers show the minimum number has not been met, and this data should be completely discarded, I’m certainly willing to accept the black line of your plot above as a good reconstruction based on all the available data.
Re: Tom P (#383),
figured out RCS yet? read that article I gave you? look at the individual series yet? understand what low initial growth does to fitting an negative exponential? understand how this impacts late growth corrections? understand how one tree can mess up this methodology?
Re: Tom P (#328),
The appearance of Steve’s plot was chosen by Steve.
The appearance of Tom P’s plot was determined by the tweaked computer instructions submitted to R by Tom P.
There is no “the” plot.
Such dodgy semantics. Is that you, Bill? (And by “is” I mean …)
Re: bender (#338),
Tom Tom the pipers son
downloaded R and away he run,
The graph’s a cheat and Tom is beat
But Tom is roaring “I’m complete!”
Re: Tom P (#330),
Tom P, you have not explained why you did not include the Schweingruber series as a stand alone on your graph as Steve M has on his. If you had done that, there would not be any confusion about what you were showing and any differences that might have occurred between your depiction and Steve M’s. I find your reply as disingenuous and evasive as your graph.
Re: Kenneth Fritsch (#362),
Simply because the code at the top of the thread plots two datasets one being a composite and one the CRU archive. It’s certainly not an unreasonable comparison.
Re: Tom P (#365),
Depends. Are defending a position as an adversarial lawyer would and/or marketing an approach as a marketer would or trying to understand a sensitivity test that is the subject of this thread?
Re: Kenneth Fritsch (#326),
I understand Tom P’s argument. After much back-pedalling, he is saying that there is no good reason for excluding Briffa’s sub-sample from a chronology intended to cover the area we are referring to as “Yamal”.
.
This, of course, has nothing to do with the question of whether Briffa’s sub-sample is representative of the trees in that broader area called “Yamal”. Which is the question Steve asks.
.
Tom P’s point, as I rephrase it here, is arguable, but not incorrect. Unfortunately, it is irrelevant to the independent point that Steve is making. Briffa’s sub-sample is not representative of other samples taken from “nearby”. QED.
Re: Jeff Id (#308),
Your tirade is groundless:
The noise is a direct result of reducing the smoothing window to avoid contamination from the post 1990 data.
I have not changed the vertical scale.
I have not removed the x axis
I showed a plot of the pre-1800 data above.
I presume by mistake, you mean not terminating the series at 1990. That wasn’t a mistake – as I made clear the reconstruction is built on the CRU and Schweingruber dataset in its entirety. Of course it is identical to the CRU reconstruction after 1990.
But why should the CRU dataset be disregarded after 1990 if it was valid before then? There seem to be no strong a priori reasons to exclude either the CRU or Schweingruber datasets at all. It was Steve who discarded the live cores from the CRU dataset, replacing them with the Schweingruber series, but if the reason is incomplete label sequences then the Schweingruber live cores are out as well.
I have made no selection criteria in including data from either series. If you, Steve or anyone else wish to deselect part of any series, please make the criteria plain so they can be validated. Anything else is cherry picking.
Re: Tom P (#317),
Because of the enormous weight placed on it relative to the other, far more numerous, samples which all end at that date. This allows the bias that Steve M is pointing to to dominate the ensemble in the last few years.
.
Please stop. Or give us the finale Ken seeks.
Re: bender (#319),
We should be reluctant to throw away any data just on the basis that there isn’t enough of it!
But any plot should weight the data by the strength of the statistics, and hence the declining numbers of trees in the last decades, in both series, should be used to:
a) appropriately weight any smoothing function;
b) increase the error bars – though these could do with some calculation in the first place!
Re: Jeff Id (#321),
The increase in the RCS chronology index extends back before 1990, so more than a couple of trees contribute.
Re: Steve McIntyre (#323),
Yes, I see that xlim does not produce a hard cutoff. The truncated gaussian windowing of 3 years should have prevented the post 90’s data getting much into the plot, though.
Of course you plotted a merged dataset first, but as I say above I don’t think it is right to discard the post 1990 data – it should be suitably weighted. This would be true whatever combination of series were selected. It will take someone with more than 24 hours exposure to R to achieve that.
Re: Tom P (#317),
As I’m sure now you are aware, the point is to show that the entire blade is based on a couple of undocumented trees. You put the trees back in and breathlessly claimed victory like an excited school girl. Then you fail to admit that you cannot disprove a sensitivity study by using only the original data post 1990. – a mistake!!! Then you do it again, repeating the same error with noisy filtering that still pulls in the Yamal only data for the last few years.
Not fooled big guy, not one bit.
BTW: If you thought that was a tirade, you should have seen the original version.
Re: Jeff Id (#309),
Jeff I wrote a post on WUWT for Tom P. I can’t recall if I posted it or not but basically I predicted that he wouldnt admit his error and would continue to flounder and would waste peoples time with his Trolling.
Re: steven mosher (#375),
We’ll see soon enough, but if I had to guess….
Since he admitted it wasn’t a correct method for sensitivity analysis, it shouldn’t do any harm to retract his statement about the ‘sensitivity’. That might go a long way for his credibility at this point.
Re: Jeff Id (#377),
He has no credibility quatloos left. He can only hope to save some face by showing he has an open mind. Imagine, coming in here demanding confidence intervals (when I already suggested that), but not daring to ask the same of Briffa!
Re: Tom P (#288),
Tom P, if your x axis to your graph is to scale the graph goes by the 1990 date to something in between 1990 and 2000. Please explain.
Re: MrPete (#280),
MrPete’s comment in the post above would appear consistent with the use of data beyond 1990 and what data was used.
Can you post all the R code that you used? As Steve M would remind that is the beauty of using R.
Re: Tom P (#288),
Tom P has pretended that doing a merged version was his idea. In my original post, I illustrated the effect of the merged series up to 1990, when Schweingruber cores end, observing that after 1990, there are only the CRU cores and the “merged” version would simply be the CRU version. The caption to Figure 3 stated:
Tom P asked me to provide a closeup and I provided him with a plot showing the contrast between the two sensitivities, observing that the green would be in between. Then he asked me for a closeup showing the green as well. While I try to be cooperative, I don’t have time to do everything and I referred Tom to my code.
As Jeff Id observed, Tom went to the trouble of getting the code and doing a graph. But he’s new to R and made some mistakes. Fine, but he’s been over-assertive about the results from those mistakes and, needless to say, people seize on such straws.
I observed inline about Tom’s code:
While Tom may think that his code accomplished a 1990 cutoff, it didn’t. As I originally observed, the closing portion of the Tom’s curve looks like Briffa’s data because it is Briffa’s data – apoint that I had avoided this error in my original Figure 3.
Additional to this are of course various comments by bender, Jeff Id and others. I am not proposing that a chronology using Schweingruber’s 34 cores rather than the 12 selected cores in the CRU archive is a valid alternative temperature reconstruction, nor do I view a blend of the CRU and Schweingruber archives as some sort of valid compromise.
Are cores taken from old (dead) wood? I’m assuming that considering the precarious nature of some of these ancient trees, one wouldn’t core through the only living strip of cambium.
So regarding this ‘old’ wood, how does one tell the age of the peripheral layer. You’d need to know at which point it stopped living. (snipworthy?)
There has been a lot of comments from Steve about trees that he thinks aren’t good proxies and should be discarded.
I would say this is wrong. He has two graphics at the beginning of this post with the Schweingruber cores included; Merged Graphic
Selection bias (e.g. Berkson’s bias)is a statistical bias is which there is an error in choosing the individuals or groups to take part in a scientific study. It is sometimes referred to as the selection effect. The term “selection bias” most often refers to the distortion of a statistical analysis, resulting from the method of collecting samples. If the selection bias is not taken into account then any conclusions drawn may be wrong.
Re: Kevin (#297),
The problem with that term – selection bias – is that you don’t know for a fact that there was a selection process to bias. You don’t know who the selector was or what selection criteria they might thave used. You can only speculate. Steve has speculated as to the population from which a selection might have been made, and as to the selection criteria that might have been used. But he has identified his guess as exactly that: a speculation. People accusing anybody of wrong-doing are going too far. As Steve has indicated, there needs to be an accounting of who did what.
If Briffa’s “few good men” are NOT extraordinarily divergent from the other samples, then why does he have a research project to study inexplicably divergent behavior in tree rings?
.
The inexplicable divergence phenomenon is a serious problem for dendroclimatologists. That a whole chronology was clearly founded on a select subset of divergers (however they were chosen) is a serious problem for Briffa. If there is a good explanation, then where is it, and why didn’t it come 10 years ago?
it seems denial is infectious p
Re: John P (#301),
so is belief
Isn’t it a bit silly that this much debate is needed on this topic; I mean, hasn’t it been shown that two cores from the same tree will yield different results? After that, it just becomes a game to prove something that we all know…
Re: Carl G (#304),
Not worthy of response.
What effect does the combining the Yamal and the Schweingruber cores in the TomP reconstruction have on the error bars in comparison to the original Briffa reconstruction?
MRe: bender (#310),
Maybe Steve would consider snipping everything in my post before “just”. I do have a bit of a temper.
Gore makes me nauseous too.
Somebody hand Tom P a skil saw so he can cut off his keyboard fingers.
It’s only a flesh wound …
#317: The criteria for “deselecting” is obvious, as it was the stated goal of this exercise: Compare Briffa’s Yamal to Schweingruber’s Yamal. Obviously, you can’t compare two things when there’s only one present (post 1990). The comparison ends at 1990… the whole point of this is to show that Briffa’s and Scheweingruber’s trees are materially different. Nobody has explained why they are different, and yet, almost the entirity of temperature reconstruction studies are based upon using one over the other.
The divergence starts ~1918-22 and ramps up significantly ~1946+
win.graph()
ts.plot(f(chron.yamal$series)-f(chron.var1$series),xlim=c(1900,2000))
abline(v=10*c(190:200))
Curious that both episodes are post-war
Is there a military base at Yamal? Or any human development that could have resulted in nitrification or a change in drainage (e.g. road construciton) at these sites in those years? We need to know exactly how these trees cluster. We need to know what led the russian dendros to that specific site. Did they core trees near a military camp where they were lodged?
Re: bender (#323),
There’s a curious thing conserning that. The Yamal Peninsula area has resently been closed from foreigners, scientists, journalists, etc., by Russian government. I meen from everyone, exept the natives. It’s full of oil and natural gas. There has been complaints from natives, that the roads and pipelines are harming the migration of the reindeers.
So, I think no army base, but oil- and gasfields.
Congratulations! And Sherlock Holmes would be proud to agree!
Bob
Tom P’s dodginess is reminescent of Lee. Without the typos.
Tom P: it is customary for rational people to acknowledge their errors before attempting to move on. Since you have been repeating yourself, seemingly endlessly, may we assume that you are finished and that no such acknowledgment will be forthcoming?
.
I don’t mind the odd sock puppet performance, but repetition is BORING.
You see, Ken, it never ends …
Bender you may not see it, as for you the mistakes are obvious, but for layman, Ken’s view is correct, if everything was all light, we could not distinguish anything, the villian extolls the hero, the light is seen better in contrast. I learn a great deal when I see you guys deconstruct an invalid criticism, and I understand far better what Steve M did here. So thank you for your patience in ansering Tom P, for on more public blogs his methods will create confusion, here his errors come into sharp focus.
For those who are new to CA, here are links you might find interesting to see the work done by CA’s amateur volunteers. We make no claim that this is professional-level work; only that it was sufficient for our purposes. Nothing hidden or held back. This is collaborative online citizen science in action.
Here’s a link to the metadata for Almagre BCP’s collected by the Climate Audit team (and audited by CA readers).
The same folder contains the other data files as well.
Now I’ve got to get back to my “real world” :).
[NOTE: please do not begin a discussion of the Almagre data. My only purpose in providing these links is so that new folk can more readily access some of the backstory. Steve and CA readers have been working with dendro data for quite some time, in some depth.]
Bender wrote:
The strongest case against Briffa’s reconstruction is to concede that trees are very weak (but noisy) treemometers, to concede a weak rise in 20th c. temps and a weak rise in tree ring widths, and to deny him from supposing a strong rise in each.
In one of my earlier comments, I got cut off by Steve (deservedly) for asking for temp data from the region (didn’t want to do my own homework). I was going to get to Bender’s point but I didn’t have the data at the time and never completed the inquiry. The tail (blade) end of the twentieth century, in the various graphs, the extreme rise shown in Briffa’s chosen proxy set seems exaggerated when compared to the very modest rise in actual, real life temp instrument records.
Re: Sonicfrog (#337),
In calculating correl(x,y) the steeper the time-series trend in x and/or y, the (seemingly) stronger and more significant the correlation. Hence the constant fuss here about “red noise”. If x is serially autocorrelated and y is serially autocorrelated, the correlation coefficient of correl(x,y) is biased high. A hockey stick blade is the strongest red noise possible. In a stochastic series, as the slopes of x and y are brought down, any stochasticity (white noise) starts to fuzz up the correlation. That’s why people like to smooth out the white noise. It inflates their correlations.
.
If Jones juices the temperature record and Briffa beefs the tree proxy, it inflates the proxy’s apparent predictive power. Boeuf au Jus. Yum.
Re bender #323 and nitrification: The expansion of the oil and gas industry on the Yamal Peninsula has forced ever-growing numbers of semi-domesticated reindeer onto ever-smaller areas of grazing land. Even in the early 1990s, the reindeer population was reckoned to be twice the optimum, and there are now even more reindeer on even less land. So it might be something to do with poop density. Or not.
Re: Viny Burgoo (#340),
In some early CA threads, the impact of late 19th century sheep grazing on Western tree growth was discussed. They ate off competing weeds and shrubs and led to a pronounced growth pulse in some Western pine species. (Craig Allen, as I recall, wrote on this. There’s a mention in MM05 (EE)).
I like people arguing with me, but I dislike people misrepresenting events. In my post, Figure 3 showed a Figure with the “merged” data set, noting that no “combination” was calculated after 1990 when only the Briffa data in question was available. Here’s another misrepresentation by Tom:
This is blatantly untrue. Figure 3 showed the plot in question. My posted code in comment 1, while very complete in the scheme of things, unfortunately left out the code for doing Figure 3 with proper windowing at 1990. I’ll add it in.
Tom’s requested a blowup of Figure 3 and here it is, adding in annual data. Legend as in Figure 3. The “combined” information is shown to 1990, since post-1990 is, as noted above, limited to the CRU version and, obviously, reverts back to the CRU. The green curve, as I said previously, is between the red curve and the black curve.
When the Briffa Yamal curve is kaufmanized, it ends up at nearly 7 sigma in its closing portion and contributes nearly 40% of the closing value in the Kaufman composite all by itself. This sort of value is going to be impossible to save.
Re: Steve McIntyre (#343),
The annually resolved data show the major divergence pulses to occur in 1922, 1957, 1972. I wonder if this is correlated with oil & gas disturbance history re: seismic lines.
Re: Steve McIntyre (#345),
I should have made the context of my request for a plot a lot clearer – I apologise. On the other hand, you took my earlier comment about the selection of the 12 cores out of context and I’d like some acknowledgement there.
It’s all too easy to forget that readers will not have been following all the threads, especially distributed amongst more than one site! I’ll try to be more careful.
Thanks for the plot and code. I’m unsure how to add in the correct truncation into the original script. Truncation is quite important here.
Could you enlarge a little, if just by a link?
Try to screw up more, we’ll be happy to help you out. 🙂
Steve, is it ok if people like Jarkko and Viny submit factual reports about human activites and other possible disturbances in the Yamal region? Not to encourage dendro-rumination. More as a “proxy” for proper metadata.
RE Bender, #294, 313, 316, 325, 327, etc,
Please be polite to visitors, even if you find them to be exasperating. How can we expect people like Steig, Kaufman, Briffa, etc, to come and explain what they did if they just get flamed whenever we disagree with what they say?
In the present instance, Briffa may unfortunately be too ill to participate right now, even if he were so inclined. But TomP sounds like he is genuinely interested and willing to engage us, so let’s just hear what he says, and correct him if he is wrong (as he apparently was with his original graph).
Re: Hu McCulloch (#348),
Ok, Hu. Sorry, folks.
Re thefordprefect #307,
This is getting OT, but nontreering proxies tend to tell a very different story than the dubious Mann/Briffa treering proxies. See Craig Loehle’s (2007, 2008) reconstruction (to which I added standard errors in the 2008 correction).
Re: Hu McCulloch (#357),
Corrected link: Craig Loehle’s (2007, 2008) reconstruction
Tks! Correction made. — HM
Little Bit Pregnant! 🙂
Pielke Jnr has picked up on this.
He’s not commenting, but does suggest that a paper should be prepared on the scienctific bit and submitted to a major journal.
http://rogerpielkejr.blogspot.com/2009/09/has-steve-mcintyre-found-something.html
Standard dendro practice requires a minimum number of cores depending on the consistency of the core “signal”. The usual practice in a dendro site article would be for the original author to discuss this issue and clearly disclose core counts. The number in an RCS standardization is held by specialists to be considerably higher than in a “conventional” standardization. If you can get a dendro to endorse a 10-core RCS construction, please do so.
Again, you have blatantly misrepresented the story. Core counts for Briffa 2000 subset were NOT published in 2002. Hantemirov and Shiyatov 2002 do not cite Briffa 2000 nor state that Briffa 2000 used their subset. The counts for the subset used in the Hantemirovc and Shiyatov reconstruction as archived do not match the CRU counts.
In retrospect, someone might have guessed that the Briffa 2000 subset counts would be low, because the H and S subset counts were low, but that would have been a guess.
The underlying problem is that Briffa 2000 is not a proper publication of the Yamal series – but people used it anyway.
At least some specialists in the field were not aware of how low the Yamal count was for the Briffa study (I have some sources that aren’t prepared to be identified publicly).
Re: Steve McIntyre (#361),
The Hantemirov and Shiyatov 2002 statistics could have been questioned in their own right at the time of publication. So was it only in 2008 that it was evident these were the relevant cores for the CRU series?
It certainly appears that the recent core counts are very similar between these two series.
I knew this quote would come in handy one day…
What makes one regard philosophers [climate scientists? Tom P?] half mistrustfully and half mockingly is not that one again and again detects how innocent they are – how often and how easily they fall into error and go astray, in short their childishness and childlikeness – but that they display altogether insufficient honesty, while making a mighty and virtuous noise as soon as the problem of truthfulness is even remotely touched on. They pose as having discovered and attained their real opinions through the self-evolution of a cold, pure, divinely unperturbed dialectic … while what happens at bottom is that a prejudice, a notion, an ‘inspiration’, generally a desire of the heart sifted and made abstract, is defended by them with reasons sought after the event – they are one and all advocates who do not want to be regarded as such, and for the most part no better than cunning pleaders for their prejudices, which they baptize ‘truths’. (Nietzsche, 1886, Beyond Good and Evil)
Tom P, let me paraphrase what you seem to be proposing as a valid method – just in case there is a disconnect in the way people are thinking:
For a smoothed 1990 value, depending on the choice of data-set up to 1990, one could arrive at 0.6, 1.4 or 2.0 as the proxy mean. There may be no good way to prefer either the high or low dataset, and the mean of all data collected up to 1990 seems the most reasonable.
We then wish to extend the series with an extra 10 years worth of a smaller subset of data – which would give a 2000 endpoint at around 2.4, even though we know that that particular series on its own was a fair bit higher than the best guess indicated by the larger series.
Is your thesis that the other series, if they could be extended would now show growth that exceeded the 12 samples which are on record, and so compensate for the lower combined sum in 1990? I would be tempted to graft the extension onto the 1990 mean, reducing the 2000 endpoint by (2.0-1.4) to come up with 1.8.
Apologies if this is statistically un-justifiable, but it seems that the point which TomP is trying to make is making no progress (backwards or forewards) and this might nudge it towards one end!
Probably should lurk more, but:
Is the Briffa Yamal series from 12 TREES or 12 CORES?
IIRC, the common practice is to take more than one core per tree.
So it’s possible that the cores that were used are not even representative of the tree’s growth.
I mean if they are good treemometers, then all cores from that tree would show the signal, not just those from one side.
As far as mapping against a temperature record, the Salekhard weather station, 100 miles from the center of the Yamal area, does not show the warming visible in the proxy.
I hate to butt in on the Tom P saga, but I have a question. If trees could be a reasonable proxy for temperature it would seem to me that the procedures for selecting and combining tree ring samples would have to be extremely stringent.
Trees selected for the calibration period would be selected based on a minimum statistical significance to the local temperature record and cross validated to verify their statistical significance to each other during the period prior to the calibration period. Then older dead tree samples that overlap a reasonable number of years of the life span of the calibration trees would be selected based on their statistical significance during that period of overlap and then compare to each other to determine their statistical significance to each other during the years prior to the overlap. This process would be repeated as the reconstruction is extended back in time. Finally, how well the samples correlate during each year would be used to determine the margin of error or confidence interval.
Is that a reasonable required progression or am I missing something?
Oh, forgot that the number of samples used per period would have to be uniform to reduce data suppression due to divergence.
Tom please go read some threads on RCS. Please go look at an individual core ( it’s actually quite fun, steves& petes cores are a good place to start ). Then you can see why using 12 cores or 5 cores is pure folly with RCS.
Watch this.
Re: Tom P (#381),
What about the fact that up to 1990 the two series so strongly disagree?
It is easy to overuse or misuse the quotable Feynman, but if there were ever a time and place to re-roast an old chestnut, that time would be now, that place would be here.
“We have learned a lot from experience about how to handle some of the ways we fool ourselves. One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It’s a little bit off because he had the incorrect value for the viscosity of air. It’s interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan’s, and the next one’s a little bit bigger than that, and the next one’s a little bit bigger than that, until finally they settle down to a number which is higher.
Why didn’t they discover the new number was higher right away? It’s a thing that scientists are ashamed of – this history – because it’s apparent that people did things like this: When they got a number that was too high above Millikan’s, they thought something must be wrong – and they would look for and find a reason why something might be wrong. When they got a number close to Millikan’s value they didn’t look so hard. And so they eliminated the numbers that were too far off, and did other things like that. We’ve learned those tricks nowadays, and now we don’t have that kind of a disease.”
The trillion dollar question is whether climate science will survive the cure?
Tom P, I just went through all the Steve M code from Post #1 above in R and printed all the plots. I did not see the composite and CRU series alone in any of the plots. What code are you specifically referring to?
I’m just cranky cause *I’m* the one has to slog through Steve’s dense R script trying to figure out how to access and reformat the vectors buried in his weirdly structured data frames in his RCS.chronology() function to calculate confidence intervals. While Tom P and Sebastian are sweetly dreaming up their next drive-by insult. Thank god mosh is here.
Fordprefect pointed out this paper by Hantemirov July 13, 2009
Click to access KHantemirovRM.pdf
It mentions taking samples from more than 100 live larches.
Also, references temperatures from Salekhard station, as Lucy Skywalker did.
Also, gets a hockey stick shape, with other peaks at 250 and 1400.
There appears to be some discussion of the 20th century warming, but my translator can’t handle.
Re: MikeN (#396),
That is pretty darned interesting. Maybe the permafrost-melting hypothesis of Jarkko can be shown to be widespread, once, like Hantemirov, you know what sites to cherry pick from. Translation would be great.
TomP 395
If YAD06 is removed how pronounced is the hockey stick you refer to in your comment “In this case the data shows it is not a hockey stick that collapses, but the case against it”
And in any event how does the case collapse as the Schweingruber series does cover the 20th century and doesn’t seem on eyeballing to have a marked 20th century uptick
Re: Ian (#397),
Because trying to discern a centennial or multicentennial trend using noisy samples less than one hundred years long is next to impossible.
Here’s a simple example. Imagine a noisy, gently upwards trending signal one thousand years long. Chop that signal into one-hundred year segments and set the average of each of these segments to zero. It is very difficult to then recover the initial trend by then stitching together the segments.
Also. . .why can’t trees move in and out of accuracy as thermometers over their lifespan? Thus making it entirely possible that the conditions that make certain trees good proxies for twentieth century temperature records might have made OTHER trees good proxies for earlier temperature records and bad proxies for modern times: Just because a certain tree matches up NOW doesn’t mean it matches up THEN. Not speaking from a highly technical standpoint, but being a horticulturist I know at least something about plants and trees. .and the number of reasons a trees rings could vary for any given period is rather large and temperature is only a small subset of that variance. Selecting the trees because they happen to agree with something on the order of ten percent or less of the sample seems a bit dodgy.
Re: Ryan (#400),
Not “also”. Failure to comply with the uniformitarian principle has been discussed ad nauseum. Newcomers should ride the blog.
One Hantemirov paper cited is this one:
That would be worth a read. For sites on the treeline margin it wouldn’t take much warming to make those sites moist and habitable. Hantemirov might have a talent for spotting such hyper-sensitive species-site combinations.
And even read it too.
Did Hantemirov archive his data? And if he did, perhaps it still requires translation?
Well, pretty much means newcomers aren’t welcome, because if you have to read everything ever printed on the blog before commenting on any of it, you aren’t going to get many. Snide derision just isn’t very attractive.
Re: Ryan (#405),
Snide derision? I would simply suggest that if you read the blog your subsequent commentary would be much more informed and that much more welcome. Then the regulars wouldn’t have to read the same comment over and over and over. (What is it makes people think they’ve thought of something no one else has before?) Read. Then write. Meanwhile, if anything I say is redundant, please let me know. Enjoy.
Call me crazy, but I was always under the impression that if the majority of a population has to be removed for a trend to exist, then does that not mean that the trend is in fact a result of the selection process, rather than being an actual observed trend? I would much prefer endless core samples (say >1000) and then plot that. Any trend in >1,000 cores has to be of interest. It could relate to a multitude of environmental factors, but at least we wouldn’t be discussing what sounds like a few trees from someone’s backyard that affect some of the most influential data out there when it comes to current goverment policy making.
How much does 1 core sample cost and how long do they take to get? Surely you could get at least somewhere in the hundreds from a few days effort. Yeah of those quite a few might be rejected for being contaminated, or failures during the actual sampling, or incorrect procedures. But I’d be willing to bet you could get more than 12 valid samples.
Re: Craig (#406),
Read the 2007 Almagre Adventure story postings here to get a sense of the answer to your sampling question. For a small experienced team, yes the number of cores obtained could easily reach hundreds with a few days of effort. We only obtained 64 cores from 36 trees (of 68 identified) (see the xls file here)… but then we were more about having fun, tham exactly working 🙂
Thanks to TheFordPrefect on another site…
Briffa has responded…
http://www.cru.uea.ac.uk/cru/people/briffa/yamal2000/
Bender, that paper is the 2002 H&S paper. There doesn’t appear to be any data, beyond the core counts. These core counts seem to be in conflict with what Steve is reporting, as I never see the numb er rise above 20.
Regarding signal-to-noise, shouldn’t they take the trees that match the local temperature record, as being the most valid? Then they assume that the trees that good now are good for their lifetime.
Then they match dead trees to these good trees, for the period 1800-1850, and use those good old trees to go back to 1600, and keep repeating.
Paul,
if you graph the annual anomalies, you do get some warming at Salekhard station.
The 60s were cooler than the 70s were cooler than the 80s and 90s, with about the same temperatures in 1980-2000 as 1925-1950.
Can someone here make out what Briffa is saying in his just-released statement? It’s written in obfuscatory language, seeking to hint at rather than state.
Separately: how can it be that the state of our knowledge of the temperature circa 1000 comes down to an analysis of a handful of larch trees in the Urals? Musn’t there be dozens of converging data from many different sources (sea beds, ice-cores, let alone written records) all establishing (or refuting) the existence of a medieval warm period, and of its extent? How could it have been common knowledge until now that there was such a period? Doesn’t the Hockey Stick deny what everyone had known even before Biffra?
Re: Harry Binswanger (#416),
Dr. Binswanger:
Fortunately, our state of knowledge about the MWP is not limited to this particular handful of larch trees. Here is a link to a study done by Loehle, C., and J.H. McCulloch (both post here frequently) that demonstrates a quite coherent MWP that is warmer than present day:
http://www.ncasi.org/publications/Detail.aspx?id=3025
For those who actually DO want to know what the folks at RealClimate said, see:
http://www.realclimate.org/index.php/archives/2009/09/hey-ya-mal/#more-1184
I dare Steve to put a link to the RealClimate article here, the way the RealClimate guys did to Steve’s dubious “science.”
Steve: I have regularly linked to realclimate. Earlier today I wrote a response to Briffa, linking prominently to his response and urging CA readers to read his article. I offered Briffa unfettered editorial access to post his own thread here to this audience. I will respond to the RC article and link to it – don’t worry about that. But there’s only so much I can do at once and responding to Briffa’s response seemed more important. Pielke Jr has commented on RC here.
Re: Geno Canto del Halcon (#417),
Please point to the source of this “dubiousness”.
Note to Steve M:
Re: plotting confidence intervals, I need help with your code.
Why do objects “tree” and “chron.var1” contain different numbers of elements (41483 vs. 41325), whereas “yamal” and “chron.yamal” contain the same (40892)?
This causes the following statements to succeed vs. fail:
tapply(chron.yamal$delta,factor(yamal$year),mean) #WORKS
tapply(chron.var1$delta,factor(tree$year),mean) #FAILS
… because the factor() statement doesn’t match up in the latter case.
Email me if you prefer.
Until then, I’m hooped.
Another paper Grudd, H. 2008. Tornetrask tree-ring width and density AD 500-2004 with 880 individual tree samples that in the conclusion says:
“The late-twentieth century is not exceptionally warm in the new Tornetrask record: On decadal-to-century timescales, periods around AD 750, 1000, 1400, and 1750 were all equally warm, or warmer. The warmest summers in this new reconstruction occur in a 200-year period centred on AD 1000. A ‘‘Medieval Warm Period’ is supported by other paleoclimate evidence from northern Fennoscandia,although the new tree-ring evidence from Tornetrask suggests that this period was much warmer than previously recognised.”
Sorry forgot the “divergence problem” quote from the Grudd paper:
“Tornetra¨sk MXD does not show this ‘‘divergence problem’ and hence produces robust estimates of summer temperature variation on annual to multi-century timescales”
Grudd was reported on first at CA.
Have not seen the link to the published paper before, but that is probably my mistake. Take a lot of time to catch up.
But it is interesting to read the CA link again Tornetrask Digital Version – Hooray! with this new Yamal/Briffa information.
This posting may be so crucial that it should have some sort of major link on main page. Other post may distract. Just a suggestion
Wow. You deserve an enormous pat on the back for your dedication and persistence in this matter. Thank you! Thank you! Thank you!
What kind-of
sciencepseudo-science do these so-called researchers claim to be doing when they won’t make their data available to be checked/double-checked/triple-checked?! I mean: if the results can’t be replicated, it ain’t science….Hey Look! A hockey stick tree! Only found in Yamal:
Hi,
Very nice work, and persistence, Steve. I understand the significance, and trust that you and Ross will be published quickly and broadly.i like this.
Treeline dynamics at Yamal:
Reforestation causes warming by decreasing albedo:
#432+#433:
more trees => more warming => more trees
Is that a positive feedback loop?
… a vegetation-related climate feedback NOT present in the GCMs?
Hi,
Yes, correct. In many cases, maximum latewood density (MXD) has a stronger summer temperature signal than ring width. I might try and do a post on this at some point.
Tree line paper of the Tornetrask area
Barnekow and Sandgren
Summer temperatures 1.5 -2 degrees C warmer in the past than today.
Steve McIntyre:
But why were these accusations false when you wrote on 28/9/09 in response to me:
and Schmidt responded on 30/9/09:
It was only later that you wrote
I really don’t know why you are dragging this up. Do you think you might owe Schmidt an apology for calling his accusation false?
Steve: Oh puh-leeze. The point here is that Briffa did not endorse Gavin Schmidt’s comments and distinguished between language used by others and the position set out here, for example in the head post, where I had clearly stated: “It is highly possible and even probable that the CRU selection is derived from a prior selection of old trees described in Hantemirov and Shiyatov 2002 as follows:…”.
The characterization in the head post proved to be correct. A prior selection of old trees had taken place. This assertion (which proved to be true) is what Schmidt objected to. In response to requests for quotations to back up his accusations, this is what he produced:
The statements attributed to me (1-3) do not support Gavin’s allegations in any way. First of all, all three statements are true. Secondly, none of them specifically attribute the selection to CRU (actually the opposite.) At 1.30 am my time (11.26 pm blog time), after observing “It’s too late tonight” in a passim comment responding to an absurd approach to sensitivity analysis by reader Tom P, I used language that was inconsistent with the position spelled out clearly on multiple other occasions, both before and after. After people spent a couple of days trawling through comments, this usage was brought to my attention and I attached a notice to the comment.
Tom, if you want to re-litigate Gavin’s comments please do so on another thread. This thread is about Briffa’s response and the fact that Briffa did not endorse Gavin’s comments. If you wish to argue that Briffa endorsed Gavin’s comments, you are welcome to do so on this thread, but let’s stick to Briffa;s comments.
Re: Tom P (#33),
Really Tom, you need to look at the whole chronology (of Steve’s remarks.) On 27/9/09 he said in http://www.climateaudit.org/?p=7168 :
This was before his remark to you in the 28th and of course before Schmdt’s remark on the 30th. Note that your quote from the fourth only shows that by the Steve had made such remarks. As I show above he made them on the 27th. Do you think you might owe Steve an apology for calling his defense false?
Re: Dave Dardinger (#35),
This has been pointed out to him countless times, yet he either can’t read or ignores it.
He is deliberately obtuse, no other conclusion can be drawn.
Re: Eric J D (#36),
Eric there are two possibilities and I favour the second.
Deliberately obtuse or just plain stupid.
In addition to quote brought up by Dave Dardinger in 439, here is another quote that both Gavin and Tom P are evading, made by Stteve on September 27 in this very thread at 7:27 p.m.
This is a very clear and concise summary of Steve’s position on the possibility that cherry picking had occurred. It notes the evidence is suggestive — but not conclusive — and specifically invites a rebuttal if one is available.
In view of these words, Gavin’s selective, partial, out-of-context quoting of Steve’s comments — selected to make it sound as if Steve had firmly concluded that cherry picking had occurred and that Briffa was the guilty party, when Steve’s words clearly indicate otherwise — is an outrageous smear and a new low even for RC. A scientist resorting to this sort of cheap smear is inexcusable.
I note also that at the time Steve made the comment above, Tom P did not disagree with it. Rather, he accepted the fact that the facts to that point were a prima facia argument that a biased selection had been made. Tom P simply tried to justify the selections, as indicated by his comments following Steve’s summary:
No indignation about an improper, unjustified, unfounded accusation from Tom P — just an attempt to justify the obvious inclusion of certain data and the exclusion of other data. It wasn’t until his attempt at justification failed that Tom P switched arguments and jumped on the “offended professional dignity” bandwagon launched by Gavin’s wholly unjustified smear.
Okay, Steve, go ahead and snip, but I had to say it.
Re: Michael Smith (#442),
Don’t snip this! This is exactly what is needed. Witnesses to the conversation. Independent accounts of events. I don’t have the patience to go back and reconstruct an argument just to illustrate why some bozo is wrong. I value these these summaries.
.
Tom P didn’t understand why what the dendros were doing was data snooping and why that’s bad. They don’t understand it themselves. Let this stand on the record, right alongside Esper’s quote.
Re: bender (#443), There are critical reasons why witnesses are asked to “Tell the truth, the whole truth and nothing but the truth.” The notion of the “whole truth” is the key portion here. Selective omissions are … how should I put it nicely … misleading!
To summarize, now with Briffa’s second response as context. This image is as disquieting as ever. You have this massive divergence between these two chronologies and Briffa saying “we did not consider those data”. Well, that’s the question, bud. Why did you not consider them? And next we have Briffa asserting that Schweingruber’s chronology is “anamolous” and that his (H&S) is “representative”. Enough with the assertions, bud. Let’s have an argument. On what basis do you assert this?
.
That is why the image was, and still is, disquieting. The criteria for choosing one over the other were, and still are, a mystery. Enough with the 27-page unresponsive responses. WHAT ARE THE CRITERIA?
.
Similarly: What are the criteria for choosing Yamal over Polar Urals. Rob Wilson tells us that it is high MWP variance. Yes, but, Rob, screening on variance inadvertently(?) screens on mean because mean and variance are correlated. Or hadn’t you thought of that when you worked out your screening criteria?
.
Cut the BS, you hosers. I’ll tell it straight up. There are no criteria. Bunch of Texas Sharsshooters is what you are. That’s why you can’t answer the question. You didn’t know there was a question to be asked. ADMIT IT!
Re: bender (#444), bender, why are mean and variance necessarily correlated? I can imagine measuring a constant signal with a variety of instruments, getting a constant mean but different variances that depend on the instrument used.
Re: Pat Frank (#447),
They aren’t “necessarily” correlated. They *are* correleated. Empirical fact. Theory? Who cares?
Re: bender (#448), bender, are you restricting your comment to tree ring time series? Or to time series in general? Or are you making a general statement that every set of means examined shows correlation with their variances? Or what?
I’m not trying to give you trouble. I’m just trying to understand what you mean (different “mean,” that one). 🙂
Re: Pat Frank (#449),
Pat, I’m talking extremely specifically – for the third time – about Rob Wilson’s last statement on this blog, where he says he screened out Polar Urals in favor of Yamal in his 2006 paper on the basis of “unnacceptably high variability during the MWP”, whereupon Kenneth Fritsh proceeded to prove that the mean tree ring index in those series is correlated with the variance during any given 50-year interval. Oops. Screen against a variable MWP and you inadvertently screen against a warm MWP. Oh those selection criteria, where DID they go?
Re: bender (#451), Thanks, bender. That clarifies it perfectly for me. 🙂
Re: bender (#444),
Bender…I want some of whatever your having. It sounds like fun.
Re: NW (#450),
NW, I’m just fed up with the drive-by dodge-’em nonsense. Tom P, Rattus, and so many others …
When you come to CA you better bloody well come prepared to make fact-based arguments.
Sorry Steven.I was so outraged when I saw your comment that I quoted that I didn’t realise you were playing Devil’s Advocate.
Steve: snip – you’re welcome to post, but blog policies discourage complaining.
There has been a lot of discussion about selecting trees that agree with the instrument record for the recent period where thermometer records exist. Most of this discussion seems to suggest that these particular selected trees are also used to project further back in time. I am under the impression that larch trees only live 200 years or so. If that is true these “calibrated” trees can only provide recent data in the time series.
If trees can be excluded on a basis of failure to agree with “calibrated” data (which seems troubling already), what is the selection criteria for earlier periods? If there exists other proxies providing “calibrated” data for all of the periods, what is the point of looking at tree rings at all? If the selection criteria is different for earlier periods, why can’t this same criteria be applied to the more recent data? There might be valid reasons (properties of fossilized trees that don’t exist for ones that aren’t fossilized perhaps), but it certainly deserves detailed explanation and some discussion of why the resulting data sets can be compared.
Hello Mr. Mcintyre
I was able to get CRU archive, starting from -202 to 1990 by following your R-code
I am just wondering whether i am able to obtain Yamal RCS chronologies from 1 to 1990, especially Schweingruber variation and little bit pregnant?
Due to the lack of knowledge in R, I couldn’t get them. If you can provide codes for that, i really appreciate it
Thanks
It’s scary that the leak confirmed this 2 months later. Steve, you sure you’re not a psychic?
It is the fact that Steve was right that probably triggered the FOIA dump, hence the psychic feel to this whole thing.
This posting must be preseved for posterity
Many/most links in the original code are broken, perhaps due to the site move?
The comments here end on Oct 1, 2009. I think it pertinent to append Briffa’s response from Oct 27, 2009, Briffa, which is a lnk off Briffa’s Oct 30, 2009. From what I can find at this moment, Steve has not directly responded to Briffa’s assertions.
The latter is at http://www.cru.uea.ac.uk/cru/people/briffa/yamal2000/
The former is at http://www.cru.uea.ac.uk/cru/people/briffa/yamal2009/
While Steve M clearly replaces Briffa’s 12 Yamal with 34 Schweingruber
Briffa claims the following:
34 is not 18. This is Misrepresentation #1 in the Oct 27 paper.
Misrepresentation #2: Briffa claims that the Steve M. samples are from “slightly south,” without mentioning that Steve’s samples are “well within the Yamal” region, while his Taimyr and Sidorova sites are 400km apart, and the Taimyr and Balschaya Kamenka are also over 400 km apart.
Misrepresentation #3: Briffa asserts that those trees were not used in his own Briffa 200 and Briffa 2008, as if that were a mark of honor, in and of itself, while it is, in fact, the very core of the Steve M. argument – that Briffa used a very small selection of 12 trees from 400 km away instead of using the nearby Khadyta River sample of 34 trees.
Steve M. argues that the more nearby trees should have been used (especially since they wer a larger population), and asks why Briffa didn’t (and instead chose the much smaller population that didn’t happen to “diverge” from the up-sloping curve the Team/IPCC desired)- and then Briffa argues that Steve M.’s choices are wrong because he (Steve) “for a sensitivity test,” selects them instead of the faraway ones Briffa chose. One must conclude that Briffa would have taken exception if ANY other trees other than the Sidirova or Balschaya Kamenka ones had been used. But the argument is, and always should be, that it doesn’t matter which trees you pick, that the trend should always be the same, and especially for larger populations.
Briffa then goes on to say,
Misrepresentation #4: Steve M. clearly stated that he did this as a sensitivity set:
For Briffa to mischaracterize this “sensitivity test” as realy being in “one version of the chronology” is mendacious. A sensitivity test that is clearly labeled as such cannot then be said to be “one version” without including the “sensitivity test” label, too.
Misrepresentation #5: Briffa stated (Oct 27):
Looking at Steve M.’s Figure 3, Briffa’s assertion is a complete misrepresentation, to the point of being a lie. From the point of divergence between Briffa 2000 curve (red) and Steve M.’s reconstruction that excludes the 12 trees in question (black), the “merged” set does not go up, but actually declines slightly. It is obvious why Briffa did not show Steve M.’s graphs on Oct 27, 2009, because they do not support his textual assertions at all, and anyone reading his rebuttal paper would have seen his – I can’t avoid the term any longer – lies.
Steve M.’s analysis stands unwavering.
Briffa cannot argue without misrepresenting – without lying. He asserts that Steve M. uses the data wrongly, and misinterprets the data, yet when he points out the particulars, Briffa comes out as a liar, telling people that Steve M. does things that he does not do and that Steve M. can’t even read a graph.
Briffa Misrepresentation #6:
The very point of Steve M.’s analysis is that Briffa’s “published analyses” – and “inference” drawn – is skewed by the 12 trees, the 12 trees brought in from afar when closer and larger populations were readily available in Schweingruber, thus the published analyses are wrong. And Steve even shows that when BOTH groups of trees are included, that the hockey stick disappears.
Nothing that Briffa argues in this rebuttal paper stands up to scrutiny.
In pleading innocence for the choice of the 12 trees and the exclusion of the larger 34-tree group from a closer, actual Yamal site, Briffa fobs the responsibility off:
All well and good, but when Steve M. later points out the very different results using the excluded trees, Briffa, instead of thanking Steve, chose instead to discount and misrepresent Steve’s input – and then chose to not include the other trees. And why did he choose to not do this? Could it be that if he did, everyone would see that the hockey stick disappeared. Or that “the message” of unprecedented warming would have been seen to disappear?
And we come to the grandaddy of all his misreprentations:
Misrepresentation #7:
“…included only the single KHAD site”??? Come on, Keith! We all know it included the balance of the trees in the study! It was not ONLY those 34 trees! Yes, it excluded the 12 – but it did not exclude ANY other trees in the study. AND it was clearly spelled out as a “sensitivity test” – which Briffa keeps on misrepresenting.
Steve’s points are two: That with the 12 instead of the 34, the curve is WRONG in its steep ascent, and that with the 12 AND the 34, the hockey stick does not exist. Briffa plays a shell game on Oct 27th and Oct 30th, 2009, distracting people from the real point: The Briffa paper is wrong, and demonstrably wrong.
I would also make the obvious point that Oct 27th 2009 was only about 3 weeks before Climategate, the AGW catastrophe to end all catastrophes.
I agree, and that set of individual tree graphs belongs in the original post as it is an important part of the argument. The agreement of these series with the 20th century temperature record appears to be accidental rather than causal.
31 Trackbacks
[…] I hardly know where to begin in terms of commentary on this difference. – Steve McIntyre, Climate Audit in Yamal: A “Divergence” Problem […]
[…] Scienmatifical report. […]
[…] post and this post are for reading slowly and rereading. They suggest that the hockey stick of global temperature […]
[…] http://www.climateaudit.org/?p=7168 […]
[…] details are on the last three days of Steve McIntyre’s site Climate Audit, and summed up beautifully on Watts […]
[…] [1] Yamal: A “Divergence” Problem, by Steve McIntyre, 27 September 2009 http://www.climateaudit.org/?p=7168 […]
[…] [1] Yamal: A “Divergence” Problem, by Steve McIntyre, 27 September 2009 http://www.climateaudit.org/?p=7168 […]
[…] de dos días, porque resumir bien la noticia estadística de Steve McIntyre en Climate Audit [–>] lleva su tiempo. Estaba seguro que alguien lo haría mejor y más rápido que yo, y en efecto. la […]
[…] Climate Audit […]
[…] Yamal: A “Divergence” Problem, by Steve McIntyre, 27 September 2009 http://www.climateaudit.org/?p=7168 […]
[…] Steve McIntyre was finally able to get enough data related to Briffa’s version and posted the history of the data issue and recent developments here. Yamal: A “Divergence” Problem by Steve McIntyre on September 27th, 2009 […]
[…] warming and commentators fighting restrictions on greenhouse gases have made much in recent days of a string of posts on Climateaudit.org, one of the most popular Web sites aiming to challenge the deep consensus among climatologists that […]
[…] a look at Climate Audit to get an idea of what happens when you pick such a small […]
[…] of the theory of man-made global warming. Global warming “skeptics” had unearthed evidence that scientists at the Hadley Climatic Research Unit at Britain’s University of East Anglia […]
[…] A more in-depth analysis of the consequences of doing so: https://climateaudit.org/2009/09/27/yamal-a-divergence-problem/ […]
[…] last week it had the bad timing to show up in a desperate UN compendium, released just days before Climate Audit published facts that promise to be the Hockey Stick’s (HS) long overdue epitaph. And those […]
[…] of the theory of man-made global warming. Global warming “skeptics” had unearthed evidence that scientists at the Hadley Climatic Research Unit at Britain’s University of East Anglia […]
[…] when there were many more cores available. Maybe because they didn't show the temperature spike? Yamal: A "Divergence" Problem Climate Audit […]
[…] science and physics should be discounted because of the study of self-proclaimed climate guru Steve McIntyre. In his erroneous study, he took a tree-ring temperature record that has been painstakingly […]
[…] Yamal Homogeneous? An Esper-Style Answer. Starting with the first of my recent posts on Yamal, I raised the issue of whether the CRU 12 actually came from a […]
[…] here is the corresponding graphic from the Yamal post in September 2009: Figure 2. Yamal chronologies from original post. Green – Yamal data from […]
[…] ever ready to defend McIntyre against real climate scientists exonerated McIntyre, pointing to a belated clarification from McIntyre: I don’t wish to unintentionally feed views that I don’t hold. It is not my belief that […]
[…] op climate audit, de site is traag van wege grote toeloopmirror bij Antony […]
[…] https://climateaudit.org/2009/09/27/yamal-a-divergence-problem/ […]
[…] McIntyre of Climate Audit seems to have broken the hockey stick for a second time. In Yamal: A “Divergence” Problem, he asks troubling questions about the Briffa tree ring chronologies used in many recent […]
[…] the modern portion of which is remarkably similar to the calculations in my posts of September 2009 here and May 2011 here, both of which were reviled by Real Climate at the […]
[…] the modern portion of which is remarkably similar to the calculations in my posts of September 2009 here and May 2012 here, both of which were reviled by Real Climate at the […]
[…] the modern portion of which is remarkably similar to the calculations in my posts of September 2009 here and May 2012 here, both of which were reviled by Real Climate at the […]
[…] modern portion of which is remarkably similar to the calculations in my posts of September 2009 here and May 2012 here, both of which were reviled by Real Climate at the […]
[…] Yamal version that Osborn attacked was shown as a sensitivity analysis in my original Sep 27, 2009 post on Yamal and not cited or “promoted” or “advocated” in any subsequent […]
[…] For professional statisticians Steve McIntyre and Ross McKitrick, it was claims of precision in paleoclimate reconstructions they knew were not possible with the data available. Across the world, scientifically literate […]