Starting with the first of my recent posts on Yamal, I raised the issue of whether the CRU 12 actually came from a homogeneous population to the subfossil population. This issue was related to the surprisingly small sample size of the supposedly “highly replicated” Yamal chronology, but is distinct. In his online response to Yamal posts, Briffa stated that they have “stressed” potential problems arising from “inhomogeneous” sources” in their “published work”.
Indeed, we have said so before and stressed in our published work that possible chronology biases can come about when the data used to build a regional chronology originate from inhomogeneous sources (i.e. sources that would indicate different growth levels under the same climate forcing).
Whether two populations are homogeneous is ultimately a statistical question. Consideration of this question as a statistical question has been blurred to date by the lack of understanding within the dendro community of the statistical issues involved in making a chronology (or within the statistical community of the difficult and interesting applied problems that dendros are trying to solve.) The disconnect is neatly illustrated by the references to Briffa and Melvin 2008, a highly technical article on standardization which nonetheless doesn’t include a single reference to an article or text by a non-dendro and which shows no awareness of how making a “chronology” ties into statistical literature on random effects (or how testing for population homogeneity relates to the statistical concept of “exchangeability”).
While Briffa’s articles occasionally contain caveats on population requirements for RCS standardization, I have been unable to locate any Briffa articles which he actually procedures for testing populations for homogeneity or the results of such tests, a lacuna that seems somewhat inconsistent with the idea that CRU has stressed this issue. In this instance, one wonders whether there is any practical difference between CRU stressing an issue and ignoring it.
While there doesn’t appear to be any relevant procedure in the Briffa corpus, Esper et al 2003 url does provide an on-point qualitative discussion of when two different populations can be combined in an RCS population (Tom P noted this in an earlier thread). It isn’t a formal test procedure, but it does provide a useful example (Jaemtland) where two populations were considered sufficiently distinct that they were not combined and does provide a basis for analysing Yamal homogeneity in a framework recognized by dendros.
In this post, I’ll provide a detailed description (and emulation) of the methodology described in Esper et al 2003, which, in another post, I will then attempt to apply to the “extended” Yamal data set recently presented by Briffa.
Esper’s example -see his Figure 8 – considered populations from two distinct sites: Jaemtland (swed023) and Trondelag (norw002), further distinguishing the Jaemtland site into two species PISY and PCAB. One attractive feature of this example as a precedent for analysing Yamal is that the Jaemtland population ends in the early 19th century, while the Trondelag population begins in the late18th/early 19th century, so that the techniques bear directly on the issue of population homogeneity between the Yamal subfossil and the “expanded” YAD-POR sites recently proposed by Briffa. To further accentuate the similarity, the Trondelag data set is a Schweingruber data set where each tree has two cores, while the Jaemtland data set is a non-Schweingruber data set with one core per tree.
Esper’s first step was to calculate age-dependence curves for each of the populations in question, plotting the resulting age dependence curves on the same quadrant. The following graphic shows the original Esper graphic, together with my emulation from available information (note Esper’s use of a biweight mean – which requires 4 samples). Esper observed that the (red) Trondelag (PISYsk) had a “significantly higher” growth rate and has a “rather different” slope than (black) Jaemtland. Esper’s comparison is qualitative, but nonetheless there is an obvious difference between the two age-dependence curves. One feels that this particular comparison could be worked up into a quantitative test (relatively easily if the comparison is between two negative exponential fits.)
 |
|
Figure 1. Esper et al 2003 Figure 8B showing age dependence curves from two nearby sites. Left – original; right – emulation from Esper data obtained from Sciencemag. The core counts for Trondelag and Jaemtland match the reported core counts in E2003, but there is no available metadata showing which cores in swed023 are PISY and which are PCAB, the two species having been mixed in the archive. The Trondelag (PISYsk) curves should match exactly; while they are close, they don’t quite. Esper uses a biweight mean instead of a simple mean or median (this is OK.)
Esper demonstrated the impact of two non-homogeneous populations on an RCS chronology in his Figure 8D shown below at left versus my emulation shown at right, together with the core counts for Jaemtland (light gray) and Trondelag (dark gray). These show first of all that I’ve accurately emulated the key features of Esper’s methodology for this particular analysis. (Further comments below graphic.)
|
|
Figure 2. Left: Esper 2003 Figure 8D; right – Jaemtland chronology as used in Esper et al 2002, together with core counts. Light grey – Jaemtland; dark grey – Trondelag.
Look first at the early 19th century transition from Jaemtland to Trondelag data and the impact on the RCS chronology. Application of a one size fits all RCS method to this inhomogeneous population results in a inhomogeneity in the RCS chronology at the hinge point linking the two populations. Esper characterized the inhomogeneity as resulting in an “artificially shifted” mean in the early 1800s.
Esper stated that the inhomogeneity is caused by the
“rapidly growing [Trondelag] samples forcing the RCS spline [SM: would be the same for negative exponential] too high for the older, slower growing samples. This feature, namely the significantly deviating growth rates and growth decreases with aging of the Trondelag samples, indicates the existence of a different population in the sense introduced earlier. It also demonstrates the biasing effects of different populations and the fundamental requirement of the RCS method: sample homogeneity”.
There’s nothing in this paragraph that ought to be objectionable to any CA reader – it’s the sort of thing that we’re regularly concerned with. And it’s precisely the sort of thing that I’m wondering about at Yamal.
There are some other interesting features to this graphic. In the 13th century, the sample replication goes down to one core from Tornetrask, inserted into the data set as a bridge. The breakpoint in the RCS time series is pretty obvious. At the start of the record, replication falls below benchmark standards and again there is an obvious breakpoint.
In this particular analysis, Esper is at least attempting to analyse population homogeneity. In contrast, despite its length, the analysis in Briffa and Melvin 2008 fails to deal in any relevant way with the problem of population inhomogeneity (and other Briffa articles are even less helpful.)
When you have inhomogeneous populations, you can’t just add everything together. This is well understood in social science, where analysts have to take care to separate the effects of different factors. Esper’s Jaemtland-Trondelag example provides a foothold for trying to quantify population inhomogeneity within frameworks familiar to and accepted by dendros.
In a forthcoming post, I’ll apply these methods to both the Briffa 2000 Yamal data set and the “moved on” data set with some interesting and perhaps surprising results.
Script: multiproxy/esper/jaemtland.txt
Climate Audit 
37 Comments
Region by region the trees are more homogenous but it still didn’t look very good. Taking the new Briffa data by region the HS was reduced considerably even though the new data had the same 12 series as the original Yamal.
The black line is the original Yamal- clipped off the top end of the graph. The red curve is all the data in the sensitivity study corrected by region.
When the whole series is RCS’d together, the curve looks very different. We’ve been wrestling a bit on this issue with delayed.oscillator at tAV. He didn’t like my point that the variance in the 20th century was mostly due to problems with homogeneity and probably not the fact that it got referred to as hockeystickization. It’s something I need to nail down better but from the early results like this figure and several hundred other plots, it didn’t seem unreasonable.
I’ll look forward to Steve’s next step.
“Consideration of this question as a statistical question has been blurred to date by the lack of understanding within the dendro community of the statistical issues involved in making a chronology (or within the statistical community of the difficult and interesting applied problems that dendros are trying to solve.)”
Change the parenthetical “or” to an “and”. . . then we can all start a round of “Kumbaya”.
Yes, I know. . . Steve, as avatar of the statisticians, clearly has been trying desperately hard to understand “the difficult and interesting applied problems” of the dendros, and mostly turned away as not a lodge member for his efforts.
Is this a pretty central issue for building any proxy from multiple trees covering different time periods?
Wouldn’t any test for homogeneity require a pretty large overlap in the years covered and a sufficient sample size of cores for both sets of data for the common period in order to be able to say that the two measures for the common years are statistically identical? Wouldn’t this type of homogeneity testing already need to be done in order to construct a proxy for a single site from multiple trees covering multiple periods, let alone different species from different locations? Aren’t the problems with stripbarks examples of what happens when homogeneity assumptions are made without testing?
I searched for a while on-line looking for homogeneity requirements in RCS, to try and understand the restrictions. In Briffa et al 1996 they write that RCS:
(Although this quote comes from Briffa 2001 in explaining why they did not use that technique in the paper.)
This is presumably to ensure the homogeneity that Steve is investigating.
“Large amounts” and “relatively small” seem to be hurdles that some recent RCS work falls at. No doubt RCS analysis techniques or definitions have improved (moved on?) since 2001.
I’m still struggling with the idea that there are magic trees that ignore the local growing conditions (light, temperature, nutrients, crowding out, rainfall etc.) and match themselves to somebody’s chosen global climate signal established from broken temperature reconstructions extrapolated and derived from something called the “instrument record” and manipulated the hell out of to show the required trend and then peer-reviewed to death (not). Bright trees. Clever trees. Dead trees. Signally trees. Noisy trees. But always Magic trees.
Re: Peter Dunford (#4),
In Melvin and Briffa’s more recent articles, they observe that “Age Banded Decomposition” is actually an alter ego for RCS (with very slight variation). Presumably the data met the criteria for neither method.
Re: Peter Dunford (#4),
I have seen no evidence of any improvement to (or even consideration of) population homogeneity methodology since Esper et al 2003. Melvin worries a lot about spline shapes, but IMO this is hugely secondary or tertiary to population homogeneity.
Scripts is at http://www.climateaudit.org/scripts/esper/jaemtland.txt
I’m quite busy at the moment but might be interested to see what happens if I apply my random effects with lots of dummies method to this data and see a) what the resulting chronology looks like and b) what the standard errors look like c) What the results of a test for population homogeneity give. Is there a nice text file version of the data sitting around somewhere?
(I wonder if your answers on the CRU 12 will be that same as mine using my non-dendro-approved-but-well-known-to-statisticians technique… I await your posting with baited breath.)
Re: JS (#8),
I’ll do up my random effects version as well. I’d like to compare notes.
The term “large” used by briffa begs for some kind of quantification. Some idea of the variances involved and the confidence interval size desired should drive the sampling requirements. Isn’t this just design of experiments 101? or am I missing something again?
Steve, I love this line:
This was the same idea I was thinking but you express it so well!
Re: Ron Cram (#11),
Please enjoy product responsibly.
Re: Ron Cram (#11), Steve “R”-funny
Trondelag doesn’t look to be simple negative exponential growth. Forestry scientists seem to have used a Weibull distribution for growth curve fitting:
Click to access 300_colbert.pdf
So might the Weibull shape parameter work as an index of exponential-ness?
Re: Chas (#14). With enough data you can fit it without any functional form restrictions at all. Once you’ve done that you can also test for the appropriateness of any given functional form restriction. I’ll bet that both Weibull and exponential are rejected for most data sets due to site-to-site idiosyncracy even though they might be reasonable approximations of a generic and highly stylised ‘representative tree’.
Re: Chas (#14),
I agree with earlier CA writers, that a continuous down-dip curve like a negative exponential eliminates the case of a tree that started to grow larger tree rings in later life. A more compound type of curve, with more inflexions seems needed – Weibull might be a candidate. The trouble is, from whence does one derive the detail that indicates which style of curve is appropriate?
Might I venture the answer? From simple experiments that should have been done before many complex papers were published.
I’m also suspicious on general principles when authors cite their own work to justify themselves. Such papers are echo chambers.
Well, Briffa and Melvin 2008 is a chapter from a textbook. I don’t find his modest citing of his own work that odd since he *has* done a large amount of the work developing the RCS method. And of course, Steve is a bit misleading here since the meat of the article is about the possible biases which can be introduced into a chronology.
Steve: I was quite precise. Do you have trouble reading? Briffa and Melvin did not use the term “inhomogeneity” or the word population in a relevant sense. They discussed a variety of biases in artesanal terms that did not connect to any statistical literature within known statistical canons or cite any text or reference by a statistician.
Re: Rattus Norvegicus (#17),
That ain’t the point. They don’t consult the mainstream statistical literature. That’s the point. And that’s a fact. A fact that did not escape notice of Wegman. Nobody learns in Team town. But hey, the science is self-correcting.
Where can I find the exact location of the Jaemtland proxy?
Re (JS#15) I was primarily suggesting it as a number that could be used when checking for homogeneity – though I suspect that the parameters from exponential fits could pick out most groupings of tree growth, in say, the Yamal collection .
On the other hand, if the trees from Trondelag (above) had grown rather more slowly, one might find it hard to pick them out from the Jaemtland trees, in a mixture, using only exponential parameters, and then one is left in the position of saying that it became cold every time one of these trees started to grow.
-I am guessing that the Weibull shape parameter would seperate them.
Re (Geoff Sherrington #19), After locating/creating the subgroups it perhaps isnt necessary to use any type of prescriptive curve form and so upticks would tend to sort themselves out. However the boundaries between groups might be very blurred — ie does one deal with X13 (in the Yamal ragbag) on its own?
Re: Chas (#21),
We’ve seen “Hugershoff” functions mentioned in connection with tree rings and in particular for the Schweingruber data. Hugershoff and Weibull are the same thing.
Okey, in Steve’s data I found it to be 63.05 north, well within my guess, but Jeamtland at that latitude stretches from about 12 to 15 in latitude, not much for a canadian but quit a lot for a european. So does anyone have an idea about the location. I suppose Esper himself didn’t do the sampling.
Re: Håkan B (#22),
63 30N; 15 30E according to NCDC information from SChweingruber. This is not always reliable.
Håkan B #22 latitude should of course be longitude.
Steve # 24
Quite right not always reliable, this one in the middle of a lake, but thanks anyway!
But while I have you at it who collected the data originally, if you know?
Re: Håkan B (#26),
Note that by providing precision only to one minute, the search box for the site covers several km.
Depending on whether the value was truncated or rounded, you can have fun searching 🙂
Here’s a link (and the second if you need it), showing the two search areas, assuming the data is correct.
(The fact that minutes are 30 for both makes me uneasy. It *could* be correct, but we’ve seen huge errors in the past, even off-by 100km.)
Re: MrPete (#27),
Why the #%%$## don’t people just use the UTM co-ords?
Depending on the individual satellite photo strip, I’ve easily been able to zoom Google Earth to exact Siberian drillhole sites off recorded UTM co-ords where you can count the boulders in the stream nearby (only summer photos, of course)
Re: ianl8888 (#29),
There is a 1996 Australian paper that goes into fair depth about why UTM was not used from the start. Countries sometimes chose their own datum points and with later GPS they all had to be tied in together.
Click to access tech_doc.pdf
It’s not the only useful paper but it should do the job. There can be no single, static projection that satisfies all needs for all time. The earth is plastic and the divergence of some tectonic plates is 100mm a year.
Other posts on CA have shown how, particularly with satellites, small projection/geodesy errors can have significant effects in climate science.
In our exploration work, mineral leases that had been pegged along magnetic N-S fencelines in the 1870s were now quite away from the fenceline. “Only the stars never change”.
It’s probably another discipline like statistics where climate scientists should have their work vetted by experts pre-publication. I do not know how many climate scientists would peg out grids that fail to account for curvature, but unless they had been exposed to the topic it would be easy to go astray.
Re: Geoff Sherrington (#30),
Thanks Geoff
I’ve had an enormous amount of quite frustrating experience trying to collate survey for mapping across about half the world. I agree that compensating for the earth’s curvature is a constant issue. However, UTM gridding has proved the most useful, albeit that the surveyors constantly change the origin
There is no absolute answer, of course, just more or less useful ones. UTM has worked for me much better than other choices but it ain’t perfect.
Re: ianl8888 (#32),
I meant to add that it obviously works least best nearer the poles. I’ve tried it in both far northern Siberia and western Antartica – the disconnect between adjacent UTM zones due to curvature is just too much
However, the frustration of people trying to locate the dendro sample sites on Google is clear
It looks like the region has been heavily logged and replanted at different times. Anyone disagree? I wonder when? It’s not uncommon to aerial fertilize plantation trees and sometimes there’s drift outside the target area and there’s stream runoff. Conclusion obvious.
Also, some of the reforest areas seem quite old. It might be complicated if plantation trees were sampled for dendro, especially if their sources (seed/cutting/sub-species?) were from elsewhere.
Re: Geoff Sherrington (#28),
I don’t think there was much logging before the 19th century, as there was no way to get the timber down to the coast. Why it was so you can read here:
Döda fallet
Re: Håkan B (#31),
Thank you for this info. In this distant part of the world I guess not many know of the lake burst and we are the wiser for your account. Historical accounts like yours help keep the blog interesting and often contain lessons – e.g. be careful of alternative solutions to problems posed by Nature unless you are good at it.
Re: ianl8888 (#33),
Did you see a spreading chasm in NE Africa nicknamed “Al Gorge” on WUWT a few days ago? Shows the problems in coordinate mapping a changing feature on the ground as compared with making a global grid reference system.
We once flew an aircaft briefly over the border from Iran to Afghanistan because of a mapping error. It excited the launch of a MiG fighter.
Re: Geoff Sherrington (#34),
I just noted a funny twist on units, measures and translations in the wikipedia article I linked to.
It states the lake was 4 km (2.5 mi) long, while the swedish version states 2.5 mil, but a swedish mil, as used in spoken language, is 10 km. So the lake was about 25 km long, although probably not in a straight line.
Here’s an article about scandinavian miles, but note that it was even worse than described in it:
Scandinavian mile
Re: Geoff Sherrington (#34),
Can’t match that, Geoff – too much 🙂
The silliest I’ve seen was an entire drilling program on someone else’s lease due to a surveyor disliking working in a hot climate (preferred the air conditioned vehicle). The recipient of this unexpected bounty of information was quite grateful, really
After reading Steve’s exposition at the start of this thread, I went out on Google Main this afternoon with the intention of learning more about the general topic of population inhomogeneity, as it affects statistical analysis, using the following search criteria:
Up pops this very thread at the top of Google’s output list.
.
So either a lot of people are reading this thread on the Internet, or perhaps this thread is considered the best real-world example of population inhomogeneity now available in Cyberspace.