William Connolley is still, shall we say, manfully pretending not to understand how sediments affected by bridge-building, ditches and agriculturally activity cannot be excellent temperature “proxies” if they correlate with NH temperature.
Amazingly, some of his readers, like PNAS editors and referees, take this sort of stuff seriously.
Just for fun, I’ve constructed an example that, IMO, contains the relevant features of the Tiljander example, combining a well known series with “rank Gavin noise”, defined here as two times log(1000) minus the log of the rank of Gavin among US names (somewhat modifying realclimate’s Gavin index, originally proposed by Lucia.)
As you see, it has a familiar hockey stick shape. It has an excellent correlation (.81; r2: 0.65) with HadCRU global temperature during the “calibration period” of 1954-2008 when Gavin ranks are available here .
Figure 1. “Proxy” plus Rank Gavin Noise
Readers are invited to identify the mystery proxy – which shouldn’t be too hard for CA readers.
While the example is constructed to be amusing, there is a fundamental point here – in general, Team methodologies assert without ever providing proof that “proxies” are a combination of “true temperature” plus white noise or low-order red noise.
The metaphor – and it is a metaphor – of “signal” and “noise” for sediment or tree ring series is one that troubles me and many other statistically oriented CA readers, however climate scientists to date have been totally unmoved by such concerns, seemingly having trouble understanding such elementary things as Mann’s misuse of the Tiljander series.
Obviously there’s a communications gap; maybe adding the concept of “rank Gavin noise” to noise repertoires of climate scientists will help bridge this gap.
Climate Audit 
54 Comments
Re: Terry (#1),
Well, to be strict about it, the notion of there being both signal and noise in temperature records, as well as proxies in which temperature has some impact, isn’t all that questionable. Putting it in these Hamming/Shannon terms, however, points to a basic question: can we distinguish the signal in the noise?
Examples like this one, and the previous examples of simply applying Mannian techniques to red noise alone, make it seem very unlikely that we can.
Re: Charlie (Colorado) (#2),
Can you identify the proxy that is combined with rank Gavin noise?
If the “noise” is sufficiently “tame” and the proxies are sufficiently consistent in some sense, then you can recover a signal using simple methods.
The problems arise if “proxies” contain rank Gavin-type noise – Mann and associates deny that they do. But it’s not enough to just assert it.
Equally a toy example like this doesn’t prove the opposite. So don’t get carried away by the example.,
MBH PC1?
Number (percentage) of males born in USA?
Looks less like a hockey stick and more like the pipe of some hookhah-smoking caterpillar. Which would be a very stupid and incorrect thing to do.
Upside down CET
Re: Arnost (#8),
Yep.
Upside down CET plus rank Gavin noise has a very strong positive correlation to HadCRU global temperature. Obviously this sort of problem is not simply theoretical as we have pretty much precisely this situation with the Tiljander sediments: an upside down proxy plus rank Gavin noise.
# rank Gavin noise downloaded from http://www.ssa.gov/OACT/babynames/index.html
gavin=read.table(“http://data.climateaudit.org/data/statistics/gavin.dat”,skip=1)
gavin=gavin[nrow(gavin):1,]; gavin=ts(gavin[,2],start=gavin[1,1]);gavin
gavin[is.na(gavin)]=1000
gavin= 2*( log(1000) – log(gavin) )
# CET annual
url=”http://hadobs.metoffice.com/hadcet/cetml1659on.dat”
test= read.table(url,skip=7)
temp=(test== -99.99);test[temp]=NA
annual=ts(test[,14],start=test[1,1])
annual=window(annual,end=2008)
Anom=ts( scale(annual,scale=FALSE),start=tsp(annual)[1])
# HadCRU GLB
hemi=”gl”
url< -paste("http://www.cru.uea.ac.uk/cru/data/temperature/hadcrut3v",hemi,".txt",sep="") #1850 2008
h<-scan(url)
start0<-h[1]
N<-length(h)
h<-array(h,dim=c(27,N/27))
h<-t(h)
hadcru3v.ann=1905 & time(x)1900,])
Data=data.frame(year,Data)
temp=Data$year>=1954
fm=lm(x~hadcru,data=Data[temp,]);summary(fm) # #0.6513,
cor(Data[temp,c(“x”,”hadcru”)]) # 0.8070355
Figure 2. Korttajarvi Thickness Versions at
http://www.climateaudit.org/?p=7556
For bender and caterpillars,
Ch 5, Alice in Wonderland.
And for bender, from the Looking Glass,
Got a suspicion that this thread can not end well…
Here’s another hockey stick. They seem to pop up everywhere. This one is definitely man-made
Details here
RGN for short. makes it appear more scientific and accepted…
When I was doing my Masters thesis, there was a lot of pressure from my supervisors to find evidence of sea level rises and falls (this was for a period c. 2 Myr ago). The problem so far as I was concerned was that I could easily find such evidence but the area was tectonically unstable (1500 Meter rise in 2 Myr)and a river estuary (now 60km inland). Any evidence was simply overwhelmed by too much noise – tidal ranges of 3 to 5 Meters were clear within the sediments and no possibility of identifying a signal within the data.
Re: KimW (#15),
To add to the predicted badness, what were supervisors like then? Tough but with good memories?
Steve,
Connolley admitted he was wrong in his comment 25. He gets it. He just doesn’t want to admit you were right all along. Says your point should have been that the varve data should have been excluded, not that it was added in upside down. Laughable position if you ask me. Your whole point was that the data was no good during the calibration period.
Re: Brian Macker (#16),
In his discussion of proxy data, Mann says:
At least they say your name now.
However, I still think the most interesting story is the alleged distortion of the GISTemp data. As quoted by E. M. Smith:
“IMHO, it is another Hockey Stick, but on steroids. I also suspect that most folks just didn’t know until recently. Folks trusted NOAA to provide clean data. Instead they cooked the record (knowingly or not). How long does it take to catch a Madoff or a Ponzi? It’s all about the appearance of trust.”
More alarming news here: http://chiefio.wordpress.com/2009/10/29/ghcn-pacific-basin-lies-statistics-and-australia/
I wish Steve or someone could verify this is all true or not. If true it surely would be the mother of all hockey sticks given so many official organizations and governments around the world rely on this data to demonstrate that global warming is still occurring.
Re: PeterA (#19),
I have just posted this with Chefio:
The reason why climatologist never bother to actually prove that their data consists of a signal and noise (white, red, brown, whatever) is that they do not (cannot?) investigate the physics of temperature proxies. Has *anyone* looked at the physics of the problem?
Re: C. Baxter (#20), in my view your remark touches upon the main problem around modern climate science. The problem with peer-review is not in its closeness and lack of transparency per se, but that this system cannot guarantee the due level of physical competence to persist among the peers in climate science.
In order to remain competent, one must continuously practice — a relevant educational background or even Phd will not do the job. If one does not practice fundamental physical research, the qualifications are lost. In the meantime, the emphasis in modern climate science is exclusively on data gathering, computer modelling and statistical data analyses at the sacrifice of doing fundamental physical science.
This is an interesting self-critical view from inside the broader climate community. Dr. Koutsoyiannis writes:
“It has also been a common practice in universities to conduct graduate or postgraduate theses that are applications of a specific software product with data from, say, catchment X. Change X with Y and you have another thesis. Of course, the second will have some differences from the first (e.g. different infilling of missing data). Even worse, this extends to research published in scholarly journals. Savenije neatly states “… There are many papers that deal with the application of an existing hydrological model, or that describe automated calibration, or that apply standard statistical methods, without much creativity, empiricism or innovation.” My experience is absolutely the same, and I wonder if this indicates any progress in science. It may rather be a regression.”
Traditional physicists are still looking away from the climate problem, as the whole history of modern physics has led them to face very different tasks and it is not easy for entire schools to change this historical direction of research priorities.
Re: Anastassia Makarieva (#21), You are correct in stating
Of course… once “the science is settled” there is no further need for pure research. That’s been done already. The only thing left is to clean up the data, software and algorithms a bit for posterity’s sake. Time would be much better spent advocating proper solutions and convincing the denialists just how wrong they are.
Re: C. Baxter (#19),
The reason why climatologist never bother to actually prove that their data is because they knew they were right.
To be honest, although this is mildly amusing, I don’t think this adds to the weight to your ‘use of the corrupted Tiljander data’ argument!
– you should probably just stick to your guns about Mann’s error, and see if you can get an acknowledgement thereof.
Also, I looked back & saw your bit about the Baker data being inverted – that is also interesting, and worth persuing.
Also, Mann says that removing the Tiljander data doesn’t significantly effect the outcome
– is this true?
– you’ve said that Mann needs to add back in the Bristle cone data in order to get the HS back
– can you run over this again – or point me to the relevant post on here?
It seems to me that the Mann-techinique will take in any old data, be it Corrupted Sediment data, inverted baker data, bristlecones, tree-rings, random data, and generate a hockey-stick, no matter what.
The IPCC wanted to ‘get rid of the MWP’, and son found someone who could do that with any old data as input, and hey-presto, a beautiful marriage was formed!
Thanks!
Re: Phil M (#23),
This is like the 10th time this question has been asked. How would you know the answer until you re-ran the leave-one out test in Figure S7 with both Tiljander AND AND AND tree rings removed? If Mann has done that, then are the results stored in a CENSORED directory somewhere in an undisclosed location?
Re: Phil M (#23),
This is like the 10th time this question has been asked. How would you know the answer until you re-ran the Figure S7 without Tiljander AND AND AND without tree rings? If Mann has already done that test and thus knows the answer, then I suppose we should be scanning the entire internet for a CENSORED directory?
Geoff Sherrington:
November 2nd, 2009 at 5:36 am
Re: PeterA (#19),
I have just posted this with Chefio:
Interesting that I run into it here, first, as I take a break from making my next posting… 😉
In the period after 1990 Australia agreed to move to a Reference Climate Station system of about 107 stations from a menu of nearly 1700 stations that had some type of records, many of not much value.
Thanks to the pointer to what was the genesis of all the “1990 or so” thermometer deletions all over the planet in the GHCN data set. “RCS”. Yet another thing to go digging into…
Please go ahead and use GISS data at your peril, but be kind to your workload and don’t invest a lot of time in it. Why not use the BOM data instead? It’s closer to the source.
Well, because I’m not using GISS data; GISTemp is using GHCN data. And it is the dropout of thermometers and it’s impact on GISTemp that I’m investigating… As near as I can tell, this move to RCS is not vetted as to impact on GIStemp. IMHO, it will cause a bogus warming of the output of GIStemp (vis the “115 year record heat” in California because we now have 4 remaining thermometers, 3 on the beach near Los Angeles, and one in San Francisco. Nothing measuring the snow in the mountains…
There are portents of evil here. When you report that “we saw that in 1992-93 there were 401 thermometers deleted”, I recalled that Error 401 in Windows is
“If you have just logged on and received the 401 Unauthorized error, it means that the credentials you entered were invalid for some reason.”
Lucky. They might have deleted 404 thermometers.
Nice chuckle! Lucky for me I’m running all this on LInux … I’d completely forgotten the last time I saw a Windows error 40x and I’m looking forward to forgetting it again!
Colour and linear trends added for amusement.
Re: Tim Channon (#27),
Tim
Can you link to the sources of that graph?
Re: Tim Channon (#27),
When I look at this, the first thing that jumps in my head is “what in the world happened in 1990?”
The red may distort the information a bit… the upturn may have started back in the early 1980s, albeit a lot slower. But the 1990 upturn is NOT what we’ve seen from CO2 charts. As asked above, where did this data come from?
Re: Dean P (#42),
Dean P, that’s an interesting observation. Looking again at the graph, I actually can see a distinct negative slope in the data between about 1875 and about 1960 (ie, cooling). So, why the sudden change trend reversal around 1960?
Re: Tim Channon (#27),
Can you extend it to more recent dates? Also, can we all remember (I’m an offender too) to label both axes of graphs accurately and concisely please? Thanks, Geoff.
Re: Tim Channon (#27), Tim
Can you provide more information about this graph? What historic published record of CO2 are you referring to? What are the vertical axis labels? Does the historic record continue up to the present?
Steve – my thinking last night was that you would need to do a follow up but unfortunately like Phil M I dont think this one gets the job done. Your Saturday night live post contains much of the requirement but is surrounded by a lot of history and snark. I dont have any real issue with the snark given the targets but it will divert attention. You may well get some new readers who dont wish to read several lengthy blog posts where the explanation is buried.
It would also help your supporters if there is a clear statement they could refer to with no snark so there is no reason for any objection. In addition William Connolley appears to be playing the “its a nit pick” card as his plan B which should also be rebutted.
In my opinion you need something along the lines of the following
1. A concise explanation in a new headline post of the mechanism by which the Tiljander proxy is inverted, Abandon your normal writing style so no restatement of information from prior posts or any snark. Dont allow comments and just refer people to the other threads
2. The post must clearly show how the current non climate part diverges and would effectively represent a modern cool period if it truly was a climate proxy. One diagram using the figure from Tiljander and a couple of short statements should suffice
3. Show that the modern non climate portion is flipped and falsely correlated to the temperature record.
4. Show that the true historical part of the proxy is pinned to the modern part and flipped upside down as a result of this pinning. Second diagram with relevent features annotated.
5. Show that use of the exaggerated modern portion results in a reduction in the amplitude of the valid part of the proxy
The errors with this proxy matter because
1. It represents a serious quality issue in proxy selection.
2. The modern blade is enhanced by an exaggerated spurious correlation.
3. The valid portion of the proxy is reduced in magnitude and is an example of how the techniques used are more likely to reduce past variability whilst enhancing the modern period.
4. The inverting of the proxy will result in it offsetting the other proxies further dampening the historical variation.
5. Any other reasons you wish to add
Re:Connolley…Here is something that I just noticed…Wolves record atmospheric carbon isotope trend better than tree rings I guess it would be very easy to extent this to Stoats….
Stable isotopes, ecological integration and environmental change: wolves record atmospheric carbon isotope trend better than tree rings
Joseph K Bump1*, Kena Fox-Dobbs2, Jeffrey L Bada3, Paul L Koch2, Rolf O Peterson1 and John A Vucetich1
(jkbump@mtu.edu)
Abstract
Large-scale patterns of isotope ratios are detectable in the tissues of organisms, but the variability in these patterns often obscures detection of environmental trends. We show that plants and animals at lower trophic levels are relatively poor indicators of the temporal trend in atmospheric carbon isotope ratios (δ13C) when compared with animals at higher trophic levels. First, we tested how differences in atmospheric δ13C values were transferred across three trophic levels. Second, we compared contemporary δ13C trends (1961–2004) in atmospheric CO2 to δ13C patterns in a tree species (jack pine, Pinus banksiana), large herbivore (moose, Alces alces) and large carnivore (grey wolf, Canis lupus) from North America. Third, we compared palaeontological (approx. 30 000 to 12 000 14C years before present) atmospheric CO2 trends to δ13C patterns in a tree species (Pinus flexilis, Juniperus sp.), a megaherbivore (bison, Bison antiquus) and a large carnivore (dire wolf, Canis dirus) from the La Brea tar pits (southern California, USA) and Great Basin (western USA). Contrary to previous expectations, we found that the environmental isotope pattern is better represented with increasing trophic level. Our results indicate that museum specimens of large carnivores would best reflect large-scale spatial and temporal patterns of carbon isotopes in the palaeontological record because top predators can act as ecological integrators of environmental change.
Re: Clivere #26
Great idea, maybe you could do it.
Wolfmometers!!!! Or would it be Carnivometers?
Brilliant. Stunning, really.
Why do they always assume that proxies that correlate to temperature are really measuring temperature?
Does varve thickness get incorporated into a climate reconstruction by a linear fit without any correction? It seems to me from skimming Tiljander’s paper that older varves may be slowly compressed over time. Since calibrations are always done with recent sediments and temperatures, how does anyone know these proxies are accurate over millennia?
In the case of tree rings, RCS corrects for the fact that trees rings are wider at a given temperature when trees are younger. Does anyone correct for the possibility that recent varves may be thicker? If water is slowly squeezed out of varve layers, then the density of the layers might increase with time before present. Comparing the density of old and new layers with the same composition and particle size might uncover a bias
Funny post. May I suggest two graphs for a more educational analogy?
Graph 1 with normal CET (where larger numbers means hotter temperatures) combined with rank Gavin noise where smaller numbers is hotter (i.e. Gavin at rank 800 is not so hot , but at rank 1 it’s the hottest name!) and so the blade points down.
Graph 2, after passing through multi-variate analysis – using modern non-bulb temperature measurements for calibration – the blade gets flipped up and CET is inverted.
It shows that old-time bulb-thermometers were inversely related to temperature and thus a terrible temperature proxy back when they were initially used. Thanks to modern Mannian statistics we can correct the old temperature records.
RE #8, #9,
Umm, what is CET?
Re: Hu McCulloch (#39),
CET = Central England Temperature
http://badc.nerc.ac.uk/data/cet/
MattN, that’s a good question. I wonder if there has ever been a well-correlated proxy that was excluded from one of these multi-proxy reconstructions because of non-climate signal contamination.
Steve M is up to his old tricks again. Well over 14 milliseconds ago, I requested he indicate where his data is archived. I have yet to receive a response. How can one verify his study? Is this peer reviewed? When does the comment period end? Will we have access to all the reviewer comments, or will Team CA claim conflicts with confidential agreements worldwide with people who have last names?
Re: Bob McDonald (#43),
You have exceeded your 12 word limit. Criticism rejected.
Ask me, Ask me! (Waving Arm)
There is this very special tree in Russia….
There seems to me another Rain in Seine issue. I question the use of US Rank Gavin data with respect to the CET proxy, should they not be local temperatures? More tele-connection I presume.
Phil M #22
A milk shake maker produced a wonderful pink milkshake which was delicious. Moreover, because it was pink it had health giving properties. It contained milk, cream, yoghurt, sugar and raspberries. He concluded that all 5 ingredients were needed to give the drink its pink coloration. Then along came an amateur scientist who disputed the claim. He performed a simple series of experiments in which he removed one ingredient at a time from the mix. He discovered that it was, in fact, only the raspberries that were responsible for the pink colour; the rest of the ingredients had no influence on the colour.
The Shaker (for that was his title) was upset at this assertion, an assertion from a man who wasn’t even a fellow Shaker, but he did not waste too much time as he’d moved on. For he was both a mover and a shaker. He now had a drink that was even pinker and, therefore, even more beneficial to health. This drink contained milk, cream, yoghurt, sugar and strawberries. Again, he asserted that all 5 ingredients were needed to get the pink colour. The amateur scientist performed the same series of experiments as before and this time he discovered that only the strawberries give the shake its pink colour.
The shaker was angry that this “amateur” was making these wild assertions but he did not waste too much time as he’d moved on again. His latest drink was the pinkest yet. It contained milk, cream, yoghurt, sugar, raspberries and strawberries. In order to prove his assertion that all the ingredients were needed, he performed his own series of experiments, based on those performed by the amateur scientist, in which he removed one ingredient at a time. In every case he still ended up with a pink milkshake. He was thus able to rubbish the amateur scientist’s claim that some of the ingredients weren’t necessary for the pink colour.
I think his logic is flawed. What do you think? Could you falsify his hypothesis if you were allowed to remove, say, two ingredients from the mix or would this be unreasonable for scientific reasons? Off the top of your head, which 2 would you start with?
Andymc
andymc,
Not knowing any better, I would start with the first ingredient (milk) and test it with each of the others. Then do the same with cream, then yoghurt, then sugar.
Having tested over 90 percent of the possible permutations and found no problems, I would check the IPCC 4AR and conclude that at over 90 percent my confidence level was very high.
I could then stop testing, publish and have a media blitz.
Trust me, my strategy is robust!
I would love to see a surface station survey of that CO2 sensor…
In forecast modeling, Connelly is correct in principle that sediments (or other potential proxies) affected by certain “errors” but are correlated with the variable of interest (temperature) can be useful proxies. However, the utility of such a proxy is seriously undermined or even negated if the nature of the “errors” changes over time. If bridge building, ditches, etc. affected the entire history of sediment in the same way, the proxy can still be used.
If (as is very likely) the nature of the errors change over time there can be a structural break in the correlation of sediment data and temperatures. (I.e., it is likely that the modern era has affected sediments in a substantially different way than the agricultural/wild period prior to the modern era.) The structural break can lead to widely erroneous and biased results.
There are, of course, ways to statistically test for a structural break in forecast modeling. I have little doubt that a structural break is present in the relationship between sediments and temperature around the early/mid 20th century.
Re #51(Andymc) – Clearly your milk shaker had never heard of the art of statistical experiment design! The hypothesis he was attempting to verify or refute relied on the assumption that the effect of each ingredient on the colour or performance of the “shake” was totally independent of the presence or absence of any other ingredient. In a mixture experiment things do get a bit complicated of course. For example, if you leave one ingredient out what about the relative effects of all the others. Well, it gets a bit too tricky to write about seriously, but this bit of fun is worth a moment’s contemplation I think.
In the original climate context, leaving out one of the presumed contributors to the (additive) parameter of interest (and I suppose we are still talking about local or areal or global temperatures) There should not be any “interaction effects” I guess, so the problem should not arise.
However, I’d be grateful for some instruction about the fundamental reason for being so fixated on tree ring data. Several references have been made to the hoped-for correlation between tree ring information of whatever sort, (or varve thickness data) and the temperature/climate prevailing at particular times of the year in the same region. What I gather is that the tree ring stuff is being assessed by its (linear) correlation with local summer temperatures JJA, perhaps MJJAS, or whatever the growing season is. However, in my book annual temperatures – the things that determine our way of life – are not just those existing in the growing months. There are several other months in which temperatures remain of interest to those living in them. What will happened to our assessment of climate if we choose to ignore the majority of the available instrumental data?
In my studies of instrumental readings for polar regions I’ve noticed that many stations do not record data from the depth of winter. Specifically, temperatures below -20C seem to have been widely replaced by missing values. I can readily sympathise with the people charged with measuring them, but surely it would be very useful to know whether the cold extremes have been getting more or less extreme. These big negative numbers ought to play their part in our estimation of global temperatures.
Am I missing something here?
Robin
One Trackback
[…] – Nov 02, 2009Wind Lobby Huffs and Puffs, But Can’t Blow the Facts Away Climate Audit – Rank Gavin NoiseWatts Up With That? – Equatorial Volcano shows signs of imminent […]