I’m up to 38 emails back and forth with Science now in trying to get data. A little more drifted in April, about which I’ll report on in a couple of posts. My most recent progress report was here
One of the questions pertained to a discrepancy between the correlation between gridcell temperatures and foxtail chronologies reported by Osborn and Briffa and the actual correlation. Osborn supplied the information that they used – they used data only going back to 1888, while the HadCRU2 gridcell goes back to 1870 and there was a significant difference between the correlations when 1870 was used instead of 1888. Since there appeared to be an ad hoc use of data, I requested an explanation of the discrepancy and corresponding information for other gridcells as follows:
6. I acknowledge receipt of a temperature data set from 1888-1990. The HadCRU2 data set contains temperature data for the gridcell 37.5N, 117.5W commencing in 1870. However, the gridcell information provided by Osborn commenced only in 1888 and the differences are material to the final result (0.045 versus 0.18 reported). What is the reason for commencing this comparison in 1888 rather than the available 1870? Why is there no notice of this in the SI? Since there is a material difference in this example, could you please provide the gridcell temperature sets in a comparable format for the other 13 Osborn and Briffa series.
Osborn and Briffa sent an explanation to Science, which was forwarded to me, together with the corresponding information. The explanation shed some interesting light on CRU temperature data, which many readers of this blog are interested in. (As you all know, CRU’s station data is confidential.)
Unfortunately, Science advised me that the Osborn and Briffa explanation was itself confidential and not for public posting without permission of Osborn and Briffa, which to date has not been forthcoming. Osborn and Briffa are both at CRU in England and intersted readers may wish to contact them personally for a confidential explanation.
Climate Audit 
34 Comments
Since when is it that an explanation for a controversial or non-obvious step in a paleoclimate study became confidential? Is it because of the threat of terrorism?
Excuse me, Steve, but this is completely f***ing ridiculous.
So it’s really the CIA Climate Intelligence Agency.
Interesting that temprature is the prime thing that comes with a Ultra secret Mega Clearance. I would have thought rainfall was much more important to keep secret.
Person 1 “Hey it’s getting a bit chilly out. Wonder what the temp is?”
Person 2 “I could tell you, but then I’d have to kill you.”
Oh wow. Interesting. While cherry picking of series can produce a hockey stick shape, its clear that truncation of temps could lead to the same effect. This is because anything that improves the capacity of random series to correlate with temps creates hockeysticks. You chop off the first part pre-1888, which is a part of a hook up, you eliminate medium frequency variation and get easy correlation with the low frequency century trend. Cool. I can feel a simulation coming on.
David, take a look at Jones and Mann 2004. They report decadal correlations, using as few as 3 measurements in one case as I recall.
Hegerl et al 2006 cited a statistics article from 1878 as a reference for total least squares (aka errors-in-variables). Perhaps one could introduce Jones and Mann to the t-statistic, recently introduced (around 1920) to test significance allowing for the number of measurements.
Steve:
Do you have Osborn’s and Briffa’s e-mail addresses. Perhaps we can all send a request for a confidential explaination.
#6 – Russ its all on the net here.
fascinating to look at the cru link. I count 5 university academics, 8 senior fellows, 17 workers and 14 PhD students. Assuming all that is current, the whole shebang costs > 850 k per annum (30x25kpa + 14*9kpa), and probably brings in >200k overheads (profit) per annum to the university.
who says climate research isn’t a business ?
yours
per
In this case, the explanation doesn’t appear to be overt truncation. However, it’s just as interesting; maybe more interesting. Too bad I can’t tell you.
Speaking of truncation – it’s interesting to see the very low late 1600s in the ECHO-G model which I’ve been working with in the past few days. Remember the truncation of data reported in MM03. They truncated the late 17th century from the Central England temperature series. Mann justified this in hte Corrigendum on the basis that Bradley and Jones 1993 had done this and he was using their data. But that rolls the truncation question back – what the hell business did Bradley and Jones 1993 have doing truncating the late 17th century temperature data in an article purporting to show that the Little Ice Age was over-rated. And truncating without any notice to the reader. There’s some back history. I’ll try to remember to do a note on Hughes and Diaz.
Gawd, I don’t believe it. The temp. data is confidential, the answers to your questions are confidential. These guys really should be embarrased. This is NOT science!
Why are the CRU station data confidential?
Anticipating: They charge a fee for access to them.
I’ll bet they are confidential for the same reason that Jones and the boys will not release the basis for the SAT calculations.
Re #12
Why was that again? Is the reason a secret also? Ah it is government work, wink wink, nod nod. Speak no more.
Re #13
As Sir Humphrey would say:
“If they don’t know what you’re doing, then they don’t know what you’re doing wrong”
Steve said:
Is it bigger than a bread-box?
It might be #14. I’ve not followed the temperature data set issues very closely, but I don’t think that any of the people who are interested in temperature data are going to be very happy with the answer. Too bad I can’t tell you the answer. Richard Alley at the NAS Panel made a really good distinction between the objectives of PhD’s and post-docs in collecting novel data and the needs of a proxy "monitoring" program in going back and updating sites. Maybe responsibility for temperature collection should be taken away from CRU. Collecting and collating temperature data is like making a Consumer Price Index. Shouldn’t this be done by a professional statistical service and not by some guys acting like primadonnas? [amended slightly in light of Pat’s comment below]
#16 “… primadonna scientists?” Goddamm it, Steve, you’re starting to sound like John A. How about some limiting adjectives, like, ‘certain scientists who think themselves primadonnas‘? Why, by George, I’ll bet as many as one or two scientists posting here don’t consider themselves primadonnas.
More seriously, I’ve known and even published with greats and even a Nobel winner, and all of them had a firm grasp on personal humility.
#13 — Jones is on record for having refused to release his data because he didn’t want Warwick Hughes to look at them with skeptical criticism.
Steve,
Are you allowed to explain why their explanation is confidential? If I understand correctly, you asked for an explanation about a discrepancy you found in an article published in Science. Maybe you stumbled upon a discovery that they are about to publish? I suspect that this is not the case. In the context of your writing, it looked like you may have found something that makes them look bad. Their less than forth coming reply seems to reinforce this impression. On what grounds is their explanation confidential?
Jason
You do seem to get lots of correspondence regarding published scientific work (of high public interest) that you aren’t allowed to share.
#18. Science only said that the comments were confidential and prior to public posting, "Any such use should be verified with the original authors." If I don’t get anywhere with O.B., then I’ll go back to Science and argue that there is no basis for confidentiality. Email addresses for Briffa and Osborn for anyone interested in their own confidential explanation are k.briffa at uea.ac.uk and t.osborn at uea.ac.uk.
With apologies for my ignorance on this subject. Can anyone tell me, or point me where I can find out :
1 What exactly are CRU keeping secret ?
2 What is the public reason they give for doing so ?
3 Re #17,Pat Frank : Where is Jones on record with this ?
4 Am I right in thinking that these data are used in all sorts of studies that go into the IPCC ?
#21 fFreddy – Re 3: Von Storch confirmed and testified at the NAS panel of the accuracy of the quote by Jones.
I’ve seen the original e-mails and the remark comes right out of the blue in what was, up till then, a very cordial and professionally courteous e-mail exchange with Warwick Hughes.
I wonder if Phil Jones regrets saying it.
Re #21 item 1 fFreddy,
There are collections of temperature data from many locations. CRU uses some subset of such collections, but does not specify which subset.
There may be other pertinent information that CRU does not specify, such as what adjustments are used for which data, what data is rejected as probably erroneous, etc.
JerryB (or someone else), why “cancellation” does not hold for GISTEMP, i.e.,
?
What I mean by this, is best explained by an example:
Select: Land=GISS2001, Ocean=None, Map Type=Anomalies, Mean Period=Annual (Jan-Dec), Smoothing radius=250cm, Projection type=regular.
Now calculate the anomality of the year 2005 (time interval begin=2005, end=2005) vs 1881-1910. You get 1.11. Next calculate 2005 vs. 1931-1960, and you get 0.73. Finally calculate the time interval 1931-1960 vs 1881-1910, and you get 0.30. So under normal logic this gives 2005 vs. 1881-1910 0.30+0.73=1.03…
Re #25 Jean S,
While I’ve noticed various oddities in some GISS calcs, including funny rounding, I can’t help with that one.
The “Input Elements” section of that page is cute: “NO land temperature data are used” and “NO ocean temperature data are used”. Hmmmm. 🙂
Re 25&26:
1. Pedantic note – smoothing is 250 km, not 250 cm.
2. Entering “none” for Land AND Ocean gives an Internal Server Error (The server encountered an internal error or misconfiguration and was unable to complete your request). I think they save search results (for quicker retrieval) so that subsequent attempts using the same inputs may give a results page with no graphics or data instead of the error message.
3. My non-expert theory on Jean’s reported peculiarity is that these are geographically weighted averages*, so when the geographic extent of data for the base period changes, maybe the weightings change. So the equation would be
where w is a number or function (I don’t know). For example, the graph for the Jean’s first set of inputs has a small, isolated segment near lat=0. Even though the data for the segment is 3 pairs of identical numbers at lat=7S, 5S, and 3S, the segment slopes downward to the right.
* You can download the data using links below the map/graph. Straight averages using Jean’s inputs are 1.418, 0.964, and 0.455, compared to the given results of 1.11, 0.73, and 0.30. (0.964 + 0.445 = 1.419, close enough.)
Re #25
The answer is: nobody knows why. Perhaps the laws of algebra are slightly different in Norwich.
The whole question should be investigated by a statistical audit team, wouldn’t you agree?
In order to get the confidential information you have to request it on special forms. Oh, I forgot, the forms are confidential.
Here’s a puzzle:
Go to GISTEMP and enter the following:
and hit submit.
Would anyone like to tell me where (other than the Antarctic Peninsula) where the global warming is?
Yes, I know that I chose the warm 1935-1945 period as the base, but isn’t that the point? Without adjustment (which is a mystery wrapped within an enigma) and without selecting a known cold period (1950-1980) as the base, where’s the warming?
All the secrecy in “Climate Science” really makes me suspicious, and it should make the policy makers suspicious, too. I think we have a “Climategate” going on here.
BTW, what is the deal about “don’t quote or cite,” relative to the AR4 stuff. Hell, it’s now public information and it should be quotable or citable, as long as the proper caveats are included. Can I discuss it with my wife?
Re my #27 (continuing this fascinating off topic discussion):
The 0.445 on my last line should be 0.455. Plus, I’ve figured out why the graph slopes downward near equator of the first plot. The first two “pairs” are actually sets of 5 points (3 of the higher and 2 of the lower), and the last “pair” is a set of 2 points.
I’ve been able to recreate all graphs with the given data (using straight averages), but not the summary numbers. Remember that the ends (poles) of the graphs are based on fewer and fewer data points, so a line shooting upwards may be misleading.
Re #30 (John A):
I’m assuming you chose the “Anomolies” map type, since that matches what you wrote, that the only significant positive area is Arc. Pen. When you choose “Time Interval: 1880-1999” you are not looking at the change from 1880 to 1999, you are requesting a comparison between the average from this time interval and the base period. You merely confirmed that the average temp. from 1880-1999 closely matches the average temp. near the midpoint of that range.
If one were looking for global warming, one would either choose the “Trends” map type with that time interval (base period is not used) or choose the “Anomolies” map type with an early base period (say, 1881-1910) and late time interval (say, 1971-2000).
One Trackback
[…] comment here from May 2006: Maybe responsibility for temperature collection should be taken away from CRU. […]