New Light on Acton’s Trick

The Guardian’s story on Oxburgh’s testimony (James Randerson here) is headlined:

Oxburgh: UEA vice-chancellor was wrong to tell MPs he would investigate climate research

and sub-headlined:

Edward Acton gave ‘inaccurate’ information to MPs by telling them the university would reassess key scientific papers following the UEA climate emails controversy.

And indeed, this is one of the few statements by Oxburgh to the Committee that can be taken at face value. Oxburgh’s evidence did shed some interesting new light on both Acton’s and oral evidence to the Committee earlier this year. Continue reading

Oxburgh Tricks the Committee: 45 Hours in Norwich

In response to a question asking Oxburgh about criticisms that the report – all five of pages of it – appeared “rushed and superficial”, Oxburgh described how the panel had patiently stayed in Norwich to see the project through. Oxburgh (my approximate transcript at about 10:51 time on program):

After we had done all the interviewing and talking and scrutiny, we kept the panel together in Norwich while the report was written and while it went through a series of drafts. So we did not go through the endless iterative procedures with which your committee must be familiar, circulating reports, getting comments here, getting them back, balancing them with someone’s opposing comments. We did it all around the table, that probably saved six weeks over normal procedures.

Q: we all appreciate . can I just clarify this point – you said that all the committee members stayed in Norwich.

Oxburgh – [unintelligible]

Q – OK. Does that mean that they were spending all their time on this report over that 2 week period?

Oxburgh – Not over 2 weeks. Probably over 4 days, 5 days something like that. They’d done a lot beforehand.

Q- How much time did each individual spend working on this report?

Oxburgh – Gosh, you mean altogether, not just in Norwich,

Q- You said that it had happened over a 3 week period but most of the time was spent in Norwich.

Oxburgh – People had done an immense amount of work before, one of the most important things. They had a really tough work schedule before they arrived. Then in Norwich, when they were there, they worked continuously. Total number of person-days spent on this was around 15. Something like that. It was… does that answer your question?

Climate Audit readers know that you have to watch the pea under the thimble. The MP was clearly left with the impression that the panel had carried out operations over a 3 week period and that “most of the time was spent in Norwich” – though it was he that said so, Oxburgh leaving the statement uncorrected, having previously created the impression by saying that the panel had stayed in Norwich “while the report was written and while it went through a series of drafts”, later leaving the MPs with the impression that they were there for “4 days, 5 days”.

Through FOI requests, we have obtained the actual schedule of the Oxburgh panel online here.

Here is the actual schedule for the panel hearings in Norwich on April 7-8.

9:30 a.m. – 9.45 a.m. Taxi to CRU (drop off Zicer Layby) Met by Acting Director, CRU Prof Peter Liss and Jacqui Churchill, VCO Coffee and Tour round CRU
9.45 a.m. – 10.45 a.m. Meeting with Phil Jones, Tim Osborn and team in CRU Library 30 minute presentation by Phil Jones followed by questions
10.45-11.00 am Coffee served in CRU library
11.00-12:30 pm Discussion – CRU Library
12:30-1:30 pm LUNCH for panel members – room number 00.2 CRU
1:30-3:30 pm Discussion – CRU Library
3.30-4.30 pm If needed: follow-up meeting with Phil Jones and Peter Liss
4.30-5.30 pm Panel private meeting
5.30 pm Peter Liss to chaperone Panel to Zicer Layby for taxis to hotel
7.00 p.m. Working Dinner at Caistor Hall

Thursday 8 April
8.45am- 9.00 a.m. Taxi to CRU (drop off Zicer Layby). Met by Acting Director, CRU Prof Peter Liss Coffee in CRU
9.15 a.m. – 10.45 a.m. Meeting with Phil Jones, Tim Osborn and team in CRU Library
10.45-11.00 am Coffee served in CRU library
11.00-12:30 pm Discussion – CRU Library
12:30-1:30 pm LUNCH for panel members – Sainsbury Centre, Garden Restaurant – Jacqui to collect and escort
1.30 p.m. – 3.00 p.m. Final Meeting
3.00 p.m. – 3.30 p.m. Coffee + Depart in taxis from Zicer Layby

Travel arrangements (obtained through FOI) show that this schedule was adhered to. Oxburgh arrived in Norwich at 6:30 pm on the evening of April 6 and had a train reservation back to Cambridge at 3.40 pm on April 8.

Their schedule lists two appointments with CRU staff (with one potential.) On April 7, they were to meet with “Phil Jones, Tim Osborn and team” from 9.45 a.m. to 10.45 a.m., described as a “30 minute presentation by Phil Jones followed by questions”. The next day, another meeting was scheduled from 9.15 a.m. to 10.45 a.m – making a total of 2.5 hours of scheduled meetings. The schedule provided for a possible “followup meeting” with Phil Jones from 3.30 to 4.30 pm on April 7.

Re-reading the schedule, it seems that the panel only spent a relatively small portion of its time actually interviewing Jones, Briffa and the CRU Team – who, by the way, seem to have been interviewed collectively rather than individually – and most of its time in “Discussion”. Scheduled “discussion” took place on the morning of April 7 (11-12.30), the afternoon of April 7 (1.30 to 3.30 and 4.30 to 5.30), a ‘working dinner”, and more discussion on the morning of April 8 (11-12.30) and the afternoon of April 8 (1.30-3), making a total of 7 1/2 hours of scheduled “discussion”, plus undoubtedly time at dinner and lunch.

Based on the schedule, most of the discussion in the morning and afternoon of April 7 took place after the Jones presentation in the morning and a half-hour of questioning.

Jones’ second scheduled interviews was in the morning of April 8, finishing at 10.30 a.m. Suppose we take the following statement of Oxburgh’s at face value:

After we had done all the interviewing and talking and scrutiny, we kept the panel together in Norwich while the report was written and while it went through a series of drafts.

If the report was actually written “after [they] had done all the inteviewing”, keeping “the panel together in Norwich while the report was written and while it went through a series of drafts” actually resulted in the panel being detained only until 3 p.m. the same day.

This was anticipated in a March 4 email from Trevor Davies to the UK government Chief Scientist John Beddington who had suggested Oxburgh to UEA:

Thank you for the intial suggestion! He [Oxburgh] has cleared April 6/7/8 in his diary for a 2-day session at UEA, and anticipates writing the report on the last day.

No doubt Oxburgh was happy to do a favor for the UK government Chief Scientist, but surely Beddington should have thought twice about asking a favour from someone who is chairman of a subsidy-seeking wind utility (Falck Renewables).

As to Oxburgh’s description of his ordeal in Norwich as “4 days, 5 days”, I’m highly sympathetic to the idea that spending almost 48 hours in Norwich seemed like “4 days, 5 days”, but using conventional time measurement techniques – such as checking the day of the week – the panel actually spent less than two days in Norwich, skipping town just as Geoffrey Boulton arrived for his one interview with CRU the next day about proxies, neither panel having bothered to compare itineraries, with Oxburgh and Hand spending an extra day in Norwich (and Graumlich only one).

Again, while Oxburgh didn’t correct the impression that the MP had been left with, Oxburgh himself didn’t expressly say that the panel had spent “most” of three weeks in Norwich. In Phil Willis’ terms, it was one more example of “sleight of hand”. But if you watch the pea under the thimble, you see that the impression that the MPs were left with does not accord with the actual itineraries and schedules.

More on other aspects of Oxburgh’s testimony tomorrow.

[Note – Sep 8 evening: In addition to the session of the full panel on April 7 and part of April 8, previously on March 30, Lisa Graumlich had visited CRU together with Hand and Oxburgh and met with Briffa in the morning from 9.15 to 10.45 and had panel discussions as well. The 15 person-days is 3×1(Mar 30) + 6×2 (Apr 7-8)=15. Oxburgh and Hand spent 3 days in Norwich, Emanuel, Davies and Huppert 2 days and Graumlich 1 day.

Josh’s cartoon says it well:

Oxburgh at Sci Tech Committee Tomorrow

Notice of the Oxburgh hearing is here at 10.30 am UK time ( 5.30 am Eastern).

The Science and Technology Committee will hold an oral evidence session following-up to the previous committee’s report on the disclosure of climate data from the Climatic Research Unit at the University of East Anglia.

The session will be on Wednesday 8 September 2010 at 10.30 am in the Thatcher Room, Portcullis House.

The Committee will take evidence from Lord Oxburgh, who headed the International Panel that was set up by the University of East Anglia to assess the integrity of the research published by the Climatic Research Unit.

The session will focus on how the review responded to the former committee’s recommendations about the review and how it carried out its work.

Watch the meeting live: Parliament TV http://www.parliamentlive.tv/Main/Player.aspx?meetingId=6581

Report: The disclosure of climate data from the Climatic Research Unit at the University of East Anglia (PDF)

An oral evidence session with Sir Muir Russell, who headed the Independent Climate Change E-mails Review, will be announced in October.

ICOADS – Hawaii

Although the formatting of the SST datasets needs to be completely freshened up, once again, before commenting, I commend the SST collaters for honoring their data by ensuring the preservation of comprehensive metadata – as opposed to their cousins at CRU and GISS. Unfortunately, there don’t seem to be any statistical analyses of SST measurements – by this, I mean, where the authors analyse the actual bias of changing measurement systems and provenance. (Although Thomson et al [Nature 2007] challenged earlier bucket adjustments, they didn’t do the sort of patient data analysis that the field cries out for. Continue reading

A First Look at ICOADS

For quite a while, I’ve urged people interested in gridded temperatures to really look at the SST data – realdata not adjusted data. SST makes up 2/3 of the record, but temperature critics spend 99.99% of their time on land data. In part, it’s because the data sets are much larger, but increased power of ordinary laptops is making these data sets accessible without work stations, whereas this was not the case even a few years ago. I’ve taken a first look at ICOADS – and since we’re talking climate science – naturally the data has some peculiar features. ICOADS collaters deserve great credit for their care with metadata – they obviously feel a responsibility towards the data that wasn’t felt by CRU (who notoriously kept only the “value added” version.) The data sets are large and rich and deserve a great deal of statistical analysis – basic cross-classifications as opposed to rushing off to make bucket adjustments. (I’m going to put this file away unfortunately, but commend it to others.) Continue reading

Tar and Z

Over the weekend (before I picked up my “regular” files), I started looking at Steve Mosher’s use of raster and zoo – both of which intrigue me a great deal, but got intrigued by something else and ended up finally figuring out how to extract .Z files within an R script without having to handle them manually. (R has utilities for .zip and .gz files, but not the older .Z format.) This isn’t anything other than a nuisance with GHCN which only has one .Z file to worry about, but was a big problem with the very large ICOADS SST data where every month of data is in its own .Z file and manual processing isn’t an alternative. It’s further complicated since the 12 monthly .Z files for each year and packaged into an annual .tar file. Continue reading

Back from Erice

Got back from the annual WFS conference at Erice, Sicily a couple of days ago. It is an interdisciplinary conference on world issues, in which climate is only a part, but an important part. As in previous visits, it was a very enjoyable visit – the conference attracts a lot of spouses, resulting in more mixing between disciplines than would occur otherwise as the spouses create their own mixing.

Antonino Zichichi, who organizes the conference, is not convinced that climate sensitivity is very great and the climate invitees had a much stronger “skeptical” bent than previous years. I ended up as a panelist in a discussion on climate with Richard Lindzen, Will Happer and Richard Wilson (of Harvard, another prominent nuclear physicist who’s taken an interest in climate.) I spent quite a bit of time with all three.

The conference had interesting presentations on science issues related to the BP oil spill – a discharge estimate of about 70,000 bbl/day seems to be the final number.

The conference always has interesting presentations from nuclear scientists and engineers and this year was no exception. I find the discussions of designs and design improvements fascinating and reassuring.

While climate sensitivity was the large scientific issue, Climategate was on a lot of people’s minds – both for its fallout on climate science and science more generally. I ended up focusing my presentation on Climategate and the inquiries, neither of which made people very comfortable.

Obviously, the tide of climate news continues. I haven’t parsed McShane and Wyner yet and need to do so. As a relaxation when I got home a couple of days ago, I spent some time looking at Steve Mosher’s recent blog – Steve has done some slick applications of R packages raster and zoo (which I haven’t explored) to the analysis of data sets. I see that the IAC report is out today and is one more thing to read – I hope that it’s less bad than the others.

Kriging on a Geoid

Geoff Sherrington and others on the First Difference Method post have requested a post for discussing Kriging.

I am new to Kriging myself, so please correct me if I make any errors here. Steve McIntyre (who may be on the beach at the moment!) is far more knowledgeable, and has posted about the topic frequently on CA. See, for starters, “Toeplitz Matrices and the Stahle Treering Network”, 3/22/08, “Antarctic Spatial Autocorrelation #1”, 2/20/09, “Steig Eigenvectors and Chladni Patterns”, and follow-up posts.

Wikipedia has a useful post on Kriging, in which it draws a distinction between “Simple Kriging”, which assumes a known constant unconditional mean for the random field, and “Ordinary Kriging”, where the unconditional mean is unknown. In the former case, the predicted value at an observed point in space will be a convex combination of the unconditional mean and an average of the observed values, while in the latter case it will just be an average of the observed values, with weights summing to 1. We are mostly concerned with “Ordinary Kriging”, though the distinction should be kept in mind.

Continue reading

Replicating McShane and Wyner

R coder mind of a Markov chain has replicated portions of the M&W work.

They write:

There are a bunch of “hockey sticks” that calculate past global temps. through the use of proxies when instrumental data is absent.

There is a new one out there by McShane and Wyner (2010) that’s creating quite a stir in the blogosphere (here, here, here, here). The main take out being, that the uncertainty is too great for the proxies to be any good.

Here’s an output from the replication:

More including R code here:

http://probabilitynotes.wordpress.com/2010/08/22/global-temperature-proxy-reconstructions-bayesian-extrapolation-of-warming-w-rjags/

The First Difference Method

Over on WUWT (http://wattsupwiththat.com/2010/07/13/calculating-global-temperature/), Zeke Hausfarther and Steven Mosher have been discussing the calculation of global temperature from station data. They list several methods of combining records, noting that most of the major indices use the Common Anomalies Method (CAM). They mention, but do not discuss, the First Differences Method (FDM).

In fact, FDM is far superior to any method based on averaging anomalies. Simply averaging anomalies relative to each station’s mean (as in the much-discussed Steig et al 2009 study of Antarctic temperaturessee below) greatly understates any trend there may be in the data, while using a common period unnecessarily restricts the available data. At the same time, FDM eliminates the need for opaquely complex adjustments for TOBS or MMTS, and automatically takes care of station moves.

Suppose, to take a purely hypothetical example, that four stations, A, B, C and D, have partial data on years 1-5, as indicated in the following table:

Table 1: Temperatures (°C)

Station Yr 1 Yr 2 Yr 3 Yr 4 Yr 5
A 13 14
B 10 11
C 22 23
D 15 16

Clearly all the stations point to a +1°C/yr uptrend in temperature.

If we compute anomalies for each station, as was done by Steig et al (2009) (to the best of my knowledge), and then average over available stations (S09 did something much more elaborate), we obtain the following:

Table 2: Anomalies

Station Yr 1 Yr 2 Yr 3 Yr 4 Yr 5
A -.5 +.5
B -.5 +.5
C -.5 +.5
D -.5 +.5
Avg. -.5 0 0 0 +.5

The Least Squares trend in the average anomalies is only 1/6 1/5 °C/yr. [(-2)^2 + (-1)^2 + 0^2 + 1^2 + 2^2 = 10, not 12 as I computed last night!]

The First Difference Method, on the other hand, first computes annual differences as available for each station and then averages the first differences as available across stations. The average first differences are then cumulated from an arbitrary starting point (Year 1 in the following table), and then may be adjusted as anomalies relative to any desired reference period (Years 1-5 in the last row of the table):

Continue reading