The Oxburgh Report stated:
The eleven representative publications that the Panel considered in detail are listed in Appendix B. The papers cover a period of more than twenty years and were selected on the advice of the Royal Society.
This statement has been questioned ever since the publication of the Oxburgh Report. That the Royal Society did not select the papers has been clear for some time.
In Oxburgh’s testimony to the Parliamentary Committee, Oxburgh stated:
Q – Right. Can you tell us how did you choose the 11 publications?
Ox- We didn’t choose the 11 publications. They were basically what… We needed something that would be provide a pretty good introduction to work of the unit as it had evolved over the years. The publications were suggested to us came via the university and by the royal society, I believe. We feel ..let me just emphasize..they were just a start… because all of us were novices in this area, we all felt that they were a very good introduction – we moved on. We looked at other publications… we asked for raw materials, things of that kind. The press made quite a meal out of the choice of publications. For anyone on the panel, this all seems over the top. It didn’t have that significance.
Q – there are two things that arise out of that. It was a small unit. Are you saying that Jones, the subject of the investigation, chose the papers that were to be investigated… and that it wasn’t the panel or royal Society?
Ox – No suggestion Jones chose them,
Q – Where did they come from?
Ox- I believe they came … I suspect that that the […] involved was Professor Liss who was acting head of the unit who’d been brought in from outside the unit…he’s been an chemical oceanographer who is broadly interested in area. he in consultation with people with royal society and maybe others outside the unit who had some familiarity.
Q -So the list did not come from the unit – you’Re absolutely categorical ?
Ox – Well I cant
Q – So the list did not come from CRU?
Ox – I can’t prove a negative. There’s absolutely no indication that it did.
Q – Your publicity said that it came from royal society. The Panel given list before royal society asked.
Ox – I… Not as far as I know. You Might be right but I don’t believe so. No certainly I don’t think that can be true.
In a recent post, I observed that the list of eleven publications was sent out as early as March 4 – well before a perfunctory email from Trevor Davies to Martin Rees and Brian Hoskins of the Royal Society on March 12 saying that Oxburgh wanted to be able to say that the list had been chosen “in consultation with the Royal Society”, even though the list had already been sent out.
I recently noticed that Lisa Williams of the UEA Registrar’s Office was shown as the author of the list version sent to panelists – thereby offering a lead towards solving the authorship of the list, which was accompanied by the statement:
These key publications have been selected because of their pertinence to the specific criticisms which have been levelled against CRU’s research findings as a result of the theft of emails.
Today – after almost six months – the riddle of who prepared the list is resolved.
Lisa Williams wrote:
Dear Mr McIntyre
In response to your recent enquiry I can provide the following information.I understand that the list of 11 papers for the Oxburgh review was collated by Prof Trevor Davies, in consultation with others. He was also the author of the statement at the bottom of the list.
Yours sincerely,
Lisa Williams
So the list was not selected by the Royal Society after all, but by Trevor Davies, the pro-VC of the University and former director of CRU. In consultation with “others”. Dare one hypothesize that these mysterious “others” will turn out to be Jones and Briffa after all?
Climate Audit 
116 Comments
This is the dawn of a new era!Can’t wait for the day our dearest friends at HMRC (IRS) will rely on taxpayer-provided evidence alone!
Q – So the list did not come from CRU?
Ox – I can’t prove a negative. There’s absolutely no indication that it did.
That statement is quite lawyerly. He’s only going to concede what’s been proven.
The only way he couldn’t prove it didn’t come from CRU is if it came from CRU. Otherwise, uncovering the source would disprove the CRU genesis.
As for negatives being unprovable, every time a criminal says he didn’t do it and a conviction follows, I suppose an injustice is being done?
Should be an acquittal, not conviction
Harold
On the contrary, a conviction proves, beyond reasonable doubt that the denial is false.
Appears Lord O got lucky with the way the question was framed. If it had been “from within the UEA” rather than “from CRU” he might have had to say yes, but since Davies was a former, rather than current, CRU member Oxburgh was able to say what he did without lying. It’s what people in the London insurance market used to call a “pick up” in which the listener is encouraged to form a conclusion directly opposed to the truth, without an actual lie being told.
If the University of East Anglia were a London Cabbie you’d be saying don’t take me the long way round guv?
He’s also obviously heard the phrase, ” Less is More.” Look how vague the answers he gives are. He says as litle as possible, while trying to sound authorative…he must of been squirming inside.
The list was obviously chosen carefully by someone, or two, who were very familiar with the subject matter. If you don’t look for something, you can’t miss it.
I note that Lisa William’s email says “the list … was collated by Prof Trevor Davies,” not that Trevor Davies actually selected the papers on the list.
Maybe the others who were consulted actually chose and provided the papers, and Trevor Davies merely collated them into a list.
One step at a time, Steve. 🙂
Re: Pat Frank (Sep 16 11:11),
The follow up question is clearly “Who provided Lisa Williams’ answers”. Perhaps they were “collated” with the assistance of “others”.
You have a point here. One has to assume that she obtained clearance from some one on high for the content of her reply. She must be aware of the significance of the question (unless she lives in an incredibly tall ivory tower!).
No doubt it’s just as stage managed as any other response – that doesn’t reflect badly on Lisa Williams of course.
I’ve just made an FOI request to the UEA for that information.
Sorry. Oversight. Here is the request.
On about 16 September 2010 you wrote the following to Dr Steven McIntyre:
Dear Mr McIntyre
In response to your recent enquiry I can provide the following information. I understand that the list of 11 papers for the Oxburgh review was collated by Prof Trevor Davies, in consultation with others. He was also the author of the statement at the bottom of the list.
Yours sincerely,
Lisa Williams
Kindly supply me with any document in the possession or control of the UEA:
(a) to or from Davies concerning the choice of the papers or the identity of the “others” in the above quotation;
(b) to or from you concerning the choice of the papers or the identity of the “others” in the above quotation;
(c) to or from anybody else concerning the choice of the papers or the identity of the “others” in the above quotation.
You make take this as a request under the FOI Act.
No doubt Palmer will change it to a request under the EIR but that would be par for his slippery course.
Yes, I saw the pea under that thimble instantly
We’re not there yet, but UAE’s thumbprint is definitively identified
Now, who were the “others” that TD consulted ?
Steve,
Going back to your commentary in your blog on 20 July about the Guardian Panel:
“At the reception, Davies challenged me in front of a reporter to withdraw some supposedly inaccurate statements at Climate Audit about the Oxburgh inquiry terms of reference. The reporter seemed to want me to make the changes on the spot at the reception. I said that I would look at the matter when I got home and that I would be more than willing to correct any inaccuracies. They sent me a copy of a statement that they released after Roger Harrabin’s story on the matter, but, thus far, have not responded to two requests to identify what, if anything, at CA requires correction. More on this on another occasion.”
So you had personal conversations with Davies, and HE challenged you about making inaccurate statements concerning the Oxburgh Inquiry? This is hilarious. The man is devious to say the least. Now he has been caught red handed and has lost all credibility. “Independent” inquiry …. Hah!
Spot on about the man’s smiling duplicity – I well recall the smirk right from the moment George Monbiot got the Guardian debate underway, before the interaction with Steve in the wine bar later in the evening. The key point that hits me right away about what Lisa Williams has now disclosed is that, whatever she may or may not have understood, Trevor Davies knew very well that the selected list left out the key papers about which Steve, Ross and many others had been exercised for so long. Whichever ‘others’ were involved, as the senior man who very well understood the game being played, things are not looking good for the Vice-C.
Wow. CA may not post 5 times a day, but the content, continuity of argument, and mountains of effort behind each post sure sets it apart. Great job chasing that pea.
Oxburgh’s response (if required to make one) would likely be that the inquiry’s conclusions would of still been the same.
Looks like the parsing is asymptotically stumbling towards a semblance of honesty – ‘selected’ and ‘chose’ have become ‘collated’, but the UEA has finally admitted that it created the ‘grouping’ or ‘set’ which Oxburgh found so helpful in framing his ‘independent’ inquiry.
I fear that Trevor Davies will be retiring soon, in the company of “others”.
Have no fear. He’ll be given a knighthood for services rendered and then given a few directorial posts with fat rewards for doing nothing of any use – nothing changes.
Yes, I am too idealistic. I forgot that what counts these days is not honesty but the ability to lie brazenly. Perhaps though, even in the modern world, the team can be faulted for their inability to lie efficiently. Perhaps this is enough of a sin against the establishment to provoke early retirements.
ZT – are you sure it is a modern phenomenon? Maybe the general global level of deceit is actually remaining roughly constant with local hot spots of activity moving from intrigue to intrigue and it is just the power of IT and FOI legislation which is making it easier to expose and publicise particular incidences? Perhaps we need a Global Deceit Anomaly reconstruction – though I’m not sure what we could use for reliable and proven proxies.
Wasn’t there a time when a word was a bond? This is England, after all.
I thought it was “a gentleman’s word is his bond” – so maybe the problem is a decrease in the number of gentlemen…but then perhaps there is a feedback loop involved too 🙂
So what to make of that email from Davies (to Rees? – I forget, offhand) in which he said that Oxburgh wanted to be able to attach the Royal Society’s stamp of approval to the list? This has never, to me, squared with Oxburgh’s consistent denial of knowing where the list came from. Might it actually have been Davies who initiated the idea of seeking RS ‘approval’, to disguise the real source?
The issue is, the Royal Society has always been supportive of the CRU, and of Anthropogenic Global Warming. So to even have their stamp of approval looks bad…but when you have a choice of them or the University, I guess they chose the lesser of 2 evils…
In your “Who Made the List” post, Trevor Davis wrote to the Royal Society:
“[The list] represent the core body of CRU work around which most of the assertions have been flying. They are also the publications which featured heavily in our submission to the Parliamentary Inquiry, and in our answers to the Muir Russell Review’s questions.”
Phil Jones and company must have been involved in compiling the information submitted to Parliament and Muir Russell, so there is now some evidence linking them to the list. The interesting questions are: 1) What documents were cited in these submissions that did NOT appear on the list of eleven? 2) Does the real controversy center around the documents they cited in these submissions and then did not recommend to the Oxburgh committee. 3) Were the same documents cited in your submission or other outside submissions to the inquiry?
Maybe someone on the Oxburgh committee will be disgusted to learn that the UEA hid the truly controversial papers cited in their submissions while they asked them to waste their time on the non-controversial papers.
Your last point, IMO, rings true with the diligence shown in Professor Kelly’s notes.
Now someone needs to speak to the Royal Society about being a rubber stamp on a coverup.
Formerly the greatest scientific body in the world…
In this climate, one hopes Miss Williams honest answer to your query won’t cost her her job.
As another lady with considerable guts said the other day, following Thucydides: “The secret to freedom is courage.” (I’ll let people google that, as the context takes us well off topic. But the phrase stuck with me.)
It should. From my past experience with her she’s a willing stooge.
I’m tempted to suggest this story will run and run.
The one thing you tell people about how to respond to a deposition is to only provide the minimum amount of information to answer the question. What’s odd about Williams’ response is this line:
“He was also the author of the statement at the bottom of the list”.
What was this statement at the bottom of this list and why was it important for Williams to include that information in her response.
According to the list that Steve linked in the previous post, the “statement at the bottom” would be this:
“These key publications have been selected because of their pertinence to the specific criticisms which have been levelled against CRU’s research findings as a result of the theft of emails.”
It seems Ms. Williams wants to be very clear that she had no part in creating the list. Can’t say I blame her.
Am I being obtuse – or is it not the case that “the statement at the bottom of the list” is utterly disingenuous, if not a downright lie ? The list of 11 papers were not the most pertinent – the list EXCLUDED the most pertinent CRU papers.
And now we know who made the statement. Not some Royal Society guy a bit away from the action – but a top man at UEA steeped in all the controversy, who had been dealing with the whole furore for many weeks and therefore cannot plead ignorance.
John Anderson
It was a downright lie.
See my comments above
Ms. Williams seems much more forthright that all the UEA bumblers around her who are more highly paid.
If someone ever decides to clean up that university, boot all the vainglorious chumps and let Lisa Williams take over.
I remember well a conversation with Prof Trevor Davies after the Guardian Debate when I raised with him the peculiar matter that none of the papers examined by Oxburgh inquiry had featured particlarly in allegations made by critics.
TD feigned surprise and challenged me to identify papers which would have been more relevant to have put before the Inquiry but unfortunately I was not close enough to the subject to be able to reel them off – but I will be glad if he is beginning to regret his attempt at subterfuge.
TD also took great pain to stress how utterly beyond reproach such eminent people as Lord O and certain members of the RS – but I wonder just how many eminent people knew about this particular subterfuge and simply kept quiet?
Re: matthu (Sep 16 15:55),
So… either TD was lying, or it wasn’t even he who wrote the list of papers but merely endorsed it, and provided another layer of screening for the real author(s) – without realizing the key papers were missing. In fact, going by Lisa Williams’ final remark, that seems the more likely.
Either way it’s unconscionable, so it seems to me
Remember that Oxburgh stated that the 11 papers were just an introduction or starting point for the investigation/discussions and that the committee “moved on” to other documents (papers?)
What other documents and who provided those documents? Were the members truly and sufficiently naive about the issues surrounding climate science and climategate to be lead down the garden path by whoever conveniently came along to provide documents. And then as second thoughts might have formed on how this might appear with regards to impartiality, we get a stumbling and confused testimony from Oxburgh.
Certainly if the papers didn’t matter and the critics didn’t matter, it makes one wonder just what did matter.
The credibility, the public relations, the state of the propaganda push … nothing else
Bishop Hill had a post which sheds some light on the investigation as viewed by panel member David Hind. Dr. Hind asks good questions, does his own research, and even reads The Hockey Stick Illusion.
http://bishophill.squarespace.com/blog/2010/6/22/the-hand-emails.html
It’s David Hand, of course. But agreed, we await his comments on the origins of the list with great interest.
Re: harold (Sep 17 13:16), ah, so David Hand did serious reading outside “the eleven” and this included HSI and CA.
Hand’s April 2010 Telegraph interview quotes Mann as saying that Bloomfield (statistician for the NAS panel on MM 2003) arrived at opposite conclusions to those of Hand.
But Bloomfield then emailed Hand to say
Another upside-down Mann statement?
There is a recurring theme that UEA looked at these investigations as a way to discover who hacked the emails. I am not left with their impression that they were wholly serving a need to evaluate thir work or honesty. So, I try to read the back-and-forth also from the point of view of “How can we find who leaked these?” It can give a different emphasis.
That said, the developing evidence for evasive answers within the old boy network is well documented, deplorable, but not unexpected in the wider scheme.
The momentum is moving from investigation to remediation.
I wish I could find this shocking, but sadly, I don’t have the energy. I think I have the CRUDS
Climate Research Unit Disenfranchisement Syndrome.
Kudos to Steve for chasing the pea through the maze to find it under one of the many thimbles.
Might we now also know the identity of the “mole”?
Oh for goodness sake. Can’t these people just give a straight answer to anything? Rotten to the core.
Like my mama used to say, “When first you practice to deceive . . .”
What’s amazing is that the great and magnificent Lord Oxburgh, with his Parliamentary mandate, along with the “complete cooperation” of the UEA, was unable to find any evidence at all that the list came from someone inside of the UEA. But the ultimate pariah, he whose name shall not be spoken, was able to find out who made the list simply by examining the Word document and then sending an email to the author.
To me the truly shocking thing is the way we are expected to accept that this kind of shoddy, misdirected and biased review as representing a vindication of the “science”.
It took decades to uncover the duplicity of the Widgery commission. The names of Oxburgh and Russell are likely to be similarly regarded within a much shorter timeframe.
“It took decades to uncover the duplicity of the Widgery commission.” I think you’ll find that all the victims families and most of the UK knew Widgery was a cover up, as they do about Hutton.
But the pattern is the same with Oxburgh and Russell, inquiries contemptuous of the public set up to give a cloak of respectability to otherwise obvious malfeasance. They don’t learn, decent people won’t let these matters go, they didn’t with Widgery and won’t with Hutton, and now Oxburgh and Russell.
I am really curious as to what papers the panel “moved on” to, once they had completed their consideration of the original 11. Did they keep any records of these, what findings did they make in respect of these other papers, how were they chosen, did they include any of the controversial papers?
Given the “rigorous schedule” outlined, it is hard to see how the panel had a lot of time for detailed analysis of any further papers. Certainly the notes released by Prof Kelly do not indicate that he looked any further,or believed that was in his remit.
Oxburgh is careful to say that the panel “were not prevented” from looking at other papers, implying that other papers were examined, but there is, as far as I know, no evidence anywhere that further material was actually considered.
As with the rest of the report, we are just expected to take their word for it. These people just can’t understand why we don’t trust them
Re: Peter Wilson (Sep 17 01:29),
Even funnier is Oxburgh’s statement “we asked for raw materials”. Sure, they worked with raw materials to produce their five page report.
It’s amazing this guy hasn’t died of embarrassment.
Does anybody really believe that they even read the eleven selected papers, much less any “moved on to” documents or “raw materials?” I’d bet a case of beer that if any of them did, it certainly wasn’t Oxburgh.
Agree. That’s why I think Oxburgh’s response will be that none of this makes any difference. There would have been no impact on his report regardless of which papers they read, what is in the papers, or whether they read any of the papers. He pretty much knew what he was going to report the day he was hired.
Peter, You have to parse the English spoken by these people very carefully. That he said they “were not prevented” from lookming at other people does not mean they did look at other material. I’d say probably not, otherwise he would have bragged about how thorough they had been.
I bet 100 quatloos that they didn’t even read the 11 papers, lest it give them a headache. The 11 papers were just an excuse.
That is a bet I wouldn’t take.
Kelly read at least some of the papers – twice – and Kelly’s reaction is well documented.
Steve – many congratulations for your excellent dective work.
May I suggest just a few steps more to completely nail them.
(1) keep on this issue – just who presented the data that Davies collated?
(2) exactly which other papers and documents were considered by the venerable committee that found CRU “not guilty”.
(3) I also suggest you go back and collate the items that you brought to the attention of the committee. These need to be held up in comparison to those that actually WERE considered”.
You are getting to the heart of the matter.
I encourage you to persist.
sniip – policy
The MSM is very quite on these issues, it is interesting to note that Graham Stringer was present at the press conf releasing Bishop Hill’s study on the various inquiries and I am sure he will get the information from this blog sent to him. Acton is up next at the Parlimentary Committee, just hope they can ask more pointed questions and more importantly not accept disingenous waffle disguised as ramblings of an old codger as a satisfactory answer. Got to give it to Oxburgh, he knew how to ramble on giving only a fleeting glimmer of real content.
It would be good if Ox could be recalled.
Also can’t wait to see how many ways the word Collated can be parsed.
If Oxburgh’s report is anything to go by (after all, what else do we have?!), in this instance “collated” could well mean: “dutifully enter the selected reference citations into a word doc so that Oxburgh can copy ‘n paste into his report, then staple the papers together.”
Might we now say that the old Ox has been gored? (Or is that “Gored”?)
I would have thought that any independant chairman of integrity would wish to know where the evidence put before him came from and who chose it. Albeit a competent chairman would have either chosen the information to view after reading the emails or delegated the task to an independant person unrelated to the now totally discredited UEA.
But after all he did not read the emails?
“Collated” is not the same as “selected”.
Definitions:
Collated
1. Examined and compared carefully in order to note points of disagreement.
2. Assembled in proper numerical or logical sequence.
Selected
1. Chosen in preference to others or because of special value.
2. Chosen or preferred items or people considered as a group.
So the question remain who selected (or chose) the 11 papers.
If it wasn’t Trevor Davies (he only collated them) then who?
All fingers point to scientists in CRU!
Re: Mac (Sep 17 06:51),
Just noticed this, and you’re 100% correct of course.
If they could have said “selected”, they would have said “selected”.
But they can’t, so “collated” it is.
They really are utterly determined to make themselves appear as untrustworthy as possible. I can think of no possible reason for the embarrassing and farcical way that they insist on these syntactic fig-leaves other than the most scandalous and obvious one.
I wonder how many more iterations it’ll take to get there?
Given the source of the papers used in the review, and the fact that Lisa Williams of the UEA was handling Oxburgh’s email, one might wonder who actually authored the ‘Science Assessment Panel’ report. Checking the author of the PDF shows that this is ‘k319’. Not very revealing, but a quick google shows that this is Annie Ogden, head of communications for UEA media relations (See: http://www.uea.ac.uk/mac/comm/media & http://eastangliaemails.com/emails.php?page=43&pp=25). Now, of course, it might well be perfectly reasonable for the media office to be making the PDF that is posted on the web, but one would have (naively) thought that the last person to alter the document would be a member of the ‘independent’ inquiry team.
Thx for the link: “UEA has broken the mould in a number of areas, from creative writing to environmental sciences”
Speaks for itself; inter-faculty teamwork.
Yup, and isn’t it a little odd that the author of Oxburgh’s review, from its document properties, is the ‘head of communications for UEA media relations’, the chief creative writer for the creative writing university.
Perhaps we should hope for Wynnes Law – ‘Negative slack tends to increase’?
presumably everyone here will make a submission? With very specific direct questions. IE What is the exact list of papers that should have been looked at?
Bishop Hill has provided a link to some new submissions to the Science & Technology committee. I found David Holland’s submission well worth a read.
I sent this to Acton about the UEA submission to teh parliament.
In an “Opinion Piece” in Appendix A1, under your signature, we find the following:
Following this month’s Muir Russell Review of ‘Climategate’, which dissected the allegations made against scientists at UEA’s Climatic Research Unit (CRU) and concluded emphatically that ‘their rigour and honesty is not in doubt’, the President of Universities UK, Steve Smith condemned the subjection of researchers to intimidation and threats:
‘Attempts to create controversy and discredit researchers in some fields serves only to erode public trust in our researchers and risks setting back progress in many key areas.’
Smith remarks that if Einstein had been subjected to such challenges when his research was in the formative stage, his reputation would have been terminally damaged before he got to the theory of relativity. Or take Darwin. Almost two decades elapsed between his voyage on HMS Beagle (1840) and publication of On the Origin of Species (1859). Had he been forced to release his momentous musings before he was ready, he might well have been stopped in his tracks like other pioneers of evolutionary theory.
Under the FOI Act, send me those documents in the UEA’s possession or control that:
(a) justify saying that any CRU researcher has been subject to “intimidation and threats.”
(b) Justify the remark about Albert Einstein.
(c) Justify the remark about Charles Darwin.
For your information, Einstein was always ready to defend any of his theses. How, in any event, could his reputation have been terminally damaged by someone “challenging” him? His work was not properly called research in any sense relevant to the CRU’s activities. Einstein did not work from any data, except the results of the Michelson-Morley experiments demonstration the invariance of the speed of light with the velocity of the source. His own work consisted of conducting “thought experiments” (his term) and describing those experiments mathematically.
He would only have been discredited had, for example, his Special Theory not accorded with empirical reality as determined by later physical measurements. I expect it was just that prospect that caused your staff to destroy and hide data and software. Jones in fact objected to an FOI request on the basis that the applicant only wanted to prove him wrong. Some scientist! You should transfer him to the Social Work department.
Smith is an ignoramus and you are trying to snow the Parliament by citing him to it.
You evident approval of Smith’s phrase “intimidation and threats” to describe criticism of research methods, results and data shows that you are of the same mind.
Then there is the following passage, which is untrue so far sceptics are concerned, but in its overwrought prediction of imminent catastrophe contains a delicious irony:
Until the line is soundly drawn and widely understood, there will be unfortunate side-effects. Any refusal or reluctance to disclose is easily read, especially by those in the grip of a conspiracy theory, as sinister. As one commentator on the CRU affair pointed out: ‘Like Desdemona’s handkerchief, Climategate offered absolute proof to those maddened by paranoia, but to the rest of us it remained just a handkerchief.’
Smear tactics may matter little in this instance, given that the remorseless upward trend of global temperatures is so carefully verified internationally, and that it seems to become grimly clearer month by month. But it is easy to think of more virulent belief-systems feeding upon the refusal of an FOI request, however legitimate the grounds for refusal.
I believe Einstein said something along the lines of (I dont know the exact quote):
“It doesn’t matter how many scientists agree with me, it only takes one to prove me wrong”
I get the feeling the list has to be from Jones and Briffa because, outside the circles of “
hethose who must not benamedinvited to give evidence” I’m doubtful if anyone understands the science sufficiently any more.* They’ve abandoned Scientific Method (the science establishments, publications, school curricula)
* They don’t teach Scientific Method so the next generation don’t even appreciate its absence
* Good brains have been diverted into the incredible computer revolution
* Meanwhile the science disciplines have grown more and more specialized, jargonized, incomprehensible
* But THE SYSTEM are still rubber-stamping THE TEAM and pretending they do understand.
Thanks for articulating the point about the relative brain drain into the computer scene. This needs to be thought about a great deal more, especially as young megaliths like Google seem to buy the AGW orthodoxy wholescale, as do Silicon Valley financiers like John Doerr. (It’s not the whole story I’m sure, as shown by the young Google techies’ positive reaction to CAGW sceptic and arch-libertarian Ron Paul actually bothering to answer their detailed questions at the Googleplex in the run-up to the primaries in 2007. That video was at one stage beating all other candidates put together, Republican and Democrat, for views on YouTube. But I may be digressing here!)
It’s a point I’ve often thought about. Something has to explain the mediocrity of climate science, including its woefully naive approach to software and simulation, as old lags like Freeman Dyson immediately spot but young hopefuls like Gavin Schmidt continuously bungle (to take two of ‘our own’ apparently made good Stateside). What does Dyson’s daughter Esther make of all this, influential as she’s been in IT circles in times past? The answer to all of the above, needless to say, is that I don’t know. But well worth raising in the context of what’s gone wrong with our recent grasp of the scientific method.
My latest thoughts about the thorny boundaries between computer and climate science have to do the closed world assumption for databases and its opposite – surprisingly called the open world assumption. It was fun to learn the other day that a Dr Reiter did the pioneering work on closed world logic in 1978 – at the University of Toronto. Closed world theory first came to my attention at the European Conference on Object-Oriented Programming in Oslo ten years later, together with the old joke “It’s the job of the real world to fit with what’s in my database.” That sure rings bells for our predicament in the climate area. Chris Essex would no doubt know more. We all have much to learn from this debacle, once the smoke and mirrors are abandoned in favour of the radical honesty of a Dyson or Feynman – or indeed a Raymond or Torvalds in the open software area.
“young hopefuls like Gavin Schmidt continuously bungle (to take two of ‘our own’ apparently made good Stateside).”
Richard, I think Gavin knows exactly what he’s doing, and hasn’t bungled anything. I expect he’s decided what’s important, and what is not (to him), knows what to attack and what to defend, when to speak up and when to keep silent.
It’s my impression that there is a coterie of scientists who’ve decided that the integrity of science is best served by suborning it in a greater service.
As you reported, in Oxburgh’s testimony to the Parliamentary Committee, Oxburgh stated:
Q – Right. Can you tell us how did you choose the 11 publications?
Ox- We didn’t choose the 11 publications. They were basically what… We needed something that would be provide a pretty good introduction to work of the unit as it had evolved over the years. The publications were suggested to us came via the university and by the royal society, I believe. We feel ..let me just emphasize..they were just a start… because all of us were novices in this area, we all felt that they were a very good introduction – we moved on. We looked at other publications… we asked for raw materials, things of that kind. The press made quite a meal out of the choice of publications. For anyone on the panel, this all seems over the top. It didn’t have that significance.
The CRU announcement of the Scientific Assessment Panel (22 March 2010) states:
The panel will have access to any publications or materials it requests, and all information considered will be listed in the Report.
If the panel considered other publications, as Oxburgh states above, why were they not listed in his report?
Oxburgh asserts that he didn’t have time to review the science, but that it didn’t matter because the peer review process is essentially the gold standard for assessing the quality of the science. So the selection of the 11 papers really had no effect on his results, he would argue.
So his gambit in front of the committee in the future will likely be: “there’s no reason to worry about the science, peer review works”.
There’s a new paper out that calls this into question. This paper is definitely worth a read:
Click to access 1008.4324v1.pdf
The paper argues that (among other things) the presents of “friendship networks” in the peer review process considerably reduces the quality of literature in a particular domain to the point where the probability of a bad paper being selected is equivalent to a coin toss.
From the abstract:
“In theory peer review works if the involved parties (editors and referees) conform to a set of requirements, such as love for high quality science, objectiveness, and absence of biases, nepotism, friend and clique networks, selfishness, etc. If these requirements are violated, what is the effect on the selection of high quality work? We study this question with a simple agent based model. In particular we are interested in the effects of rational referees, who might not have any incentive to see high quality work other than their own published or promoted. We find that a small fraction of incorrect (selfish or rational) referees can drastically reduce the quality of the published accepted) scientific standard. We quantify the fraction for which peer review will no longer select better than pure chance.”
That linked paper has the quote of the month, IMO, “nepotism network”.
Re: Peer review
There’s an interesting assessment by Maciej Henneberg here:
http://naturalscience.com/ns/articles/01-02/ns_mh.html
Of peer review, Henneberg says “it is neither the best means of evaluating contributions to science nor the one most commonly used during the period in which the modern scientific method developed. Throughout history, most scientists published their views without formal review and peers published their criticisms openly.”
The work of Steve McIntyre and others, whether published on the web or in the journals, represents a return to that tradition of open and attributable criticism of published work.
Alfred Burdett–
Thanks for this. Until you mentioned it, I had not realized the truth of this. So here I take the risk of going OT, but it is now clear to me that NOT-peer-review is the only way to go: Publish without peer review and then critique without anonymity. The readers of all this can make up their own minds and eventually the truth will settle in. SM is–I agree with you–right on the money.
Shallow Climate – I’m not even sure that anonymity matters too much in blog land. It is the fact comments, arguments and rebuttals are out in the open in front of a very wide and well informed readership that makes it so powerful. I follow all this at a distance and there are many commentators I respect who post with “anonymous handles”. The way people present their points and evidence, as well as how they respond to others, demonstrates (or not!) critical thought in action and this is what IMO is essential to the progress of science. Peer review on the other hand largely takes place behind closed doors, potentially with reviewers afforded status independent of the quality of their thinking. Translating the blogged approach into a referenced canon of work presents challenges but nothing insurmountable IMO.
I have to disagree. As a working experimental scientist, I don’t want to have to wade through the blizzard of garbage analyses that would arrive, should peer review be abandoned.
Peer review is far from perfect, but it does weed out the worst sorts of nonsense that people get into their heads.
Willy-nilly publication of everything would not be an improvement.
In past times, journal editors would have made some sort of judgment whether to publish or not. What appeared in a scientific magazine in the 19th century, say, would have reflected the views of the editor. This can be a very narrow filter, and one that is more susceptible to personal idiosyncrasy.
At least with peer review, an attempt is made to provide a dispersed and more objective judgment about the quality of the work. Peer review, then, is an improvement over the prior state of affairs.
There are multiple journals available, where a worker can re-submit a manuscript rejected elsewhere. Chances are very good, though not perfect, that a valid piece of science will be published somewhere. If the science in some obscure journal solves a problem for someone, it will typically be found, used, and come thereby up into the light.
The best thing would be for the content of all journals to be abstracted into the web, where they are available for keyword searches. That would help elevate even obscure works into sight, where their utility can be evaluated and perhaps rescued.
But opening the door to uncritical publication is a recipe for an avalanche of garbage.
Pat,
Maciej Henneberg did not advocate “Willy-nilly publication of everything”. Far from it. He advocated having editors, i.e., people from among the most experienced in the field, acting as gatekeepers.
You suggest that if journal editors decided on what was acceptable they could impose “a very narrow filter.” But we know that editors of peer-reviewed journals can do that (without taking personal responsibility) by their selection and guidance of reviewers.
I think the real limitation on Henneberg’s proposal is the number of first-rate people who would undertake the task of being a real editor. Consider a medium-sized journal receiving, say, 500 papers a year. The editor(s) would have to evaluate every paper — one or two a day, and because authors would demand it, they would have to state carefully and precisely the reasons for every rejection. In addition, they would have to do the reviewers’ job of telling the authors of otherwise acceptable papers what improvements and corrections were needed before final acceptance. I have known editors do all this. But it involves a gigantic amount of work.
Re Curious and Pat Frank,above: Frankly (no pun intended), I find value in both your comments. So, thanks! But, perhaps not surprisingly, I lean more toward the views of Curious, who, I think, expresses my sentiments more completely (and more splendidly) and I did, or probably could. I am not too concerned about the “avalanche of garbage” stuff, thinking rather that under this format there would be more (legitimate) pressure on editors to vet submissions to their journals; now editors can hide behind “peer review” to a good extent, absolving themselves of due diligence if they so choose. (This has been discussed here at CA.) At any rate, I fear that the more I write here the more I am in danger of getting red-carded for being OT, although I would love to read more discussion of this topic. Thanks again to you both–it’s stimulating.
The virtual universe is vast and has room for any amount of ‘garbage’. The trick is to have everything published online but separated into compartments ranging from mainstream consensus,for and against, then through lesser or unqualified authors, and out to maveric, eccentric and apparently wacky.
Those who do the initial sort would not have the last word as crowd-ranking of contributions could result in moving a paper from one compartment to another.
If a political mob insist on voting a flawed paper into prime spot then its errors can be pointed out and seen by all. Its supporters may keep it were it doesn’t belong but it can do no harm and can be regarded as a caution or a standing cause of amusement.
Pat and Shallow Climate – I agree we are straying OT but I think the possibilities for blogging style critiques are evidenced by what has happened in the “climate science” arena. I also think that it has clearly demonstrated that peer review does NOT weed out garbage – Anthony and Jeff have both highlighted some pretty outrageous papers.
Pat, I respect your views and acknowledge I am not working in science and I’m just an interested observer. But I do think having a blog review phase for a publication could have benefits; to a greater or lesser extent we all work within the bunkers of our knowledge and experience and blogland has an amazing power to very rapidly and effectively bring new aspects and skills to a debate.
Maybe one to take to unthreaded?
There’s nothing wrong with a competent blog review, curious, but let’s note that Steve McIntyre’s blog is not a run-of-the-mill site. It’s top 0.01%. Whatever mechanism evolves for web publication, I’m all for a crank-science filter very early on.
SM must be asleep at the switch here, not red-carding for OT. SO: I am a paladin of markets, of the marketplace, including the Marketplace of Ideas and the Marketplace of Information. And we know that the famous “invisible hand of the marketplace” will–and perhaps in very short order–weed out garbage (for example, just as it weeded the Yugo automobile quickly out of the marketplace.) If there is no “peer review”, no apparent fire-wall, wouldn’t you yourself be extra-careful to vet and firm up your own submission? Your reputation will be riding on it. If it gets out that you write/wrote “garbage”, no one visiting the market is even going to read anything else from you. Although, if we have to have peer review (restricted access to the marketplace), peer review could be improved in various ways, such as a statement by each reviewer, included in the publication, describing exactly what that reviewer did and did not do in his/her reviewing. (Remember SM’s now-famous statement that when he, as a reviewer, once asked for data and code, the editor said that that was the first time in 20–or was it 30?–years that anyone reviewing had ever made such a request?!) But it’s still better to have free markets, better ultimately for everyone involved, the “sellers” as well as the potential “buyers”. Then each “seller” will be legitimately forced to provide high-quality product: no “peer review” to provide some real or phony aura of imprimatur.
SC,
Re: “If there is no “peer review”, no apparent fire-wall, wouldn’t you yourself be extra-careful to vet and firm up your own submission? Your reputation will be riding on it.”
That’s one of the things many in the scientific community like about peer review: if it’s published it must be true! Hence a strong reluctance to consider other options.
I agree, that “if we have to have peer review … [the process] could be improved in various ways.” I think that reviewers of published articles should be named, thereby giving reviewers greater responsibility for their decisions or recommendations. It would also be useful to allow reviewers to append a comment on the published work. This would encourage serious effort at constructive comment.
I don’t think reviewers need to be named. I think that would stifle dissent too much. I do think that Editors should require rigorous and substantive reviews, not simple handwaving or saying the paper is “bizarre”.
And comments on papers need to have a reasonable limit on size. We seem to see often enough that a coherent comment isn’t possible with the restrictions that are imposed in many journals.
I have seen this done in some engineering journals. In one case, it was remarkable to see that quite a famous reviewer had misunderstood the purpose of the technique being describd.
Seeing as this section is nicely off topic now – I’ll note that one of Muir Russell’s stated aims in his own ‘independent inquiry’ was to address/salvage the reputation of peer review in general, e.g. “MR – queried whether there are any persons who might support/explain the robustness of the peer review process. Eg Editor of Nature. Nature publish rebuttals and criticisms. Agreed this would be a good idea. Seems there is an attack on peer review which it would be helpful to address.” (thanks to Lisa Williams for the excellent notes – seems that Lisa Williams of the UEA worked on both these “independent” inquiries). This quote is from the famous and cordial Phil Jones interview of 12-Dec-2009, which lasted a perfunctory 40 minutes:
Click to access MR%2018%20Dec%20final%20P%20Jones.pdf
…that should be 18-Dec-2009, sorry.
Hmmm, if the head of the inquiry doesn’t read blogs, including skeptical blogs
which covered Climategate, may read newspapers which did a poor job covering it until late and had 11 hand-picked reports which include no critiques of the HS, I wonder if he read any of the Climategate emails? On a positive note,
I understand the Garden Restaurant at the Sainsbury Center at UEA does a
killer chicken masala.
“I am United States Senator John Yerkes Iselin, and I have here a list of two hundred seven persons who are known by the Secretary of Defense as being members of the Communist Party!”
Mrs. Iselin: [at meal time] I’m sorry, hon’. Would it really make it easier for you if we settled on just one number?
Sen. John Yerkes Iselin: Yeah. Just one, real, simple number that’d be easy for me to remember.
[Mrs. Iselin watches her husband thump a bottle of Heinz Tomato Ketchup onto his plate]
As happens so often when things are confused, “the trick” is to come up with the right question. And, in this case, to find out to WHOM to ask that question.
Nice going, Steve, Mr Bloodhound.
No, it isn’t over yet – but you’ve got the scent!
“These key publications have been selected because of their pertinence to the specific criticisms which have been levelled against CRU’s research findings as a result of the theft of emails.”
I am sure it has been said before, but can someone please provide the proper list of publications that actually meet the criteria, as described above. “Pertinence to the specific criticisms”. Doesn’t have to be eleven, just pertinent to the criticisms.
Whatever the list, it should be defined by the protagonist and continually brought to the attention of the relevant parties. They looked at this list, when they clearly should have looked at …….. what is this list?
The proper list is anything on paleoclimate, plus the work done by CRU for IPCC and WHO. The latter were excluded out of hand by the panels.
From CA, April 15th 2010, A “Fair Sample”? :
“I understand that the list of 11 papers for the Oxburgh review was collated by Prof Trevor Davies, in consultation with others.”
This is an interesting choice of words. ‘I understand that’ is not normally used when someone means ‘I know from first-hand experience’, but rather ‘I have investigated and come to the following conclusion’, or some such. Am I just splitting stylistic hairs?
Because if you say “I am *told* that …” then the next question is “and who is the teller?” Substitute “understand” for “told” and the teller disappears.
The most revealing word in Trevor Davies statement is “pertinence” in describing the “papers” in a context of “criticism” brought be the “emails”.
One online dictionary definition has it defined as meaning “pertaining or relating directly and significantly to the matter at hand”. By selecting that word he states unequivocally that he knows the specific criticism(s), and the subject of the papers.
Yet we know the major criticism(s) have to do with the “Hockey Stick” papers, and that the selected list did not include one such paper.
Other than buffoonery or that he lied outright in his word selection, it leaves Davies use of the word “pertinence” in a counter intuitive context. If the papers selected had in fact contained the Hockey Stick papers, the team could not expect the “right” outcome. So the pertinence of the papers selected only have to do with the outcome they produce by the Ox Panel in relation to the “criticism” brought by the “emails”.
In using the word pertinence, he did not lie so long as one accepts that the direct relationship the papers had to the criticism is in producing the Ox Panel outcome. Thus he must admitted as such, or claim buffoonery – lest we conclude he was lying outright.
So his use of the word pertinence reveals either his utter buffoonery, a simple individual yet brazen lie, or that he admits his participation in a conspiracy to influence the Ox Panel’s outcome with falsehood.
Logic provides no alternative.
Steve often talks about peas and thimbles, and distraction techniques. One og the AGW-ers neatest distraction techniques is to go on about peer review, planting the false distraction about peer-review = good (and incidentally claiming that E&E, which published the M&M paper which got the ball really rolling, isn’t peer reviewed when they know full well it is). Peer-review, outside of medicine for obvious reasons, is in fact a very low threshold, reviewers only ask pretty basic questions like, does the argument hold together, have most recent work been overlooked, does it add anything to what we know; and then on to second order questions like, whats the point of table 4 in relation to the text. In short, reviewers are not re-doing an authors work, and their recommending it for publication does very definitely not mean they are confirming the work is right or good in any way. Mainly they are saying, is it worth chucking this message in a bottle into the sea to see what will become of it? Because of course, real review takes place post publication, when people try to replicate work that makes exceptional claims. And isn’t there some chap at Penn state who’s work, peer reviewed and all that, seems a bit hard to replicate?
Oxburgh’s language does not inspire public confidence. Where we are entitled to expect professional addressing of specific issues, we are given imprecision, hesitancy and obfuscation such as : “came by…I believe….we feel…we all felt…I suspect that…. no indication that….I don’t think that can be true,”
Why eleven papers?
Is there a problem with the site or is Steve just taking some well deserved down time? This is the last post I can see and only very sparse comments over last 6 days.
Unbelievable. You have absolutely uncovered this matter, after much waiting. Lisa Williams seems oblivious to the fact of what she’s uncovered. Good show, old boy.
6 Trackbacks
[…] “Inquiry”: Defendants decide on admissibility of evidence By ktwop Steve McIntyre is upto his admirable sleuthing […]
[…] by Jeff Id on September 16, 2010 Steve McIntyre got an answer as to who chose the papers to review for the Oxburgh report – Turns out it was Trevor […]
[…] […]
[…] då man påstått att en del av det granskade materialet varit rekommenderat av Royal Society. Se här. McIntyre har i tidigare inlägg också hudflängt Muir Russels rapport, se t.ex. […]
[…] Who Chose the Eleven? An Answer The Oxburgh Report stated: The eleven representative publications that the Panel considered in detail are listed in […] […]
[…] SOURCE […]