Andrew Montford has succeeded in prying some important documents from the Oxburgh “inquiry”. These raise several important issues.
The attachments here include Michael Kelly’s notes – see page 81 on.
These offer a few glimpses of sanity that were suppressed by Oxburgh in the “report”.
Here is an interesting comment about IPCC (leaving aside, for now, the lack of “humility” in Jones’ exchanges with Mann):
Up to and throughout this exercise, I have remained puzzled how the real humility of the scientists in this area, as evident in their papers, including all these here, and the talks I have heard them give, is morphed into statements of confidence at the 95% level for public consumption through the IPCC process. This does not happen in other subjects of equal importance to humanity, e.g. energy futures or environmental degradation or resource depletion. I can only think it is the ‘authority’ appropriated by the IPCC itself that is the root
cause.
Good question. How does this “morphing” take place, especially when the scientists in question act as Lead Authors and Coordinating Lead Authors of IPCC. Kelly continues:
(4) Our review takes place in a very febrile atmosphere. If we give a clean bill of health to what we regard as sound science without qualifying that very narrowly, we will be on the receiving end of justifiable criticism for exonerating what many people see as indefensible behaviour. Three of the five MIT scientists who commented in the week before Copenhagen on the leaked emails, (see http://mitworld.mit.edu!video/730) thought that they saw prima facie evidence of unprofessional activity.
“Receiving end of justifiable criticism”. I presume that Kelly is staying pretty quiet these days.
Kelly previously made a complaint that would not be opposed by the severest IPCC critic:
(i) I take real exception to having simulation runs described as experiments (without at least the qualification of ‘computer’ experiments). It does a disservice to centuries of real experimentation and allows simulations output to be considered as real data. This last is a very serious matter, as it can lead to the idea that real ‘real data’ might be wrong simply because it disagrees with the models! That is turning centuries of science on its head.
and
(ii) I think it is easy to see how peer review within tight networks can allow new orthodoxies to appear and get established that would not happen if papers were wrtten for and peer reviewed by a wider audience. I have seen it happen elsewhere. This finding may indeed be an important outcome of the present review.
It would have been an “important outcome of the present review” had this finding appeared in the Oxburgh “report”.
Or here;
My overriding impression that this is a continuing and valiant attempt via a variety of statistical methods to find possible signals in very noisy and patchy data when several confounding factors may be at play in varying ways throughout the data. It would take an
expert in statistics to comment on the appropriateness of the various techniques as they are used. The descriptions are couched within an internal language of dendrochronology, and require some patience to try and understand.
I find no evidence of blatant mal-practice. That is not to say that, working within the current paradigm, choices of data and analysis approach might be made in order to strain to get more out of the data than a dispassionate analysis might permit.
The line between positive conclusions and the null hypothesis is very fine in my book.
I worry about the sheer range and the ad hoc/subjective nature of all the adjustments, homogenisations etc of the raw data from different places
Climate Audit 
60 Comments
Michael Kelly’s comments are absolutely spot on.
Judith, you are going to get another kicking for saying that but bless you for standing up. Truth will out at the end.
Well said, Professor Michael Kelly. He makes some cogent remarks about the process under review. I will look with some interest for the replies and retorts he receives for these comments.
Of course, the comments could be like: Professor Kelly sees no malfeasance and completely clears the people and process of any wrongdoing, and disregard, in the process, the more critical remarks – it has been done before.
While the world is waking up to the unjustifiable hiding of inconvenient data, the rearguard of Deep Climate and Arthur Smith try to prove that a guy named “Steven Mosher” is wrong about the far more trivial issue of what methods went into the part of the data that were NOT hidden. Lame.
Art, Deep: hiding inconvenient data is bad. Your auditing is being duly applaued. But your take-down is truly mis-directed. Let’s have your take on Dr. Kelly’s remarks. Right here. Right now.
Arthur Smith plays climate auditor, can’t fully replicate methods (forced to guess – like all of us – at unspecified parameters), yet concludes S. Mosher is “full of it”:
http://arthur.shumwaysmith.com/life/content/steven_mosher_even_fuller_of_it
If a guy who makes one mistake is “full of it”, what would that say about, say, IPCC, using Michael Kelly’s assessment as a template?
Bender, I think Arthur’s auditing should be welcome on a site labeled climateaudit.
Steve: None of Arthur’s criticisms of CA are valid. I don’t have time to pick every spitball off the wall right now but will try to do so in time.
MikeN, I think Arthur should do more of the auditing and drop the urgency of his ad hominem. All I’ve seen Arthur achieve, in his voracious efforts to “take down” Mosher and Fuller, is further condemnation of Mann et al’s secrecy and lack of clarity with regard to the “stick”.
I posted, about six hours ago on the page of Smith’s that Bender linked, a response to the following comment by Smith from yesterday:
———-
But the bottom line is this: I can’t see how the claim that Mann deleted the post-1960 portion of the Briffa series and then “padded” it with post-1960 instrumental values can possibly hold. And by the way, McIntyre has also made that very claim.
———-
My comment was reported to be “queued for moderation by site administrators and will be published after approval.”
Six hours later, and nothing.
Unfortunately, I did not remember to copy this particular comment, so I have lost the exact text. But the main point of it was, I had looked at the Briffa 2000 paper, and copied Figures 1 and 5 into a graphic program (MS Paint, sorry, it’s all I have), and compressed the time axis of Fig. 5 to match that of Fig 1, and compressed the y-axis just a little to match up the 2-sigma-range in each direction, and before compressing I had highlighted the smoothed trendline of the tree-ring average on Fig. 5 so that when I compressed it, it would still be distinguishable. And when you line up the two time series with each other, it is painfully obvious that Fig. 1 used instrument temperature data in place of tree ring data.
I asked Smith if he could not see this. But since he won’t allow my comment to post, he has an excuse for not answering.
I guess the central point I was driving at was, if the Briffa 2000 paper contained a series with grafted instrument data, and Mann just copied that, then Smith could be technically correct that Mann did not delete “the post-1960 portion of the Briffa series and then” pad it with post-1960 instrumental values. Fine, Smith, but even if you’re right, how could anyone argue with a straight face, after climategate, that Mann should be given the benefit of doubt as to whether he knew what he was copying? Especially with _both_ the real _and_ the false data available in the same paper, within a few pages of each other?
And Smith’s argument still amounts to little more than saying that the Briffa 2000 data doesn’t go down — an assertion that can only be true if you define “the Briffa 2000 data” to mean the data plotted in Fig. 1, i.e., the data that are truncated at 1960 and replaced with temperature data!
So … whoop-de-doo. The “Briffa 2000 data” don’t go down. (Not!) But what have you accomplished, Smith!? You’re openly defending the use, in a number of spaghetti charts, of two series THAT BOTH CONTAIN THE SAME TEMPERATURE RECORD, IN THE SAME TIME RANGE!
Charts in which both of those two series are factored (in one way or another) into various assessments of overall temperature trends — and in which the real, declining tree-ring data from Briffa (which represents, not just new data from Briffa, but all of the northern boreal forest series that he selected for inclusion in his hemispheric average).
How can any honest auditor possibly think that such a spaghetti chart is in any way valid, either scientifically or for use in policy-making?
RTF
Correction:
Charts in which both of those two series are factored (in one way or another) into various assessments of overall temperature trends — and in which the real, declining tree-ring data from Briffa (which represents, not just new data from Briffa, but all of the northern boreal forest series that he selected for inclusion in his hemispheric average) are excluded.
RTF
He was looking at AR4, for which Briffa is the editor. I started to review it, but after seeing the figure in question, I find it highly unlikely that they tried to hide a decline on that chart. It makes no difference. It looks like a different visual trick entirely.
I can also see no thread on ClimateAudit that contradicts his post, which focused on AR4.
Yes, what I’m trying to say (have been on a couple of threds, actually) is that if data were copied (to AR4, to TAR, to WMO, to _wherever_, it simply doesn’t matter) FROM — pay attention now — FROM, Briffa 2000, Fig. 1, and if Briffa 2000, Fig. 1, contains an instrument graft or splice (which is an idea I originally got by following the subtext of Steve’s comments on the “desmogging” posts), then that means Briffa was the original source of the false data. This is confirmed by comparing Fig. 5 with Fig. 1 in Briffa 2000.
Does that clear up anything for you?
Further, I am not saying that McIntyre has explicitly written this anywhere (though I certainly don’t rule it out). I’m just saying that the proof is right there in Briffa 2000 by comparing the two figures. It doesn’t take any scripting or any blown-up screen shots. All you have to do is compress Fig. 5 so that it can be compared on the same scale with Fig. 1. And it’s totally obvious. A high-school student could see it, for God’s sake.
This is all so simple. The other issues, while not unimportant and not unworthy of discussion, are surely secondary to this.
RTF
According to the B2000 Fig captions, Fig 1 uses both temperature recons and ring width series. Whereas Fig 5 is based on mxd density chronologies. So you are effectively comparing apples and oranges. The two series shouldn’t match exactly – and moreover the modern divergence that you point to is precisely what concerned Briffa, motivated him to suggest not using this proxy after 1960.
Bender,
I don’t know what verbiage you’re looking at. What I see in the Fig. 1 caption is,
———-
The bottom curve is the average of the other data sets after rescaling to give equal mean and variance (over the common period 1601-1974), also plotted as 50-year smoothed values.
———-
RTF
Sounds like apples-to-apples to me. Other than the fact that you have said that Fig 5 was old data, which I did not understand. Still, adding in the new Briffa data obviously could not cause the change we’re looking at, so it’s still pretty close to apples-to-apples, in my view.
RTF
How can any honest auditor…
That really is your answer isn’t it? An HONEST auditor couldn’t.
On Fig. 1, for the period 1950-end, all of the series except one of the Yamals go down, not up.
In your words:
———-
The uptick at the end of series h in Fig 1 appears to be attributible solely to none other than Yamal’s “stay thirsty” tree.
———-
RTF
What are you – a lawyer? So now I need to rephrase to avoid a lawsuit?
This is a numbers game and text only takes you so far. So don’t take my text and suppose there’s more precision in it than the language itself will allow.
I rephrase: The *degree* of uptick appears to be heavily dependent on one tree.
The only way to know for sure is to recreate the series without that tree.
Mr. McIntyre,
What just happened?
Some of my comments have just disappeared after Bender responded to them, and some of the rest have been moved to the bottom of the page.
Is there some way of reverting the conversation back to the way it was?
RTF
Steve: Chill. the comments were OT for this thread and moved to the new thread on Arthur Smith. Nothing was deleted. I used to be able to move comments easily, but can’t in the present setup.
Prof. Kelly’s comments are well worth reading in full, eg. “Your critique of the paper by McLean, Freitas and Carter (2009) hinges on arcane aspects of statistical analysis, and they stand by their comments. I have recommended publication of data with a controversial explanation precisely to get the debate going. In other areas of science the best winds out by attrition: why not here?”
I am hopeful, thinking that if reviewers are removed enough from the field there will eventually be a positive outcome.
I think “winds out” may be a typo for ” wins out”. And that has been my sentiment about this controversy for some time. Why not encourage it to play out in the journals.
mikep, seriously? You don’t know why? It’s called gatekeeping. The climategate emails were full of examples of CRU getting worthy papers tossed. If it was an even playing field, then it would play out in the journals. But the internet can defeat gatekeeping and is currently winning. Stay tuned.
Ron, I agree. the point I was trying to make was that in normal scientific controversy this is what I would expect to happen, but ,as you say, it has been deliberately stopped from happening in this field. I think part of the problem must be the small size of the field, with relatively few practitioners.
As Judith notes, Kelly’s comments seem to capture what many here have been saying and arguing for years. Now that his comments are public, will he formally restate them or will he walk away from them?
http://www.eng.cam.ac.uk/~mjk1/
Mike Kelly’s question to Keith Briffa, given his comments on Briffa’s papers, speaks volumes as to what he thinks about the science of dendroclimatology:
“(7) Given that the outputs of your work are being used to promote the largest revolution mankind has ever contemplated, do you have any sense of the extent to which the quality control and rigour of approach must be of the highest standards in clear expectation of deep scrutiny?”
After reading Prof Kelly’s comments it is hard for me to see how he could have signed off on the report as it is written.
There are some interesting remarks from Oxburgh in his letter to the team on 15 March:
“There is clearly a high level of stress that has been generated by aggressive and abusive blogs…”
– I wonder which blogs this refers to?
Also interesting to see David Hand asking what the ‘specific criticisms’ were of the CRU papers. He is fobbed off by a suggestion to look at the CRU HoC submission, which he meekly accepts as an answer.
And there’s an email to Hand mentioning a Mcintyre criticism of Briffa 95.
A certain m Mann seems very keen to talk to Hand.
Also Oxburgh claims, after the report is published, not to know how the papers were selected.
Lots of interesting stuff for Steve to chew over here…
1. This is Kelly’s review of the less controversial papers!
2. Striking assessment of the significance of dendrochronology is revealed in the question he recommends is asked of Briffa and Jones: “Given that the outputs of your work are being used to promote the largest revolution mankind has ever contemplated, . . .”
We’ve found an honest man.
Shame he’s now doing his best to hide from the rays of Diogenes’ lamp.
Joliffe would have been handy on that panel.
Well done to Kelly for raising salient points.
Shame on the “Inquiry” for blatently ignoring them!
Crikey
It is too bad that no records were kept of Briffa & Jones’ answers to Prof. Kelly’s questions.
Prof. Kelly:
US CCSP Report 2006, p. 11:
Lol, i’d have loved to see what someone like Feynman would have thought of this.
“But either way there’s Nature and she’s going to come out the way She is. So therefore when we go to investigate we shouldn’t predecide what it is we’re looking for only to find out more about it.”
“It doesn’t matter how beautiful your guess is or how smart you are or what your name is. If it disagrees with experiment, it’s wrong. That’s all there is to it.”
A beguiling statement, but it’s never been true.
An interesting example, anon, but it does not support your statement. I’d say it probably does the exact opposite.
Ain’t it beautiful. Another good find, Ross.
LOL! So they really do think that when measured data disagrees with their computer models, that it is the measured data that has to be wrong. God help us!
Yeah, this has been a source of amusement and outrage to me for some time now. I’d noticed the same quote Ross pointed out. And then NPR ran this headline: “The Mystery of Global Warming’s Missing Heat”
http://www.npr.org/templates/story/story.php?storyId=88520025
Confirmation bias run amok.
Is this the same person who wrote the hide the decline e-mail?
“An elegant theory which does not fit good experimental data is a bad theory. Here the starting data is patchy and noisy, and the choices made are in part aesthetic, or designed to help a conclusion rather than neutral.”
Thank goodness somebody in the field understands the difference between an experiment and a model run. When I was reading Schneider’s ‘Science as a Contact Sport,’ he continually referred to his computer games as experiments, and I got the impression that he actually considered that they were experiments.
‘Science as a Contact Sport’ was directed at a general readership, and you’d think that, if Schneider did understand the difference, he’d have been careful to explain it.
Steve: Kelly is from outside the field.
Prof. Kelly’s remark “….. allows simulations output to be considered as real data” is most insightful. When did we cross the threshold into believing such simulations are “real world”? How did it happen?
“I have remained puzzled how the real humility of the scientists in this area, as evident in their papers, including all these here, and the talks I have heard them give, is morphed into statements of confidence at the 95% level for public consumption through the IPCC process.”
We should be welcoming of scientists trapped on the other side of the orthodoxy.
Where did the humility go? Humility is displayed when authors put their names to their own papers. Their reputations at risk.
However, when somebody is writing anonymous guff for the IPCC, there is no need for balance or for being careful.
The recent comment “Thank goodness somebody in the field understands the difference between an experiment and a model run” was too optimistic.
Michael J Kelly is a professor of technology at Cambridge. His background is physics (electronics in particular). So we are still not sure if there is anyone in the field of climate science who understands the difference between an experiment and a computer model run.
If their writings are to be taken as evidence, writings including emails, I believe that all climate scientists confuse model runs and empirical data. The confusion is plain to see in their language. There is no such thing as “the output of a model run.” Once a computer run has finished, or has been stopped by its managers, someone will select some numbers and call them the output. But there is no rational basis for selecting one set of numbers rather than another as “the output.” To call something “the output” is to promote confusion with the genuine empirical output of an experiment. The proper way of speaking would be to say something along the lines of “our latest computer run gave us some numbers that can be associated with empirical phenomenon X and many numbers that cannot be associated with anything.” Of course, there is nothing in the association that makes it unique. It is all based on best hunches and amounts to pre-science.
Hasn’t Michael Mann publicly stated that ‘nobody spliced the surface temperature data onto the proxy record.’
Then why does he tell Dr. Hand [twice in one e-mail]:
“That the “blade” of the hockey stick is determined entirely by the CRU instrumental temperature
record and in no way could possibly be influenced by technical issues involving conventions used in
PCA to summarize proxy records.”
[See page 106 of Hand’s e-mails, Mann to Hand dated 4-15-10]
When Hand asks:
“In the von Storch and Zorita paper, Fig 1 legend says the 1900-1980 centering is given in red, but the label at the top of the figure indicates that the 1000-1980 is in red. Do we know for sure that the red one really is 1000-1980?”
[page 108]
Mann replies:
“thanks for bringing this to my attention. I had not noticed this before, and honestly I don’t actually know which is which-would need to contact von Storch.” [then recommends W&A paper]
[page 109]
I believe that the quote was “no one in this field has ever spliced…”. And that always makes me wonder what ‘field’ he is talking about. If they can claim that at the time of that statement they were talking about some field other than dendroclimatology, the statement is true, however intentionally misleading it otherwise is.
@Benjamin – perhaps Feynman might have extended thus (CAPS):
“It doesn’t matter how beautiful your guess is or how smart you are or what your name is – OR HOW MANY OF YOU THERE ARE. If it disagrees with experiment, it’s wrong. That’s all there is to it.”
Mr. Kelly is a real scientist, the kind of scientist who should be peer reviewing journal articles. His statement,
My overall sympathy is with Ernest Rutherford: “If your experiment needs statistics, you
ought to have done a better experiment.”
is spot on.
In the end, he appears to show a level of sympathy for the CRU team and doesn’t want to completely throw them under the bus, at least publicly.
It is a shame that it takes FOI to publish the real review. Thank you Mr. Montford for bringing this to the public.
Why do you think page 80 starts at paragraph 6? Just missed a page in the copying or what?
Wow, what many of us have been saying for years.
Kelley’s notes have just become the “inquiry”.
Thank you Andrew Montford for releasing these rays of sunshine. Prof. Kelly’s comments lay out in stark relief the differences in the quality of science expected in climate science vs. traditional experimental science. I especially identify with his taking offense at climate scientists equating computer runs with experiments and their outputs with actual data. Hear, hear! I only hope that Kelly doesn’t somehow end up on some academy blacklist as a result of his truthful and devastating comments being thus liberated.
Prof Kelly is an engineer (electronics), engineers work in a world where a likelihood of something being correct with a 95% probability means it needs more work. In the review of of one of Jones’ paper he asks:
“In neither of these papers is there any overt malpractice, but one can’t eliminate the possibility of conscious or unconscious bias in the choices of data. I just do wonder if a different hypothesis was being tested whether the same approach could give a very different answer.”
Are the data infinitely flexible?
To be fair he was given the task of finding if there was any malpractice at the UEA. He didn’t say he found none, but implies in several places that there could have been but it’s unprovable. He talks throughout of conscious or unconscious bias in the selection of data. It’s a good report and should be read by all those who think that Oxburgh is the final say on whether there has been any malfeasance at the UEA/CRU.
Steve: in the original announcement, Oxburgh wds supposed to reappraise the “science”. It was a bait-and-switch.
Very good piece by Prof. Kelly. What I particularly like is the lack of polemics, that there is no axe to grind, just a simple, clear, objective assessment of the palaeoclimate studies.
I think these remarks will chime with many people who read CA who work in engineering or the hard science sectors. Those that do tend to have their work demonstrated in the field, where the results will be unequivocal: they are either wrong or right. And when your work is subjected to the scrutiny of the real world, you have to learn an awful lot of humility very quickly.
The lack of an objective criteria for selection of data and methods in the palaeoclimate reconstructions sticks out like a sore thumb and leaves huge potential for subconcious experimenter bias. No need to invoke fr**d or malfeasence: bias is enough. Professor Kelly sees this and reports it as it is. The only question I have is how can a panel which includes a member who produces such a thoughtful and insightful piece of analysis, end up producing the fig leaf of a report that we saw?
Who or what convinced Kelly to allow his views to be unrepresented in the inquiry report that bears his name?
I am guessing that there must have been an impressive fixer operating behind the scenes.
All the series go up from ~1870 to ~1930. After that they vary, but general drop down (though not so much as they rose). Net effect is an overall 20th c. rise. Key point: It is the uptick from ~1960 onward in (h) that appears to be extremely dependent on the anomalous Yamal (f).
For this reason I have no reason to buy into your “instrumental padding” hypothesis for Fig 1. Prove me wrong.
The approaches taken by these computer based scientists remind me of my former life as a macro-economist.
Economics considers itself a science (social science that is). At its heart is teh analysis of human behaviour, but it is then wrapped in statistical data collections and mathematical analysis. From this we create macro “models” of the economy. Media etc chomp at the bit for the forecasts (unemployment, GDP, inflation, interest rates and exchange rates etc) that come from the mouths of these all knowing economists. In the background the models are “recalibrated”, dummy variables are added etc in an attempt to make the modelled output match reality.
Surprisingly enough humans and the global economy are a little too complicated for these forecasts to be particularly accurate (although you will never see a confidence interval reported).
It still amazes me that people (mainly politicains and investors) put some much weight on the comments of economists. It amazes me even more that people beleive the modelled outcomes of the global environment given the relative scarcity of true data and the inherent errors in it collection, let alone the obvious complexity of the environment as a whole!
The bottom two autmamtically geneated posts after the article refer to very different Kellys.
in the above story ….
“Three of the five MIT scientists who commented in the week before Copenhagen on the leaked emails, (see http://mitworld.mit.edu!video/730)”
That Video URL has changed and now is here ?
http://video.mit.edu/watch/the-great-climategate-debate-9529/
One Trackback
[…] SOURCE […]