Comments on: Santer et al 2008

By: Skiphil

Skiphil — Mon, 03 Dec 2012 22:28:22 +0000

See post at blog Niche Modeling for account of how Ben Santer has grudgingly begun to acknowledge some reality, while failing to cite prior contributions of critical authors from CA etc.:

"I suppose that now faced with a disparity between models and observations that can no longer be ignored, [Santer] has had to face the inevitable. That’s hardly a classy act. Remember Douglass, McKitrick, McIntyre and other climate realists reported the significant model/observation disparity in the peer-reviewed literature first. You won’t see them in Santer’s list of citations...." Santer: Climate Models are Exaggerating Warming – We Don’t Know Why

By: Peer review of scientific work - another example of a flawed basis for public policy « Fabius Maximus

Thu, 22 Jan 2009 01:09:20 +0000

[...] project of auditing climate research papers, Steve McIntryre wrote a measured analysis: “Santer et al 2008“, 16 October 2008. He then sent the following request (see here for full details of this [...]

By: SD or SE: What the heck are ‘beaker’ and the other talking about? | The Blackboard

Fri, 24 Oct 2008 17:59:16 +0000

[...] Climate Audit? It’s discussed in several threads related to “Santer17″. (See 1, 2 etc.) Are you wondering what it all [...]

By: MC

MC — Thu, 23 Oct 2008 12:21:31 +0000

Re: Carrick (#230),
I agree. I have read both papers and have a number of issues:
1) The use of autocorrelation to the real world signal is incorrect. The real world signal clearly has structure, how much will be determined by averaging measured data. The use of autocorrelation artifically inflates ‘confidence intervals’ (Santer et al term) and it means you are imposing a linear trend on the data which is not correct without some stated assumptions.
2) The fact that models cannot match ENSO variations even to some degree says to me the models are not good enough. Using Santer’s increased CIs (which appears reasonable expect no sqrt(N-1) demoninator in eqn 9) the paper even shows how you can get an anomaly of around 0.5 ± 0.5 (i.e. 100% error). This paper should have been a critique on efficacy of models.
3) Douglas et al does not average out the ENSO oscillations (some only have single runs) so he should be using autocorrelation for consistency (even though this is a wrong method to use)
4) The statement that y0 = phi0 + noise in the real world is incorrect (see 2). You CANNOT assume that ENSO is random. This is what is measured. THEORY MUST FIT FACTS
5) In general the skill of a model is how well it predicts more than one property. For example in plasma modelling using Particle-In-Cell (PIC) codes which are not dissimilar to GCMs, if you get a good agreement with electron temperature but number density and plasma potential are way out the model is declared to have a problem. Why? Because enough is known about plasma interactions that you have an idea of the parameter set and bounds to be used.
All these papers show is that it appears that people in the Climate community are trying to make facts fit their theories. They have already settled on a ‘consensus’ about the assumptions and the results. Douglas et al and Santer et al are not addressing the fundamental assumptions and methods about how much skill models have in representing measured data and how much parameter variation is needed to achieve this. This is much deeper problem

By: Carrick

Carrick — Thu, 23 Oct 2008 02:31:22 +0000

Craig, that’s a good analogy.

If none of the bids meet the specs, you reject them and send a letter explaining how all of the bids failed in meeting the specs to all bidders. You don’t go through some contorted logic to explain to your boss that some aggregate of the various bids aren’t that far away from the original specifications. (Try this and see whether it works.)

In this case, the clear fact is the particular set of physical assumptions made by any of the known models do not reproduce the observed experimental data. But it seems completely obvious that some choice of physical assumptions would be able to do so—one hardly needs to perform a jimmied-up statistical analysis to come to that conclusion!

What is it that that Sanders is really saying? That in spite of the fact that none of the models can reproduce observations, they are so all-over-the-place that somehow in ensemble they become consistent?

That deserves a hearty “what the ______”.

I’m not sure these guys know how to apply the scientific process at all, if this is an example of their “finer work”.

By: Alan Wilkinson

Alan Wilkinson — Wed, 22 Oct 2008 22:57:52 +0000

Re: Steve McIntyre (#224),

Regarding the relative accuracy and consistency of the RSS and UAH datasets, I presume you are aware of Douglass and Christy’s discussion and conclusion on the matter (in favour of UAH) here (Appendix A):

http://arxiv.org/ftp/arxiv/papers/0809/0809.0581.pdf

By: Craig Loehle

Craig Loehle — Wed, 22 Oct 2008 21:10:06 +0000

Re: Carrick (#221), In support of Carrick's point, we can consider the set of models as candidate companies bidding for a technical job. The question is, do they meet the specs? Testing one by one, they all fail (or not). We don't say that the variance of the bidders is such that we can not reject the possiblity that a bidder exists who would pass. But that seems to be what Beaker wants.

By: DeWitt Payne

DeWitt Payne — Wed, 22 Oct 2008 19:13:34 +0000

Re: Kenneth Fritsch (#223),

I think we are starting to repeat ourselves...

Yes, but for some reason it didn't sink in until recently. Sometimes repetition is necessary. Sometimes you have to hit the mule with a two by four more than once to get his attention.

By: Carrick

Carrick — Wed, 22 Oct 2008 14:05:46 +0000

Kenneth Fritsch:

I did post this figure early in the thread at the link below. I think my rendition is easier to read.

Sorry… I managed to miss that one.

I still think the bigger issue is whether an ensemble of deterministic model outputs, varying only due to slightly different physics assumptions, should ever be treated as if they were normally distributed.

I would say the correct answer is they should not.

By: Alan Wilkinson

Alan Wilkinson — Wed, 22 Oct 2008 06:15:50 +0000

Re: Wolfgang Flamme (#185), Sorry I'm very late replying to this, but I have seen no good reason to believe that the n samples from the parent model distribution are random or unbiased and therefore there is no sound basis for computing any kind of confidence measures from this sampling of model outputs.