Comments on: Bürger and Cubasch Discussion

By: TCO

TCO — Sat, 23 Sep 2006 16:09:04 +0000

Scoreboard.

By: bender

bender — Sat, 23 Sep 2006 15:46:21 +0000

TCO is ALWAYS right. Argument against him is futile.

By: TCO

TCO — Sat, 23 Sep 2006 15:41:26 +0000

I agree with the rejection. The editors validated my point of view. TCO was right. Bender was wrong.

By: Jo Calder

Jo Calder — Fri, 22 Sep 2006 22:29:00 +0000

The C&B paper was rejected, but the editorial comment is worth a look.

By: John Creighton

John Creighton — Fri, 08 Sep 2006 23:37:12 +0000

#137 Bender Bayesian statistics are nothing but a way to incorporate prior information into an estimate. If you assume aprori that a curve is a hockey stick then to get otherwise, you need sufficient information (inverse of the covariance matrix) to over come this biased assumption. Of course if you’re initial assumption is no correlation between the proxies and data, then the aprori information provides a useful method to eliminates the over determined aspect of some multi proxi studies. The aprori information can come from either expert knowledge or by a recursive least squares algorithm. On faulty method I could see the hockey stick team trying is using previous studies as aprori information and incorporating a second study as new and supposedly independent information.

Anyway, I think the best use of Bayesian statistics in climate modeling is for the initialization of global climate models. I previously suggested a maximum entropy assumption as the aprori information for initialization global climate model.

By: TCO

TCO — Fri, 08 Sep 2006 23:20:33 +0000

I’m honestly curious.

By: TCO

TCO — Fri, 08 Sep 2006 23:19:43 +0000

Maybe you did or didn’t. When I raised the issue, it was because it was something I wanted to discuss or consider. I don’t remember your making that response when I raised the issue. I think you just blew me off. And I still (naively, openly, wonderingly) wonder if there is any benefit to a model created by division of the overall calibration set into “calibration” and “verification”. Would one be just as well off using the entire period and aikake criteria or the like.

By: bender

bender — Fri, 08 Sep 2006 23:03:39 +0000

Of course true out-of-sample tests are better. The only reason they’re not often done is because they’re typically prohibitively expensive. I’m sure I said that already.

By: TCO

TCO — Fri, 08 Sep 2006 22:47:28 +0000

And yes, bender you did note the central point on the jacknifing well.

By: TCO

TCO — Fri, 08 Sep 2006 22:46:46 +0000

When I finally broke down and read through the poor explication, I found some interesting things in there (basically their point as elucidated in the lessons learned). I think though:

-overly broad in claims of applicability to the field and to MWP (analysis not sufficient or even geared to the overall claims)
-their sucess with early flavor paper went to their head and they bogged the paper down by having the flavor stuff so prominent when the real story in this case was NOT susceptability of recons to alternate methodologies but the basic jackknifing in and of itself on specifically MBH and regEM papers RE/CE.
-some vague and unsupported claims about degrees of freedom, within population etc.

On the technical side, I’m still very puzzled about the validity or importance of splitting the sample and doing a “verification”. That “verification” is still really a part of the overall calibration of the model. I think the true verification comes from prospective (truly out of sample) tests. That’s what the econometrics purists say. In addition, is a model that is more regular to jackknifing really any better then one selected by using the entire instrument period explicitly for calibration (and perhaps monitoring degrees of freedom and aikake criteria and the like to make sure the model is “simple” and therefore hopefully not overfitted.) It would be interesting to see which performed better for an “after sample test” during last 25 years, the MBH variant used by splitting the instrument period into calibration and “verification” (scare quotes on purpose), versus one that just comes from using the entire period for calibration.