<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:media="http://search.yahoo.com/mrss/"
		>
<channel>
	<title>Comments on: MBH98-Style Pseudo-Confidence Intervals for Loehle</title>
	<atom:link href="http://climateaudit.org/2007/11/21/mbh98-style-error-bars-for-loehle/feed/" rel="self" type="application/rss+xml" />
	<link>http://climateaudit.org/2007/11/21/mbh98-style-error-bars-for-loehle/</link>
	<description>by Steve McIntyre</description>
	<lastBuildDate>Thu, 20 Jun 2013 06:24:11 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.com/</generator>
	<item>
		<title>By: Ulises</title>
		<link>http://climateaudit.org/2007/11/21/mbh98-style-error-bars-for-loehle/#comment-119599</link>
		<dc:creator><![CDATA[Ulises]]></dc:creator>
		<pubDate>Fri, 07 Dec 2007 16:35:49 +0000</pubDate>
		<guid isPermaLink="false">http://www.climateaudit.org/?p=2405#comment-119599</guid>
		<description><![CDATA[#279 :

&lt;blockquote&gt;Note that you are using mean of the data as a reference, not the true global mean temperature.&lt;/blockquote&gt;

Isn&#039;t the true global mean unknown, otherwise sampling were unnecessary? A CI, according to my books, should bracket the respective population parameter with the specified probability. It is necessarily spread around the sample mean, the best estimate available for the true mean. Did I get something wrong ?]]></description>
		<content:encoded><![CDATA[<p>#279 :</p>
<blockquote><p>Note that you are using mean of the data as a reference, not the true global mean temperature.</p></blockquote>
<p>Isn&#8217;t the true global mean unknown, otherwise sampling were unnecessary? A CI, according to my books, should bracket the respective population parameter with the specified probability. It is necessarily spread around the sample mean, the best estimate available for the true mean. Did I get something wrong ?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Hu McCulloch</title>
		<link>http://climateaudit.org/2007/11/21/mbh98-style-error-bars-for-loehle/#comment-119598</link>
		<dc:creator><![CDATA[Hu McCulloch]]></dc:creator>
		<pubDate>Fri, 07 Dec 2007 16:26:15 +0000</pubDate>
		<guid isPermaLink="false">http://www.climateaudit.org/?p=2405#comment-119598</guid>
		<description><![CDATA[Spence --
  Thanks.  A more ambitious future study would utilize only
studies that reported confidence intervals, and then use this information
in addition to their conformity to the consensus to establish weights
and to compute CI&#039;s for the consensus.  Craig has found such CIs for
#11 (Holmgren), but I doubt that all the others were so careful.
UC --
  The choice of a reference period is admitedly arbitrary, but Craig&#039;s choice of the bimillennial average is quite reasonable for this data set.]]></description>
		<content:encoded><![CDATA[<p>Spence &#8211;<br />
  Thanks.  A more ambitious future study would utilize only<br />
studies that reported confidence intervals, and then use this information<br />
in addition to their conformity to the consensus to establish weights<br />
and to compute CI&#8217;s for the consensus.  Craig has found such CIs for<br />
#11 (Holmgren), but I doubt that all the others were so careful.<br />
UC &#8211;<br />
  The choice of a reference period is admitedly arbitrary, but Craig&#8217;s choice of the bimillennial average is quite reasonable for this data set.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Spence_UK</title>
		<link>http://climateaudit.org/2007/11/21/mbh98-style-error-bars-for-loehle/#comment-119597</link>
		<dc:creator><![CDATA[Spence_UK]]></dc:creator>
		<pubDate>Fri, 07 Dec 2007 08:21:16 +0000</pubDate>
		<guid isPermaLink="false">http://www.climateaudit.org/?p=2405#comment-119597</guid>
		<description><![CDATA[Interesting work, Hu.

UC,

&lt;blockquote&gt;Note that you are using mean of the data as a reference, not the true global mean temperature.&lt;/blockquote&gt;
Good point, I would nuance it slightly differently though (and this adds to the changing of number of proxies or the use of weighted means): the CIs presented are not for the global mean, but for the average of the 18 locations (which may be slightly different), or for the average of the 16 locations if we remove some on S/N grounds.  Furthermore, if we weight to reduce the variance, this may increase the disparity between the mean of the locations and global mean.

To get the best estimate of global mean, I suspect, requires a balancing act between:

* Geographical representation (most area covered)
* Weighted mean (to get the best S/N for the given locations)
* Relationship between this and the global mean (perhaps derived by relating limited numbers of weather station data to complete networks?)

I would have thought, given an arbitrary set of proxies and a S/N estimate for each, it should be possible to derive an &quot;optimal&quot; set of weights to minimise CIs &lt;i&gt;for the global mean&lt;/i&gt; on this basis.

I believe someone already did a plot similar to this in an earlier thread (limited number of stations vs. full network).]]></description>
		<content:encoded><![CDATA[<p>Interesting work, Hu.</p>
<p>UC,</p>
<blockquote><p>Note that you are using mean of the data as a reference, not the true global mean temperature.</p></blockquote>
<p>Good point, I would nuance it slightly differently though (and this adds to the changing of number of proxies or the use of weighted means): the CIs presented are not for the global mean, but for the average of the 18 locations (which may be slightly different), or for the average of the 16 locations if we remove some on S/N grounds.  Furthermore, if we weight to reduce the variance, this may increase the disparity between the mean of the locations and global mean.</p>
<p>To get the best estimate of global mean, I suspect, requires a balancing act between:</p>
<p>* Geographical representation (most area covered)<br />
* Weighted mean (to get the best S/N for the given locations)<br />
* Relationship between this and the global mean (perhaps derived by relating limited numbers of weather station data to complete networks?)</p>
<p>I would have thought, given an arbitrary set of proxies and a S/N estimate for each, it should be possible to derive an &#8220;optimal&#8221; set of weights to minimise CIs <i>for the global mean</i> on this basis.</p>
<p>I believe someone already did a plot similar to this in an earlier thread (limited number of stations vs. full network).</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: bender</title>
		<link>http://climateaudit.org/2007/11/21/mbh98-style-error-bars-for-loehle/#comment-119596</link>
		<dc:creator><![CDATA[bender]]></dc:creator>
		<pubDate>Fri, 07 Dec 2007 07:41:06 +0000</pubDate>
		<guid isPermaLink="false">http://www.climateaudit.org/?p=2405#comment-119596</guid>
		<description><![CDATA[Much better. Now to audit the method.]]></description>
		<content:encoded><![CDATA[<p>Much better. Now to audit the method.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: UC</title>
		<link>http://climateaudit.org/2007/11/21/mbh98-style-error-bars-for-loehle/#comment-119595</link>
		<dc:creator><![CDATA[UC]]></dc:creator>
		<pubDate>Fri, 07 Dec 2007 07:24:09 +0000</pubDate>
		<guid isPermaLink="false">http://www.climateaudit.org/?p=2405#comment-119595</guid>
		<description><![CDATA[#278,

Interesting work! Note that you are using mean of the data as a reference, not the true global mean temperature. That reminded me of Baron Munchausen (bootstrap in English version, in German version he draws himself up by his own hair (?)).

Our search for scientific CIs is progressing, though. Let me try the same method with Mann&#039;s network..]]></description>
		<content:encoded><![CDATA[<p>#278,</p>
<p>Interesting work! Note that you are using mean of the data as a reference, not the true global mean temperature. That reminded me of Baron Munchausen (bootstrap in English version, in German version he draws himself up by his own hair (?)).</p>
<p>Our search for scientific CIs is progressing, though. Let me try the same method with Mann&#8217;s network..</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Hu McCulloch</title>
		<link>http://climateaudit.org/2007/11/21/mbh98-style-error-bars-for-loehle/#comment-119594</link>
		<dc:creator><![CDATA[Hu McCulloch]]></dc:creator>
		<pubDate>Thu, 06 Dec 2007 14:22:37 +0000</pubDate>
		<guid isPermaLink="false">http://www.climateaudit.org/?p=2405#comment-119594</guid>
		<description><![CDATA[Valid CIs for Loehle Global Temperature
by Hu McCulloch
(Perhaps Steve might want to start a fresh thread on this)
I have now constructed valid CIs for Craig Loehle&#039;s unweighted average global temperature series, using the method I described at #11 above (in the 11/21 thread #2405).  The exact standard errors vary with the number and identity of the proxies that are included at any point in time, but generally are around 0.16 dC.  A plot of a 95% CI (+/- 2se) about Craig&#039;s series is linked at http://www.econ.ohio-state.edu/jhm/AGW/Loehle/OLSCI.gif and should appear below:

The MWP is significantly above the bimillennial average  during approximately 850-1050.  There is also a significantly below-average LIA approximately 1450-1750.

Detailed graphs of the individual series etc are in a (still very preliminary) note at &lt;a href=&quot;http://www.econ.ohio-state.edu/jhm/AGW/Loehle/LoehleGraphs.doc&quot; rel=&quot;nofollow&quot;&gt;http://www.econ.ohio-state.edu/jhm/AGW/Loehle/LoehleGraphs.doc&lt;/a&gt;.  As discussed there, the variances of the 18 series differ considerably.  Two of them, #3 Cronin and #10 deMenocal, have such high variances that they are actually detrimental to the variance of the unweighted average of the series.  Two others, #6 Korhola and #13 Viao, have very low variances about the mean and so are very informative.

Because of the unequal variances, Weighted Least Squares (WLS) estimates that weight each series in inverse proportion to its variance are more efficient.  The standard errors are generally around 0.10 dC, a considerable improvement over the unweighted Ordinary Least Squares (OLS) estimates above.  The WLS estimates with a 95% CI are linked at http://www.econ.ohio-state.edu/jhm/AGW/Loehle/WLSCI.gif and should appear below:


The significant portions of the MWP and LIA are about the same with WLS as for OLS, so thanks to Craig&#039;s data set, the MWP is alive and well.  (The little bump in the late 20th century at the right end of the graphs must be Al Gore!)

Only rarely were all 18 series active.  Usually 14-16 were availble, though occasionally the number fell as low as 12.  If there is popular request, I could post the graph of this that appears in the paper linked above.

I was able to replicate Craig&#039;s series from the raw data Steve posted to within an average absolute difference of 0.0018 dC and a max abs difference of 0.0919 dC, but only by using a 29-year rolling mean rather than 30 as indicated in the paper or 31 as Craig says he in fact used in #263 above.  The average abs difference was 0.0105 dC and the max abs difference 0.203 dC using a 31-year centered rolling mean, and comparably bad with a 30-year rolling mean, whether 14/1/15 or 15/1/14.  I take this to mean that he in fact used a 29-year rolling mean.]]></description>
		<content:encoded><![CDATA[<p>Valid CIs for Loehle Global Temperature<br />
by Hu McCulloch<br />
(Perhaps Steve might want to start a fresh thread on this)<br />
I have now constructed valid CIs for Craig Loehle&#8217;s unweighted average global temperature series, using the method I described at #11 above (in the 11/21 thread #2405).  The exact standard errors vary with the number and identity of the proxies that are included at any point in time, but generally are around 0.16 dC.  A plot of a 95% CI (+/- 2se) about Craig&#8217;s series is linked at <a href="http://www.econ.ohio-state.edu/jhm/AGW/Loehle/OLSCI.gif" rel="nofollow">http://www.econ.ohio-state.edu/jhm/AGW/Loehle/OLSCI.gif</a> and should appear below:</p>
<p>The MWP is significantly above the bimillennial average  during approximately 850-1050.  There is also a significantly below-average LIA approximately 1450-1750.</p>
<p>Detailed graphs of the individual series etc are in a (still very preliminary) note at <a href="http://www.econ.ohio-state.edu/jhm/AGW/Loehle/LoehleGraphs.doc" rel="nofollow">http://www.econ.ohio-state.edu/jhm/AGW/Loehle/LoehleGraphs.doc</a>.  As discussed there, the variances of the 18 series differ considerably.  Two of them, #3 Cronin and #10 deMenocal, have such high variances that they are actually detrimental to the variance of the unweighted average of the series.  Two others, #6 Korhola and #13 Viao, have very low variances about the mean and so are very informative.</p>
<p>Because of the unequal variances, Weighted Least Squares (WLS) estimates that weight each series in inverse proportion to its variance are more efficient.  The standard errors are generally around 0.10 dC, a considerable improvement over the unweighted Ordinary Least Squares (OLS) estimates above.  The WLS estimates with a 95% CI are linked at <a href="http://www.econ.ohio-state.edu/jhm/AGW/Loehle/WLSCI.gif" rel="nofollow">http://www.econ.ohio-state.edu/jhm/AGW/Loehle/WLSCI.gif</a> and should appear below:</p>
<p>The significant portions of the MWP and LIA are about the same with WLS as for OLS, so thanks to Craig&#8217;s data set, the MWP is alive and well.  (The little bump in the late 20th century at the right end of the graphs must be Al Gore!)</p>
<p>Only rarely were all 18 series active.  Usually 14-16 were availble, though occasionally the number fell as low as 12.  If there is popular request, I could post the graph of this that appears in the paper linked above.</p>
<p>I was able to replicate Craig&#8217;s series from the raw data Steve posted to within an average absolute difference of 0.0018 dC and a max abs difference of 0.0919 dC, but only by using a 29-year rolling mean rather than 30 as indicated in the paper or 31 as Craig says he in fact used in #263 above.  The average abs difference was 0.0105 dC and the max abs difference 0.203 dC using a 31-year centered rolling mean, and comparably bad with a 30-year rolling mean, whether 14/1/15 or 15/1/14.  I take this to mean that he in fact used a 29-year rolling mean.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: UC</title>
		<link>http://climateaudit.org/2007/11/21/mbh98-style-error-bars-for-loehle/#comment-119593</link>
		<dc:creator><![CDATA[UC]]></dc:creator>
		<pubDate>Tue, 04 Dec 2007 07:34:46 +0000</pubDate>
		<guid isPermaLink="false">http://www.climateaudit.org/?p=2405#comment-119593</guid>
		<description><![CDATA[&lt;blockquote&gt;I believe they had both already said it was meaningless crap, leading me to believe trying to do a real one was similarly meaningless.
&lt;/blockquote&gt;


Let&#039;s not make conclusions yet, this is a young thread ;) MBH98 CIs are clearly meaningless crap (we know the method).

For Loehle07, we haven&#039;t even tried yet. #11 and Loehle07 Fig. 3 are OK starting points, but both methods provide finite CIs for any input data. Next step is to read the original publications. If the original reconstruction errors can be shown to be independent of target temperature, we can come back to the global scale. If not, then there&#039;s more work to do..]]></description>
		<content:encoded><![CDATA[<blockquote><p>I believe they had both already said it was meaningless crap, leading me to believe trying to do a real one was similarly meaningless.
</p></blockquote>
<p>Let&#8217;s not make conclusions yet, this is a young thread <img src='http://s1.wp.com/wp-includes/images/smilies/icon_wink.gif' alt=';)' class='wp-smiley' />  MBH98 CIs are clearly meaningless crap (we know the method).</p>
<p>For Loehle07, we haven&#8217;t even tried yet. #11 and Loehle07 Fig. 3 are OK starting points, but both methods provide finite CIs for any input data. Next step is to read the original publications. If the original reconstruction errors can be shown to be independent of target temperature, we can come back to the global scale. If not, then there&#8217;s more work to do..</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Sam Urbinto</title>
		<link>http://climateaudit.org/2007/11/21/mbh98-style-error-bars-for-loehle/#comment-119592</link>
		<dc:creator><![CDATA[Sam Urbinto]]></dc:creator>
		<pubDate>Tue, 04 Dec 2007 01:00:30 +0000</pubDate>
		<guid isPermaLink="false">http://www.climateaudit.org/?p=2405#comment-119592</guid>
		<description><![CDATA[Hu, it&#039;s been fairly clear to me (at least I thought it was...) that the exercise was in how pointless (???) doing it was.  I don&#039;t know enough to comment on your #11  It seems that Jean S. did the inital work and UC confirmed it and graphed it.  I believe they had both already said it was meaningless crap, leading me to believe trying to do a &quot;real one&quot; was similarly meaningless.

Did you try your method and post the results?  If you did, I missed it (or didn&#039;t understand it).

I&#039;m just an agnostic bystander, although I do at least semi-understand the basic stuff like confidence intervals and standard deviation, at least as far as normal distribution goes.  Beyond that, I get lost.  In order to do error bars at all, don&#039;t you need to know or be able to calculate the +/-?]]></description>
		<content:encoded><![CDATA[<p>Hu, it&#8217;s been fairly clear to me (at least I thought it was&#8230;) that the exercise was in how pointless (???) doing it was.  I don&#8217;t know enough to comment on your #11  It seems that Jean S. did the inital work and UC confirmed it and graphed it.  I believe they had both already said it was meaningless crap, leading me to believe trying to do a &#8220;real one&#8221; was similarly meaningless.</p>
<p>Did you try your method and post the results?  If you did, I missed it (or didn&#8217;t understand it).</p>
<p>I&#8217;m just an agnostic bystander, although I do at least semi-understand the basic stuff like confidence intervals and standard deviation, at least as far as normal distribution goes.  Beyond that, I get lost.  In order to do error bars at all, don&#8217;t you need to know or be able to calculate the +/-?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Hu McCulloch</title>
		<link>http://climateaudit.org/2007/11/21/mbh98-style-error-bars-for-loehle/#comment-119591</link>
		<dc:creator><![CDATA[Hu McCulloch]]></dc:creator>
		<pubDate>Sun, 02 Dec 2007 20:22:24 +0000</pubDate>
		<guid isPermaLink="false">http://www.climateaudit.org/?p=2405#comment-119591</guid>
		<description><![CDATA[Craig wrote in # 264,
&lt;blockquote&gt;
One of the issues that JEG had was that the c.i. should reflect how well the series were calibrated to temperature. This is not addressed by Hus idea, and is what I was attempting above.
&lt;/blockquote&gt;

As I tried to make clear in #261, the method I proposed back in #11 does not require one to know in advance how well each series was calibrated, since that error just shows up in Vi along with the error of the locality&#039;s true temperature&#039;s ability to measure global temperature.  The time series of residuals will tell you the combined effect, provided Vi is constant over time.

&lt;blockquote&gt;
Also not addressed by Hu is that the data at any one point in time may be more variable than at others.
&lt;/blockquote&gt;

Admittedly, but as I already noted, your 30-year smoothing will make them much more homoskedastic than they appear in the plots of your raw data that Steve posted on an earlier thread.  If this is still a concern, one could fit a GARCH model to each study&#039;s residuals and still have lots of DOF to spare.  But some structure has to be imposed, and as a first cut I think constancy is better than no CIs at all.

&lt;blockquote&gt;
How about simple confidence intervals on the mean computed from the data at each point in time? If the data at year t were vary variable, the c.i. would be wider, which is what you want.
&lt;/blockquote&gt;

This works if you&#039;re willing to assume that all studies have equal variances (or known variances), both with respect to their ability to measure local temperature, and with respect to their locality&#039;s ability to measure global temperature.  I think that&#039;s a lot worse than just saying they&#039;re unknown and unequal but constant over time.

Sam Urbinto (#265):
&lt;blockquote&gt;
Seriously, I would think that what UC and Jean S. have already done is fine. Or that the task is either a) impossible or b) not needed. Or both.
&lt;/blockquote&gt;

Even UC admits that the CI&#039;s he computed were all wrong -- they were just intended to be an illustration of how bad Mannian CI&#039;s would have been in this context.  And Jean S never computed an alternative that I am aware of.]]></description>
		<content:encoded><![CDATA[<p>Craig wrote in # 264,</p>
<blockquote><p>
One of the issues that JEG had was that the c.i. should reflect how well the series were calibrated to temperature. This is not addressed by Hus idea, and is what I was attempting above.
</p></blockquote>
<p>As I tried to make clear in #261, the method I proposed back in #11 does not require one to know in advance how well each series was calibrated, since that error just shows up in Vi along with the error of the locality&#8217;s true temperature&#8217;s ability to measure global temperature.  The time series of residuals will tell you the combined effect, provided Vi is constant over time.</p>
<blockquote><p>
Also not addressed by Hu is that the data at any one point in time may be more variable than at others.
</p></blockquote>
<p>Admittedly, but as I already noted, your 30-year smoothing will make them much more homoskedastic than they appear in the plots of your raw data that Steve posted on an earlier thread.  If this is still a concern, one could fit a GARCH model to each study&#8217;s residuals and still have lots of DOF to spare.  But some structure has to be imposed, and as a first cut I think constancy is better than no CIs at all.</p>
<blockquote><p>
How about simple confidence intervals on the mean computed from the data at each point in time? If the data at year t were vary variable, the c.i. would be wider, which is what you want.
</p></blockquote>
<p>This works if you&#8217;re willing to assume that all studies have equal variances (or known variances), both with respect to their ability to measure local temperature, and with respect to their locality&#8217;s ability to measure global temperature.  I think that&#8217;s a lot worse than just saying they&#8217;re unknown and unequal but constant over time.</p>
<p>Sam Urbinto (#265):</p>
<blockquote><p>
Seriously, I would think that what UC and Jean S. have already done is fine. Or that the task is either a) impossible or b) not needed. Or both.
</p></blockquote>
<p>Even UC admits that the CI&#8217;s he computed were all wrong &#8212; they were just intended to be an illustration of how bad Mannian CI&#8217;s would have been in this context.  And Jean S never computed an alternative that I am aware of.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Christopher</title>
		<link>http://climateaudit.org/2007/11/21/mbh98-style-error-bars-for-loehle/#comment-119590</link>
		<dc:creator><![CDATA[Christopher]]></dc:creator>
		<pubDate>Sat, 01 Dec 2007 23:27:20 +0000</pubDate>
		<guid isPermaLink="false">http://www.climateaudit.org/?p=2405#comment-119590</guid>
		<description><![CDATA[In order to do error analysis Id need the functional form with coefficient values and their standard errors. R2 is not enough. The idea is to form CIs for all these coefficients and propagate that through every single computational step you do in your paper until you get to your ensemble proxy. Do any of the original papers for the proxies have such information?]]></description>
		<content:encoded><![CDATA[<p>In order to do error analysis Id need the functional form with coefficient values and their standard errors. R2 is not enough. The idea is to form CIs for all these coefficients and propagate that through every single computational step you do in your paper until you get to your ensemble proxy. Do any of the original papers for the proxies have such information?</p>
]]></content:encoded>
	</item>
</channel>
</rss>
