I agree, Lucia. Nic is far too polite.

When I first read the paper, I just thought it was a bad paper. However, after actually testing the magnitude of the authors’ model error, which is misinterpreted by the authors to be all “natural variability” within the GCMs, I came to the conclusion that it is much worse than just bad. Terrible tripe which leaves the reader diminished. So bad it’s not even wrong.

I’m still moderating it. Thanks for the exhaustive search and the lucid language.

=========

Thanks.

]]>I was concerned about the auto correlation of the regression residuals from the M&F regressions and requested of and received from Jochem Marotzke the ERF (Effective Radiative Forcing) data that I required to replicate the M&F regressions. When I set to doing this I realized that what I had first thought was the authors approach was incorrect. In fact the time invariant variables, alpha and kappa, used in the regressions limit the regression approach to what M&F published. As it turns out auto correlation was not a problem, but in repeating the regression I found some rather problematic issues in the details of the regression results and in looking in detail at the ERF model time series.

My analysis below does not take into consideration the problems connected to the circular nature of the regression. Rather I attempt to show that with a small adjustment to the regression the results change dramatically and would require a very different conclusion than the one acquired from M&F. My alternative approach here will have all the circularity problems of the M&F approach. My preferred approach was discussed at https://climateaudit.org/2015/02/05/marotzke-and-forsters-circular-attribution-of-cmip5-intermodel-warming-differences/#comment-753507 and shows for the period 1975-2014 that the GMST deterministic trends for the observed data sets are smaller than all but 2 or 3 of the CMIP5 models.

I used the ERF, alpha and kappa data provided by Marotzke in combination with the RCP4.5 CMIP5 model GMST (Global Mean Surface Temperature) time series from KNMI. I have linked the data and results from an Excel file in a Dropbox link below. The data I used were from 61 model runs whereas M&F used 75 model runs. The extra runs in M&F were the result of using the historical runs and adding to those runs from the years 2006-2012 from the RCP4.5 scenario series. RCP4.5 includes all the historical part up to 2005 and then from there the RCP4.5 scenario. In order to avoid having to join different series I simply used the 61 RCP4.5 model runs that were avaialable. I did the 15 and 62 year trends as was performed in M&F and obtained very similar results. The results are shown in the tables in the two links below. I show the regression coefficients and p.values and the overall regression p.value, the auto correlation of the regression residuals, the standard deviation of the residuals and the standard deviation of what M&F termed the deterministic part of the regression or regression error.

It becomes obvious looking at the p.values for the regression coefficients and overall regression that for the 15 year trends the regression model cannot used for its implied intended purposes of using the model differences in the independent variables ERF trends, alpha and kappa to predict differences in temperature. It can also be seen looking at the standard deviations of the residuals and the regression error that the variation in regression residuals is large compared to that due to regression – as was the conclusion in M&F. In contrast for 62 year trends, the ERF coefficient and overall regression p.values show statistical significance while the coefficients for alpha and kappa, in general, are not different than zero. For the longer trend period the standard deviations for the residuals and regression are nearly equal for most of the regressions.

The temperature series are different for each model run but the 17 individual models used have only one corresponding realization of ERF (and alpha and kappa which are time invariant) per model. This situation might pass muster if one considers that ERF (and alpha and kappa) are deterministic for a given model and will not thus change much from run to run while the temperature runs are influenced by chaotic effects and can change by goodly amounts. I nevertheless did the same regressions where I averaged the multiple temperature runs for a given model to match the 17 ERF series. I do not show the results, but the only major difference was that the fewer degrees of freedom reduced the frequency of obtaining coefficients different than zero and p.values for over all regression less than 0.05.

What was quite revealing to me (as a layperson relatively new to this area of climate science) is what is shown in the two links of plots of the 17 model temperature and ERF series. (The ERF series are always on top of the temperature series in these plots.) An overwhelming amount of variation year to year is in the ERF series compared to the GMST series. What is quite evident is that the temperature tracks ERF and that the problem with the regression is the noise in the deterministic ERF series. Recall that the trends that M&F reference in their paper are determined over the period by delta T and delta ERF and evidently used, as I did in my regression trend calculations above, the last year in the trend period for T or ERF minus the first year in the trend period. In these linked plots I show a trend line in red for ERF and GMST that was derived from Singular Spectrum Analysis (SSA) using a window of L=67 and combining the first 2 principle components. I used these trends lines and reran the regressions for 15 and 62 years. Those results are linked in two links and tables below. Noticed the dramatic improvement in the regression results, and, particularly so, where the series are expected to see the effects GHG forcing. The regression residuals are much reduced. Notice also that the kappa coefficient does not have p.values indicating values different than zero for any of regressions for either time period or using the M&F method or SSA trends, while alpha begins to consistently show some significance with the SSA trends for 62 year periods.

In conclusion looking at the details of M&F type regressions, as I have done here paints a very different picture than that provided in the M&F paper. The deterministic differences in models is indeed evident in the GMST series.

Two links to two tables showing regressions per M&F methods:

Two links showing plots of model ERF and GMST series with SSA Trends:

Two links to two tables showing regressions using SSA trends:

Link to Excel file with data and results in Dropbox:

https://www.dropbox.com/s/hzfqws74791p56e/M%26F_Regression_Using_MF_Data.xlsx?dl=0

]]>There is an inbuilt assumption in models that internal climate variability is quasi-random and it probably isn’t (see even Mann’s post on Realclimate).

Marotzke & Forster(2015) found that 60 year trends in global surface temperatures are dominated by underlying climate physics. However, the data show that climate models overestimate such 60y decadel trends after 1940.

The main reson for their controversial regression analysis is to seperate forced temperature rise from ‘random’ natural variation **in the models**!

“This is why I believe that the IPCC is failing to give adequate weight into investigating paleoclimate, and other extreme, annual variability.”

I believe that valid temperature reconstructions would provide a great testing ground for climate models. The currently used approach of selecting temperature proxies for reconstructions after the fact of how well they emulate the modern instrumental record can be readily shown as flawed and biased. The validity of the proxy response must be shown prior to selection and then when valid proxies are shown to exist those responses are used regardless of how well the response correlates with the modern temperature record. There is little effort made in this direction, or even, for that matter, of obtaining out-of-sample tests on proxies already used in temperature reconstructions by updating the proxies.

I think the poster called Frank does a good job of summarizing the problems with M&F here:

http://rankexploits.com/musings/2015/new-thread/#comment-135623

Ken,

I think our only difference is in the definition of what is unknown and what is unknowable, location of an electron for example. I am sure we agree that there is a big difference between unpredictable and undecipherable. In M&F I think its a reasonable suspicion of most skeptics that their method is transferring kappa effects to be interpreted as internal variability. Looking from the skeptic’s eye generally the bias of M&F is in finding importance and influence with the suspect that brought one into the investigation, AGHG, and dismissing other factor significance or irrelevance. We are biased too in looking for only deterministic trends that are on the 15 to 100-year scale. This is why I believe that the IPCC is failing to give adequate weight into investigating paleoclimate, and other extreme, annual variability. Who believes that we have a good grasp on dynamics of either? And, why should we believe there would not be clues within both?

In my last comment on Mar 7 below I realize now that M&F’s criticism of Nic in their last paragraph on CLB was for not appreciating their that their method handles variability of alpha and kappa better than his (which is unspecified). It’s still unclear to me whether M&F used a delta T with pre-industrial being the zero point for every interval or used the interval’s start year as zero. If the later their results are confounded by the fact that the diagnosis for all variables was relevant for pre-industrial response to CO2 at pre-industrial temperature. Climate resistance may very well increase with increasing temperature as seen in paleo-plots of the interglacial temperature ceiling, which we are near.

For kappa it only makes sense that the temperature delta should be relative to the pre-industrial. Perhaps this is why, as M&F admitted, their method broke down with temperature down-swings, they presumed to be from vulcanic model over-forcing.

Regardless of the above, M&F’s fundemental flaw IMO is placing any relevance to model to model comparison unless their paper is only making model to model conclusions, as in Forster 2013. If they are not claiming to be making conclusions about the observed forcing and resistance then they did not do much to correct the misconception of the world press.

]]>“I think it is important to keep perspective that chaos exists only to the extent that one lacks understanding to predict events. Five-day weather forecasts would be viewed as absurd in the 1930s. Internal variability is simply the amount of unexplained phenomena. If you dare to widen the field of view to 1, 5 or 20ka the variability problem grows at every scale. It’s not nature’s chaos, its our still infantile understanding.”

I am not sure what you mean here by these comments but I think the chaotic behavior in climate is here to stay and will require treating it as a stochastic phenomena. The does not mean that we cannot attempt to separate those effects from deterministic ones. Climate models with multiple runs do and should show variability and that “noise” can be captured by an ARMA model. The problem with comparing climate model output (generally in the form of temperature trends) to observed and obtaining statistically significant differences is that an individual model run cannot time some of those noise excursions that can affect shorter term temperature trend. Further the observed results come from a single realization and we do not have nor can we obtain other realizations of it without representing the observed temperature series as an ARMA model. I suspect a number of researchers in the climate field are hesitant to apply stochastic models to that part of a temperature series that remains as residuals after removing an estimate of the deterministic trend.

In the introduction to this thread, Nic Lewis stated that he did agree with the authors that the internal variability or that part of temperature series that can be represented as noise makes the comparison of model output to the observed difficult over 15 year periods when attempting to show statistical significance. I agree with that position and is why I have become more interested in the areas of model to observed comparisons that Nic Lewis has been studying, i.e. the deterministic values of ECS and TCR. After all the main issue of AGW is that part of climate (temperature here) for which man is responsible and that would be the forcing caused primarily by GHGs and aerosols. M&F made an attempt in their paper discussed here to make a separation of the variability in the deterministic trends and noise in these series but in my layperson’s view the attempt was weak and pretty much failed.