Emergent constraints on climate sensitivity in global climate models, Part 1

Their nature and assessment of their validity

A guest post by Nic Lewis

There have been quite a number of papers published in recent years concerning “emergent constraints” on equilibrium climate sensitivity (ECS) in comprehensive global climate models (GCMs), of both the current (CMIP5) and previous (CMIP3) generations. The range of ECS values in GCMs has remained almost unchanged since the early days of climate modelling; in the IPCC 5th Assessment Report (AR5) it was given as 2.1-4.7°C for CMIP5 models.[i]

From the IPCC 1st Assessment Report (FAR) to AR5, the main cause of the large uncertainty as to ECS in GCMs has been the difficulty of simulating clouds and their behaviour.[ii] This has led to cloud feedback differing between GCMs even as to its sign – and to little confidence that the true level of cloud feedback lies within its range in GCMs. Progress in understanding cloud behaviour and related convective dynamics and feedbacks has been painfully slow. We shall see in this 3-part article that emergent constraint approaches have the potential to offer useful insights into cloud behaviour, however the main focus will be on to what extent they narrow the uncertainty range of ECS in GCMs.

Emergent constraint studies typically identify a quantitative measure of an aspect of GCMs’ behaviour (a metric) that is well correlated with their ECS values across an ensemble of GCMs. They then compare observational estimates of that metric with its value in each GCM. Often they use regression to fit a linear relationship between the metric and ECS values in the GCM-ensemble, and assert that the range of ECS values spanned by the segment of the regression line consistent with observational estimates of the metric is most credible. Alternatively they derive a constrained ECS range directly from the ECS values of GCMs whose behaviour is consistent with observational estimates of the metric. Sometimes more than one metric is used. In most cases, such emergent constraint studies have favoured ECS values in the upper half (3.4–4.7°C) of the CMIP5 range.

Probably the best known emergent constraints study is Sherwood et al (2014).[iii] That study has often been cited in support of arguments that ECS is unlikely to be in the lower half of the IPCC’s 1.5–4.5°C range, contrary to what is suggested by energy-budget studies that relate warming and heat uptake during the instrumental period to the estimated change radiative forcing. The abstract to Sherwood et al (2014) says this about the spread of ECS in GCMs:

The spread arises largely from differences in the feedback from low clouds, for reasons not yet understood. Here we show that differences in the simulated strength of convective mixing between the lower and middle tropical troposphere explain about half of the variance in climate sensitivity estimated by 43 climate models. The apparent mechanism is that such mixing dehydrates the low-cloud layer at a rate that increases as the climate warms, and this rate of increase depends on the initial mixing strength, linking the mixing to cloud feedback. The mixing inferred from observations appears to be sufficiently strong to imply a climate sensitivity of more than 3 degrees for a doubling of carbon dioxide.

The Sherwood emergent constraint is linked to feedback from changes in low clouds, which affect reflected shortwave (SW) solar radiation. Differences between GCMs in SW low cloud feedback are known to be the main factor behind their wide spread of ECS values, so it makes sense that an emergent constraint would involve SW cloud feedback. Indeed, a paper published earlier this year (Qu et al 2018)[iv] concluded that all useful emergent constraints on ECS work via SW low cloud feedback, saying:

Here a statistical method (including a backward selection process) is employed to achieve a better statistical understanding of the connections between four recently proposed emergent constraint metrics and individual feedbacks influencing ECS. The relationship between each metric and ECS is largely attributable to a statistical connection with shortwave low cloud feedback, the leading cause of intermodel ECS spread. This result bolsters confidence in some of the metrics, which had assumed such a connection in the first place. Additional analysis is conducted with a few thousand artificial metrics that are randomly generated, but are well correlated with ECS. The relationships between the contrived metrics and ECS can also be linked statistically to shortwave cloud feedback. Thus any proposed or forthcoming ECS constraint based on the current generation of climate models should be viewed as a potential constraint on shortwave cloud feedback, and physical links with that feedback should be investigated to verify that the constraint is real. In addition, any proposed ECS constraint should not be taken at face value, since other factors influencing ECS besides shortwave cloud feedback could be systematically biased in the models.

The key point is that, since SW low cloud feedback accounts for a large part of the overall variation in GCM ECS values, any metric that in GCMs is well correlated with ECS is almost bound to be strongly correlated with SW cloud feedback. But, as Qu et al. go on to say, the reality of the physical links between the emergent constrain metric and SW feedback must be investigated, and even where it exists other biases in GCMs may affect the validity of their ECS values. That is a very important point.

A recent paper, Caldwell et al. 2018,[v] systematically reviewed emergent constraints on ECS, analysing the 19 previously-proposed constraints detailed in Table 1. They omitted emergent constraints that were impracticable to test or had already been found not to be robust. They also omitted two emergent constraints that targeted high-latitude clouds and were poorly correlated with ECS, which they took to mean that only constraints on tropical clouds had a strong impact on ECS.

The correlations with ECS given in Table 1 are for ensembles of all CMIP5 models for which the data needed to calculate the metric for the emergent constraint involved were available. The ensemble therefore varies between constraints as to both size and constituent models. Correlations shown in brackets are not significant at the 90% probability level. Because many models are related, and the likelihood of data-mining for high correlation constraints having taken place, the significance test is weak and (as Caldwell et al. say) should be regarded as just screening out constraints that are almost certainly not significant. In many cases the original study tested the constraint on CMIP3 models or a combination of CMIP3 and CMIP5 models, and may have obtained a stronger correlation. However, a valid emergent constraint should persist between model generations.

Name of constraint Year Correlation in CMIP5 Description
Covey 2000 0.35 Amplitude of seasonal cycle of surface temperature
Volodin 2008 –0.60 Difference between tropical and southern-hemisphere midlatitude total cloud fraction
Trenberth 2010 [–0.22] Net TOA radiation averaged over the southern hemisphere
Fasullo D 2012 [–0.26] Southern hemisphere zonal-average mid-tropospheric RH in dry-zone between 8.5°-20°S
Fasullo M 2012 [0.15] Tropical zonal-average lower-tropospheric RH in moist-convective region
Qu 2013 [–0.29] BL cloud amount response to SST variations in subtropical stratocumulus regions (after removing Estimated Inversion Strength contribution)
Klein ctp-tau 2013 –0.74 Error in the distribution of cloud-top pressure and optical thickness for regions between 60°N & S
Klein TCA 2013 –0.71 Error in total cloud amount for regions between 60°N and S
Su 2014 0.58 Error in vertically-resolved tropospheric zonal-average relative humidity between 40°N and 45°S
Sherwood D 2014 0.40 Strength of resolved-scale mixing between BL and lower troposphere in tropical E Pacific and Atlantic
Sherwood S 2014 0.37 Strength of mixing between BL and lower troposphere in tropical convective regions
Sherwood LTMI 2014 0.65 Sum of Sherwood S and D constraints
Brient Shal 2015 0.38 Fraction of tropical clouds with tops below 850 mb whose tops are also below 950 mb
Zhai 2015 –0.73 Seasonal response of BL cloud amount to SST variations in oceanic subsidence regions between 20-40°latitude
Tian 2015 –0.60 Strength of double-ITCZ bias
Brient Alb 2016 –0.71 Sensitivity of cloud albedo in tropical oceanic low-cloud regions to present-day SST variations
Lipat 2017 –0.46 Latitude of the southern edge of the Hadley cell in austral summer
Siler 2017 0.54 Extent to which cloud albedo is small in warm-SST regions and large in cold-SST regions
Cox 2018 0.63 Strength of global-average surface temperature variations and temporal autocorrelation

Table 1. Short description of each emergent constraint on ECS tested in Caldwell et al. 2018, per their Table 1, and their calculation of its correlation with ECS in CMIP5 models (from their Table 2). Correlations in brackets are not significant with 90% probability, assuming model independence.

The Caldwell findings are very interesting. Its abstract makes these points:

Several constraints are shown to be closely related, emphasizing the importance for careful understanding of proposed constraints. A new method is presented for decomposing correlation between an emergent constraint and ECS into terms related to physical processes and geographical regions. Using this decomposition, one can determine whether the processes and regions explaining correlation with ECS correspond to the physical explanation offered for the constraint. Shortwave cloud feedback is generally found to be the dominant contributor to correlations with ECS because it is the largest source of inter-model spread in ECS. In all cases, correlation results from interaction between a variety of terms, reflecting the complex nature of ECS and the fact that feedback terms and forcing are themselves correlated with each other. For 4 of the 19 constraints, the originally-proposed explanation for correlation is borne out by our analysis. These 4 constraints all predict relatively high climate sensitivity. The credibility of 6 other constraints is called into question due to correlation with ECS coming mainly from unexpected sources and/or lack of robustness to changes in ensembles. Another 6 constraints lack a testable explanation and hence cannot be confirmed. The fact that this study casts doubt upon more constraints than it confirms highlights the need for caution when identifying emergent constraints from small ensembles.

Caldwell et al. also point out that:

One problem with emergent constraints is that large inter-model correlations between current climate and future-climate quantities are expected by chance in multi-model databases. As a result, emergent constraints without a solid physical basis should be viewed with scepticism. Unfortunately, most emergent constraints in the published literature lack a satisfying physical explanation.

Caldwell et al. therefore regard a proposed emergent constraint as not credible if it lacks an identifiable physical mechanism; is not robust to change of model ensemble; or if its correlation with ECS is not due to its proposed physical mechanism. For the last point, they examine the decomposition of the correlation by source (types of feedback, and forcing from a doubling of CO2)  and the geographical location of the principal sources of correlation.
Caldwell et al.’s eventual assessments of the 19 proposed emergent constraints are shown in Table 2. In two cases they combined a pair of constraints that are similar and well correlated. They note that those two pairs (Volodin/Siler and Zhai/Brient Alb) are similar in that both assume cloud changes track SST in a climate-invariant way, but that although Volodin and Siler are both strongly correlated with Brient Alb, they are poorly correlated with Zhai.

Name Credible? Why?
Covey no not robust to change in ensembles
Volodin/Siler unclear no testable mechanism
Trenberth no not robust to change in ensembles, proposed mechanism is not the main source of correlation
Fasullo D no not robust to change in ensembles, no testable mechanism
Fasullo M no not robust to change in ensembles, no testable mechanism
Qu uncertain not robust to change in ensembles, CMIP5 correlation not due to proposed mechanism
Klein ctp-tau unclear no proposed mechanism
Klein TCA unclear no proposed mechanism
Su unclear no testable mechanism
Sherwood D yes correlation is due to proposed mechanism and region
Sherwood S no CMIP5 correlation is not due to the proposed mechanism
Sherwood LTMI no combination of credible and non-credible mechanisms
Brient Shal yes correlation is mainly due to proposed mechanism and region
Tian unclear mechanism isn’t clear enough to test
Zhai/Brient Alb yes correlation is due to proposed mechanism and region
Lipat uncertain proposed region is important, but isn’t the dominant source of correlation
Cox uncertain proposed mechanism is unrelated to individual feedbacks and regions

Table 2. Assessment of proposed emergent constraints by Caldwell et al 2018. Reproduction of their Table 4

The fact that only 4 out of the 19 emergent constraints pass Caldwell’s basic tests of credibility is disturbing, and implies that results from emergent constraint studies should be treated with considerable scepticism. Some of the papers proposing failed constraints are highly cited, e.g. Trenberth (where the originally reported strong –0.73 correlation fell to a very weak –0.22 in CMIP5 models) has 219 citations.[i] Somewhat surprisingly, all three of the proposed emergent constraints that Trenberth and Fasullo formulated had insignificant correlations with ECS when tested on CMIP5 models.

In Parts 2 and 3 of this article I will examine the 4 constraints, all favouring high ECS values, that Caldwell et al find credible, and formulate conclusions.


Nicholas Lewis                                                                      March 2018

[i] Trenberth, K. E., and J. T. Fasullo, 2010: Simulation of present day and 21st century energy budgets of the southern oceans. J. Clim., 23, 440–454.

[i] Table 9.5 of AR5. The method used appears to slightly overestimate the ECS of the least sensitive CMIP5 models and to underestimate the ECS of the most sensitive models. A better estimate of the range would be from slightly under 2°C to slightly above 5 °C. This is almost identical to the 1.9–5.2°C range in the IPCC 1st Assessment Report (FAR).

[ii] See, e.g., Section 5.2.1 of the FAR. Note that I am including problems in simulating convection as part of the cloud simulation problems, as the two issues are intimately connected.

[iii] Sherwood, S.C., Bony, S. and Dufresne, J.L., 2014. Spread in model climate sensitivity traced to atmospheric convective mixing. Nature, 505(7481), p.37-42.

[iv] Qu, X., A. Hall, A. M. DeAngelis, M. D. Zelinka, S. A. Klein, H. Su, B. Tian, and C. Zhai, 2018: On the emergent constraints of climate sensitivity. Journal of Climate, 31 (2), 863–875, doi:10.1175/JCLI-D-17-0482.1.

[v] Caldwell, P, M Zelinka and S Klein, 2018. Evaluating Emergent Constraints on Equilibrium Climate Sensitivity. J. Climate. doi:10.1175/JCLI-D-17-0631.1, in press.


  1. Jaap Titulaer
    Posted Mar 19, 2018 at 2:36 PM | Permalink

    This post on WUWT seems related:

    Global warming on trial and the elementary error of physics that caused the global warming scare
    2018-03-19 By Christopher Monckton of Brenchley

    Abstract: In a dynamical system, even an unamplified input signal induces a response to any feedback. Hitherto, however, the large feedback response to emission temperature has been misattributed to warming from the naturally-occurring, non-condensing greenhouse gases. After correction, the theoretically-derived pre-industrial feedback fraction is demonstrated to cohere with the empirically-derived industrial-era value an order of magnitude below previous estimates, mandating reduction of projected Charney sensitivity from 3.3 +/- 1.2 K to 1.2 +/- 0.15 K.

    Steve: you will find very little, if any, discussion of Monckton on this blog. I discourage people from thinking in over-simplistic terms. If you wish to pursue, better to do so at WUWT.

    • Jaap Titulaer
      Posted Mar 20, 2018 at 9:09 AM | Permalink

      Sure Steve, no problem 🙂

  2. AntonyIndia
    Posted Mar 20, 2018 at 1:10 AM | Permalink

    The 3 authors of the Peter Caldwell, Mark Zelinka & Stephen Klein (2018) paper on which much of the above is based published in 2014 on related matters together with Ben Santer – the high profile UEA educated climate scientist: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1002/2014GL059205
    All above except Klein work at the Lawrence Livermore National Laboratory.

    • Posted Mar 20, 2018 at 5:34 AM | Permalink

      I’m unaware of what Ben Santer’s contribution to the Caldwell et al 2014 paper was. I know both the second author (Chris Bretherton – one of the best known experts on clouds) and the third author (Mark Zelinka). I consider both of them to be good scientists.

  3. barn E. rubble
    Posted Mar 20, 2018 at 5:36 PM | Permalink

    RE: Mark – Helsinki, “Disappointing Steve . . .”

    How so? The post referred to was somewhere else and comments should be there, no? That should end it but I’m guessing it was Steve’s comment about Monckton that’s at issue here. This site tends to be very ‘technical’ and on point to the thread (for the most part; exceptions like this one) and is moderated as such. Monckton (who I often find interesting) does tend to over simplify things to reach a wider audience. He does that on purpose. Steve’s view of Monckton maybe different from mine and obviously yours. However, comments re: posts from everywhere? Doesn’t help.

    Steve: Many critics of this site have criticized me for not parsing theories proposed by “skeptics”.
    I don’t have time or energy to do so, plus there are many other venues to do so. Nor did I wish to defend analyses that I hadn’t done myself. So it’s a bright line issue for me. Sorry.

  4. Matt Skaggs
    Posted Mar 20, 2018 at 9:33 PM | Permalink

    Yikes, a reference to Monckton on a post by Nic.

    Nic, maybe this isn’t, and really never was, your place, now that the archives include far-right political posts about Hillary Clinton and silliness about Tom Brady.

    Nic wrote:

    “They then compare observational estimates of that metric with its value in each GCM.”

    I have exhausted my interest in GCMs, but I would still like to peruse the observational data. Where do these scientists find it? Are there websites? Specifically, are there summaries of changes in measurement data over the satellite era that are available to mere mortals? I have been looking in vain.

    • Posted Mar 25, 2018 at 7:58 AM | Permalink

      Matt, much of the observational data involved is available online in various places, although sometimes it needs significant processing before use. The first thing to do is to look at the study that involves observational data you are interested in. Usually nowadays it will state the name of the dataset and where the data comes from, and even if it doesn’t give a URL typically a search engine will then locate it.

  5. mpainter
    Posted Mar 21, 2018 at 5:15 AM | Permalink

    I’ve said this before, but cloud dynamics are poorly understood because the microphysics of cloud formation and behavior are not well understood. This area has been a field of endeavor for many decades and little progress has been made. It is ludicrous to use the product of a GCM to try to achieve what real science cannot. Thus climate scientists.

  6. dfhunter
    Posted Mar 21, 2018 at 6:01 PM | Permalink

    Hi Nic

    thanks for the post & info

    maybe a stupid question, but why –

    “They also omitted two emergent constraints that targeted high-latitude clouds and were poorly correlated with ECS, which they took to mean that only constraints on tropical clouds had a strong impact on ECS”

    can see the logic of this(a bit), but high-latitude clouds are not to be dis-mist, or did I miss the answer ?

    • Posted Mar 22, 2018 at 11:38 AM | Permalink

      Hi dfhunter,

      I agree that the behaviour of high-latitude clouds is also important. But for an emergent constraint to usefully constrain ECS in GCMs, there needs to be a high correlation across an ensemble of GCMs (e.g., CMIP5) between ECS and whatever the emergent constraint metric is. And that wasn’t the case for the two emergent constraints involving high latitude clouds that Caldwell et al omitted. It is possible that some other emergent constraint metric involving high-latitude cloud behaviour may be found that is strongly correlated with ECS in GCMs, but Caldwell et al obviously think that is unlikely.

      • dfhunter
        Posted Mar 22, 2018 at 4:21 PM | Permalink

        Thanks for the reply Nic,

        I keep getting confused between real world & GCM worlds 😦

        should read the post more carefully before commenting.

        thanks again

4 Trackbacks

  1. […] There have been quite a number of papers published in recent years concerning “emergent constraints” on equilibrium climate sensitivity (ECS) in comprehensive global climate models (GCMs), of both the current (CMIP5) and previous (CMIP3) generations. The range of ECS values in GCMs has remained almost unchanged since the early days of climate modelling; in the IPCC 5th Assessment Report (AR5) it was given as 2.1-4.7°C for CMIP5 models.[i] […]

  2. […] Part 1 of this article the nature and validity of emergent constraints[i] on equilibrium climate […]

  3. […] Part 1 of this article the nature and validity of emergent constraints[1] on equilibrium climate […]

  4. […] Part 1 of this article the nature and validity of emergent constraints[1] on equilibrium climate sensitivity (ECS) in GCMs were discussed, drawing mainly on the analysis and assessment of 19 such constraints in Caldwell et al. (2018),[2] who concluded that only four of them were credible. An extract of the rows of Table 1 of Part 1 detailing those four emergent constraints is given below.[3] […]

%d bloggers like this: