A New Puzzle: Two Versions of the Sommer Report

A recent David Holland FOI has turned up an astonishing new riddle about the relationship between UEA and the Muir Russell panel: there are two different versions of the Sommer Report on the Backup Server, both dated 17 May 2010 and both entitled “UEA – CRU Review: Initial Report and commentary on email examination”. One version was included in the Muir Russell archive of online evidence – see here – it was only two pages long. A different 10-page version was produced by UEA in response to David Holland’s FOI – see documentation or here as html. The longer version contains details not included in the (apparently) expurgated version published by Muir Russell. The short version is derived from the longer version. Although the two versions of the report are both said to have the same author and bear the same date, there are differences in formatting that, in my opinion, point strongly to the shorter version having been prepared by someone other than Peter Sommer for reasons that, at present, are not entirely clear. If, on the other hand, Sommer himself did prepare the shorter version as well as the longer version, the UEA appears to have withheld correspondence documenting their reason for requesting a second version of the report and whether the second version was backdated. Continue reading

AGU Honors Gleick

If I was hoping to think about more salubrious characters than Lewandowsky, Mann and Gleick, the 2012 AGU convention was the wrong place to start my trip. All three were prominent at the convention. Continue reading

Checking in and travel plans

As readers have noticed, I’ve been tuned out for a few weeks. No single reason.

I did a considerable amount of fresh work on issues related to Hurricane Sandy, but found them hard to reduce to a few posts. So I’ve got topics in inventory.

I also had a bout of periodic weariness. I just turned 65 and continue to tire more quickly than I used to. Various leg injuries have contributed to a decline in fitness as well. Nor is it easy to continue to muster enthusiasm for analysis of dreck from people like Mann, Lewandowsky, Gleick, Gergis, Briffa, Jones etc. Wading into WG2 is even worse.

I’ve also been busy on some business issues. Trelawney Mining got taken over last June (it was a considerable success; our first property visit was mentioned in a CA post a few years ago) and we’re working on a new venture. One of my sons is involved in small-cap mining as well and I’ve been helping him as well. I find it very hard to focus on more than one thing at once.

Also, my wife and I are going to visit our daughter in New Zealand and our son in Thailand over Christmas. I’m going to be away for about 4 weeks. The flight to New Zealand goes from San Francisco so I’m going to go to AGU this year, starting tomorrow. And will try to blog on anything interesting.

BBC’s “Best Scientific Experts”

There is an unusual story developing as a result of an ongoing FOI request from Tony Newbery, some excellent detective work by Maurizio Morabito – see discussion at Bishop Hill here. Also see context from Andrew Orlowski here.

Several years ago, the BBC stated in a report:

The BBC has held a high-level seminar with some of the best scientific experts, and has come to the view that the weight of evidence no longer justifies equal space being given to the opponents of the consensus [on anthropogenic climate change].

Tony Newbery (see Harmless Sky blog) was curious as to the identity of these “scientific experts”, and filed a Freedom of Information Act request. Rather than simply complying with the request, the BBC refused the request. Tony appealed to the ICO and lost. The ICO agreed that the BBC was a “public authority” but held that the information was held “for journalistic purposes” and exempt:

The Commissioner is satisfied that in view of the fact that the purpose of the seminar was to influence the BBC’s creative output, the details requested about its organisation, contents, terms of reference and the degree to which it impacted upon changes to Editorial Standards by BBC News constitute information held by the BBC to a significant extent for the purposes of art, literature or journalism. Information about the content of the seminar was used to shape editorial policy and inform editorial decisions about the BBC’s coverage and creative output. The details about the arrangements for the seminar are held to facilitate the delivery of the event and to ensure that the appropriate people were in attendance.

Tony appealed to the Information Tribunal. The BBC appeared with six lawyers. BBC official Helen Boaden argued that the meetings had been held under Chatham House rules and that the identity of the participants was therefore secret. Tony was again given short shrift, with the members of the Tribunal being surprisingly partisan, as reported by Orlowski.

Out of left field, Maurizio located the information on the Wayback machine here. Rather than the participants being the “best scientific experts” as claimed, they were almost entirely NGO activists. And rather than the meetings being held under Chatham House rules as Boaden had told the tribunal, seminar co-sponsor IBT had published the names of attendees of the meeting, describing the purpose of the meetings as follows:

The BBC has agreed to hold a series of seminars with IBT, which are being organized jointly with the Cambridge Media and Environment Programme, to
discuss some of these issues.

The document located by Maurizio includes names from other meetings as well. The names are presently being fisked at Bishop Hill and Omnologos.

For the record, I do not share the visceral disdain for the BBC coverage of most commentators at Bishop Hill. I am not exposed to BBC regular programming and my own experience with the BBC (mostly arising from Climategate) has been constructive. I thought that their recent reprise on Climategate was as balanced as one could expect. I also think that their original coverage of Climategate was fair under the circumstances. While Roger Harrabin approached Climategate from a green perspective (something that does not trouble me – indeed, on a personal level, I like most green reporters), in my opinion, he treated his obligations as a reporter as foremost in his Climategate coverage, and, as a result, his coverage of Climategate was balanced. Indeed, I think that one of the reasons that he was particularly troubled by the Climategate conduct and dissatisfied by the “inquiries” may well have been the inconsistency between the Climategate attitudes in private and the public posture of green organizations in the seminars that were the subject of Newbery’s FOI.

Update: Ironically, Harrabin is not listed as an attendee at the Jan 2006 conference on climate change that was the subject of the OI request (though he attended other conferences and was involved in starting the seminar program.) Further update – however, other information indicates that he was at this conference and that the list is in error on this point.

Update: The non-NGO “experts” were Robert May (a population biologist and former Royal Society president), Mike Hulme of East Anglia, Dorthe Dahl-Jensen of the Niels Bohr Institute in Denmark (an ice core specialist), Michael Bravo of Cambridge (a specialist in the history of Antarctic exploration and public policy), Joe Smith of the Open University (active in BBC science progamming), Poshendra Satyal Pravat, Open University, who was then doing a PhD in theories of social and environmental justice and Eleni Andreadis of the Harvard Kennedy School (public policy). Virtual no representation from climate science.

Harvard-Kennedy School Class of 2006: One of BBC’s scientific experts at the 2006 meeting was Eleni Andreadis, then studying at the Harvard Kennedy School. She made a short film of interviews with HKS graduates (see here here).

Another member of the Harvard-Kennedy class of 2006 is very much in the news today: Paula Broadwell was also a student at the Harvard Kennedy School in 2006, where she met David Petraeus after a lecture.

Nic Lewis on Statistical errors in the Forest 2006 climate sensitivity study

Nic Lewis writes as follows (see related posts here, here)

First, my thanks to Steve for providing this platform. Some of you will know of me as a co-author of the O’Donnell, Lewis, McIntyre and Condon paper on an improved temperature reconstruction for Antarctica. Since then I have mainly been investigating studies of equilibrium climate sensitivity (S) and related issues, since climate sensitivity lies at the centre of the AGW/dangerous climate change thesis. (The equilibrium referred to is that of the ocean – it doesn’t include very slow changes in polar ice sheets, etc.) Obviously, the upper tail of the estimated distribution for S is important, not just its central value.

People convinced as to the accuracy of AO-GCM (Atmosphere Ocean General Circulation Model) simulations may believe that these provide acceptable estimates of S, but even the IPCC does not deny the importance of observational evidence. The most popular observationally-constrained method of estimating climate sensitivity involves comparing data whose relation to S is too complex to permit direct estimation, such as temperatures over a spatio-temporal grid, with simulations thereof by a simplified climate model that has adjustable parameters for setting S and other key climate properties. The model is run at many different parameter combinations; the likelihood of each being the true combination is then assessed from the resulting discrepancies between the modelled and observed temperatures. This procedure requires estimates of the natural spatio-temporal covariability of the observations, which are usually derived from AO-GCM control runs, employing an optimal fingerprinting approach. A Bayesian approach is then used to infer an estimated probability density function (PDF) for climate sensitivity. A more detailed description of these methods is given in AR4 WG1 Appendix 9B, here.

I have concentrated on the Bayesian inference involved in such studies, since they seem to me in many cases to use inappropriate prior distributions that heavily fatten the upper tail of the estimated PDF for S. I may write a future post concerning that issue, but in this post I want to deal with more basic statistical issues arising in what is, probably, the most important of the Bayesian studies whose PDFs for climate sensitivity were featured in AR4. I refer to the 2006 GRL paper by Forest et al.: Estimated PDFs of climate system properties including natural and anthropogenic forcings, henceforth Forest 2006, available here, with the SI here. Forest 2006 is largely an update, using a more complete set of forcings, of a 2002 paper by Forest et al., also featured in AR4, available here, in which a more detailed description of the methods used is given. Forest 2006, along with several other climate sensitivity studies, used simulations by the MIT 2D model of zonal surface and upper-air temperatures and global deep-ocean temperature, the upper-air data being least influential. Effective ocean diffusivity, Kv, and total aerosol forcing, Faer, are estimated simultaneously with S. It is the use of multiple sets of observational data, combined with the joint estimation of three key uncertain climate parameters, that makes Forest 2006 stand out from similar Bayesian studies.

Forest completely stonewalled my requests to provide data and code for over a year (for part of which time, to be fair, he was recovering from an accident). However, I was able to undertake a study based on the same approach as in Forest 2006 but using improved statistical methods, thanks to data very helpfully made available by the lead authors, respectively Charles Curry and Bruno Sanso, of two related studies that Forest co-authored. Although Forest 2006 stated that the Curry et al. 2005 study used the Forest 2006 data (and indeed relied upon that study’s results in relation to the surface dataset), the MIT model surface temperature simulation dataset for Curry et al. 2005 was very different from that used in the other study, Sanso et al. 2008. The Sanso et al. 2008 dataset turned out to correspond to that actually used in Forest 2006. The saga of the two conflicting datasets was the subject of an article of mine posted at Judith Curry’s blog Climate Etc this summer, here , which largely consisted of an open letter to the chief editor of GRL. Whilst I failed to persuade GRL to require Forest to provide any verifiable data or computer code, he had a change of heart – perhaps prompted by the negative publicity at Climate Etc – and a month later archived the complete code used for Forest 2006, along with semi-processed versions of the relevant MIT model, observational and AO-GCM control run data – the raw MIT model run data having been lost. Well done, Dr Forest. The archive (2GB) is available at http://bricker.met.psu.edu/~cef13/GRL2006_reproduce.tgz .

The code, written in IDL, that Forest has made available is both important and revealing. Important, because all or much of it has been used in many studies cited, or expected to be cited, by the IPCC. That includes, most probably, all the studies based on simulations by the MIT 2D model, both before and after AR4. Moreover, it also includes a number of detection and attribution studies, the IPCC’s “gold standard” in terms of inferring climate change and establishing consistency of AO-GCM simulations of greenhouse gas induced warming with observations. Much of the core code was originally written by Myles Allen, whose heavily-cited 1999 Allen and Tett optimal fingerprinting paper, here, provided the statistical theory on which Forest 2006 and its predecessor and successor studies were based. Allen was a co-author of the Forest et al. 2000 (MIT Report preprint version of GRL paper here), Forest et al. 2001 (MIT Report preprint version of Climate Dynamics paper here) and Forest et al. 2002 studies, in which the methods used in Forest 2006 were developed.

The IDL code is revealing because it incorporates some fundamental statistical errors in the derivation of the likelihood functions from the model-simulation – observation discrepancies. The errors are in the bayesupdatenew2.pro module (written by Forest or under his direction, not by Allen, I think) that computes the relevant likelihood function and combines it with a specified initial distribution (prior) using Bayes theorem. There are three likelihood functions involved, one for each of the three “diagnostics” – surface, upper-air, and deep-ocean, which involve respectively 20, 218 and 1 observation(s). The bayesupdatenew2 module is therefore called (by module bu_lev05.pro) three times, if an “expert” prior is being used. Where the prior used is uniform, on the first pass bayesupdatenew2 also computes a likelihood for a second set of discrepancies and uses that as a “data prior”, so the module is only called twice.

Each likelihood is based on the sum of the squares of the ‘whitened’ model-simulation – observation discrepancies, r2. Whitening involving transforming the discrepancies, using an estimate of the inverse spatio-temporal natural variability covariance matrix, so that they would, in a perfect world, be independent standardised normal random variables. The likelihoods are computed from the excess, delta.r2, of r2 over its minimum value, minr2 (occurring where the model run parameters provide the best fit to observations), divided by m, the number of free model parameters, here 3. The statistical derivation implies that delta.r2/m has an F_m,v statistical distribution, in this case that delta.r2/3 has an F_3,v distribution, v being the number of degrees of freedom in estimating the spatio-temporal natural variability covariance matrix. The math is developed in Forest et al. 2000 and 2001 out of that in Allen and Tett 1999, and I will not argue here about its validity.

The statistical errors I want to highlight are as follows:

(a) the likelihoods are computed using (1 minus the) cumulative distribution function (CDF) for a F_3,v(delta.r2/3) distribution, rather than its probability density function. A likelihood function is the density of the data, viewed as a function of the parameters. Therefore, it must be based on the PDF, not the CDF, of the F distribution. The following code segment in bayesupdatenew2 incorporates this error:

r2 = r2 – minr2 +1e-6
nf = 3
pp= 1.- f_pdf(r2/float(nf),nf,dof1)

where dof1 is the degrees of freedom used for the diagnostic concerned. Despite the name “f_pdf”, this function gives the F-distribution CDF.

(b) the same calculation, using the F_3,v(delta.r2/3) function, is used not only for the surface and upper-air diagnostics, but also for the univariate deep-ocean diagnostic. The use of the F_3,v distribution, with its argument being delta.r2/3, is based on delta.r2 being, when the model parameter settings are the true ones, the sum of the squares of 3 independent N(0,1) distributed random variables. That there are only 3 such variables, when the diagnostic involves a larger number of observations and hence whitened discrepancies, reflects the higher dimensional set of whitened discrepancies all having to lie on a 3D hypersurface (assumed flat) spanned by the parameters. However, when there is only one data variable, as with the deep-ocean diagnostic (being its temperature trend), and hence one whitened discrepancy, delta.r2 is the square of one N(0,1) variable, not the sum of the squares of three N(0,1) variables. Therefore, the deep-ocean delta.r2, divided by 1 not 3, will have a F_1,v distribution – implying the unsquared whitened deep ocean discrepancy r will have a Student’s t_v distribution. Here, delta.r2 = r2, since a perfect fit to a single data point can be obtained by varying the parameters, implying minr2 = 0.

(c) there is a statistical shortcoming in the Forest et al 2006 method in relation to the use of an F-distribution based PDF as a likelihood function. A geometrical correction to the F-distribution density is need in order to convert it from a PDF for the sum of the squares of three N(0,1) distributed variables to a joint PDF for those three variables. The appropriate correction, which follows from the form of the Chi-square distribution PDF, is division of the F-distribution PDF by sqrt(delta.r2). Without that correction, the likelihood function goes to zero, rather than to a maximum, at the best-fit point.

There may be many other errors in the IDL code archive – I’ve identified a couple. Any readers who are familiar with IDL and have the time might find it interesting to study it, with a view to posting any significant findings – or even to rewriting key parts of it in R.

As it happens the F_3,v PDF is not that different from {1 – CDF} once the parameter combination is well away from the best fit point, so the effect of error (a) is not very substantial. Nevertheless, that Forest made this error – and it was not a mis-coding – seems very surprising.

The effect of (c), which is partially compensated by error (a), is likewise not very substantial.

However, the difference in the behaviour of the likelihood function resulting from error (b) is substantial; the ratio of the Forest 2006 to the correct likelihood varies by approaching 3x as the parameters are moved away from the best fit point. And the deep-ocean likelihood is what largely causes the estimated PDF for S ultimately to decline with increasing S: the two other diagnostics provide almost no constraint on very high values for S.

In addition to the Forest 2002 and 2006 papers, I believe these errors also affected the Forest et al. 2008 Tellus A and the Libardoni and Forest 2011 GRL papers, and probably also 2009 and 2010 papers lead authored by Forest’s regular co-author Sokolov. It is to be expected that there will be multiple citations of results from these various studies in the AR5 WG1 report, . I put it to Myles Allen – who seems, along with Gabi Hegerl, to be the lead author of Chapter 10 primarily responsible for the sections relating to climate sensitivity – that in view of these serious statistical errors, results from the affected papers should not be cited in the IPCC report. However, whilst accepting that the errors were real, he expressed the view that the existence of these statistical errors didn’t really matter to the results of the papers concerned. His reasoning was that only error (b) had a potentially substantial effect, and that didn’t much matter since there was anyway considerable uncertainty in the ocean data that the studies used. I’m not sure that I agree with this approach.

I would be surprised if the basic statistical errors in the IDL code do not significantly affect the results of some or all of the papers involved. I would like to test this in regard to the Forest 2006 paper, by running the IDL code with the errors corrected, in time to put on record in my “expert reviewer” comments on Chapter 10 of the Second Order Draft of IPCC AR5 WG1 report what the differences in Forest 2006’s results resulting from correcting these errors are, if significant. At least Myles Allen and Gabi Hegerl will then be aware of the size of the differences when deciding whether to ignore them.

Unfortunately, IDL seems to be a very expensive product – the company selling it won’t even give any pricing information without many personal details being provided! There is an open source clone, GDL, which I have tried using, but it lacks too much functionality to run most of the code. I’ve implemented part of the IDL code in R, but it would take far too long to convert it all, and I couldn’t be confident that the results would be correct.

So, my question is, could any reader assist me by running the relevant IDL code and letting me have the results? If so, please can you email me via Steve M. The code in the GRL2006_reproduce archive should be fully turnkey, although it is written for an old version of IDL. I can supply an alternative, corrected version of the bayesupdatenew2.pro module and relevant information/instructions.

In case any of you are wondering, I submitted a paper to Journal of Climate in July detailing my reanalysis of the Forest 2006 datasets, focussing on improving the methodology, in particular by using an objective Bayesian approach. That was just before Forest released the IDL code, so I was unaware then that he had made serious statistical errors in implementing his method, not that they directly affect my paper.

Nicholas Lewis

For the convenience of readers who would like to look at the bayesupdatenew2.pro code without downloading a 2GB archive file, it is as follows:

pro bayesupdatenew2,prior,data,post,expert=expert,hik=hik,mtit=mtit,usegcms=usegcms,alf=alf,dt=dt,indiv=indiv,yrng=yrng,$

if (n_params() eq 0) then begin
  print, 'Usage: bayesupdatenew2,prior,newp,post,expert=expert,$'
  print, '   hik=hik,mtit=mtit,usegcms=usegcms,alf=alf,dt=dt,$'
  print, '   indiv=indiv,yrng=yrng,label=label,dataprior=dataprior,$'
  print, '   dof1=dof1,dof2=dof2,noplot=noplot,igsm2=igsm2,mcmc=mcmc,i2post=i2post'
  print, ' prior = a priori estimate of pdf'
  print, ' data = r2 data used to estimate new pdf'
  print, ' post = a posteriori estimate of pdf'
  print, ' dataprior = 1 if using r2 values for first prior'
  print, ' dof1, dof2 = degreees of freedom for likelihood estimators'
  print, ' dof2 = used for dataprior'
  print, 'i2post = 1, use igsm2 posterior for aerosol prior'

if (not keyword_set(yrng)) then yr = [0,10.] else yr = yrng
if (not keyword_set(dof1)) then dof1 = 12
if (not keyword_set(dof2)) then dof2 = 12 

;set colors

; prior - taken from somewhere old posterior or expert judgement
; data - provided by new diagnostic
; post - returned by updating procedure

kk= findgen(81)*.1 & ss = findgen(101)*.1 +.5 & aa = -findgen(41)*.05+0.5

np = n_elements(data)
;print, 'np = ',np

; create probability from r2 values
r2 = data
pp = fltarr(np)

r2neg = where(r2 le 0., nu)
print, 'Number of negative points=', nu

if (nu ne 0) then r2(r2neg) = 0.

if (keyword_set(igsm2)) then begin
  minr2 = min(r2(1:50,0:75,0:40)) 
  print, 'Minrp2 in igsm2 domain:',minr2
endif else minr2 = min(r2)

print, 'minr2 =',minr2
print, 'Range r2:',range(r2)

r2 = r2 - minr2 +1e-6
nf = 3
;dof = 12
;for i = 0,np-1 do pp(i)=  1.- f_pdf(r2(i)/float(nf),nf,dof)
print, 'Computing p(r2) for HUGE data vector'
pp=  1.- f_pdf(r2/float(nf),nf,dof1)
help, dof1


;interpolate prior to r2 points
; note: no prior for aa = alpha
if (keyword_set(expert)) then begin

; returns 3d joint prior on r2interp grid
;  priorx = krange & priory = srange & priorz = jprior = jointprior
endif ;else begin
;  priorx = kk & priory = ss & priorz = prior 

if (not keyword_set(expert) and  keyword_set(dataprior)) then begin
    r2p = prior 
    r2pneg = where(r2p le 0., nup)
    print, 'Number of negative points=', nup
    if (nup ne 0) then r2p(r2pneg) = 0.

    print, 'Range(r2p)', range(r2p)
    if (keyword_set(igsm2)) then begin
      print, 'Keyword set: igsm2'
      minr2p = min(r2p(1:50,0:75,0:40)) 
      print, 'minr2p ',minr2p
    endif else minr2p = min(r2p)
    r2p = r2p - minr2p + 1e-6
    print, 'Computing p(r2) for HUGE prior vector'

    prior2 =  1.- f_pdf(r2p/float(nf),nf,dof2)
    print,'Range prior2 before ', range(prior2)

    help, dof2
    if (keyword_set(igsm2)) then begin
      prior2(0,*,*) = 0.        ;KV = 0.
      prior2(51:80,*,*) = 0.    ;KV > 25.
;      prior2(*,76:100,*) = 0.   ;CS > 8.
      prior2(*,76:100,*) = 0.   ;CS > 8.
;      prior2(*,*,0:4) = 0.      ;FA > 0.25 
    endif else begin
      prior2(0,*,*) = 0.
      prior2(*,96:100,*) = 0.
endif else prior2 = prior


; multiply probabilities
post = prior2 * pp

; interpolate to finer grid to compute integral
; separate into aa levels
nk = findgen(81)*.1
nk = nk*nk
ns = findgen(101)*.1 + 0.5

; estimate integral to normalize

ds = 0.1 & dk = 0.1 & da = 0.05
;totpl = fltarr(6) ; totpl = total prob at level aa(i)
;for i =0,5 do totpl(i) = total( smpostall(i,*,*) )/(8.*9.5*2.0)   

;where intervals are k=[0,8], s=[0.5,10.], a=[0.5,-1.5]
totp = total(post)/(8.*9.5*2.)


;normalize here
post = post/totp
;print, post

smpostall = post

if (not keyword_set(noplot)) then begin 

;estimate levels for contour from volume integral
clevs = c_int_pdf3d([0.99,.9,.8],smpostall)
;clevs = c_int_pdf3d([0.90],smpostall)
rr= range(smpostall)
print, 'range post:', rr(1) -rr(0)
;clevs = findgen(3)*(rr(1) - rr(0))/4+min(smpostall)
print,' clevs =', clevs
if (not keyword_set(indiv)) then !p.multi=[0,2,4] else !p.multi=0
pmax = max(post)
;print,'max(post), imax', max(post), where(post eq max(post))
ccols = [indgen(3)*50 + 99,255]

pane = ['(a) : ','(b) : ','(c) : ','(d) : ','(e) : ','(f) : ','(g) : ']
titl= ['F!daer!n = 0.5 W/m!u2!n','F!daer!n = 0.0 W/m!u2!n','F!daer!n = -0.25 W/m!u2!n',$
       'F!daer!n = -0.5 W/m!u2!n','F!daer!n = -0.75 W/m!u2!n',$
       'F!daer!n = -1.0 W/m!u2!n','F!daer!n = -1.5 W/m!u2!n']

alevs = [0,10,15,20,25,30,40]
for i = 0,6 do begin
  ii =  alevs(i)
  ka = nk & sa = ns

  contour, post(*,*,ii), sqrt(ka), sa, $
  levels=clevs,c_labels=0,/cell_fill, c_colors=ccols, $
  title=pane(i)+mtit+' : '+titl(i),$
  xtitle='Effective ocean diffusivity [cm!u2!n/s]',$
  ytitle='Climate Sensitivity [!uo!nC]', $
  xticks = 8, xtickv=findgen(9),$
  xtickname=strmid((findgen(9))^2,6,4); , charsize=chsz

  contour, post(*,*,ii), sqrt(ka), sa,/overplot, $
  imax = where(post(*,*,ii) eq pmax, icount)
  ix = where(post(*,*,ii) eq max(post(*,*,ii)))
;  print, imax, ix
  if (icount ne 0) then oplot,[sqrt(ka(imax))],[sa(imax)],psym=sym(1) else $
  oplot, [sqrt(ka(ix))],[sa(ix)], psym = sym(6)
;  for j=0,ni-1 do  oplot, [sqrt(ka(j))],[sa(j)], psym = sym(11)
  if (keyword_set(usegcms)) then begin
    if (keyword_set(label)) then $

  if (keyword_set(dt)) then begin
      dtlabs = replicate(1,31)
      dtlevs =  findgen(31)/20.
      dr =  range(dtdata.dt)
      ddr = dr(1) - dr(0)
      if (ddr lt 0.5) then dtlevs = findgen(31)/50.
      contour,dtdata.dt,sqrt(dtdata.k),dtdata.s,/overplot, levels=dtlevs,$ 
        c_labels=dtlabs, thick=5
      contour,dtdata.dt,sqrt(dtdata.k),dtdata.s,/overplot, levels=[obs],$ 
        c_labels=dtlabs, thick=5, c_linestyle=5

if (keyword_set(alf)) then begin
  na = -findgen(41)*.05+0.5
  set_plot, 'ps'
  device, /encapsulated, /helvetica, font_size=18
  device,/color,bits_per_pixel=8,xsize=20, ysize=5./7.*20
  i = where( na eq alf, nl)
  i = i(0)
  if (nl lt 1) then print, 'No matching aerosol forcing' else begin
    ii = where( na eq alf,ni)
    ka = nk & sa = ns
    contour, post(*,*,ii), sqrt(ka), sa,$
    xtitle='Rate of Ocean Heat Uptake [Sqrt(K!dv!n)]',ytitle='Climate Sensitivity (K)',title=mtit+' : '+titl(i),$
    levels=clevs,c_labels=0,/cell_fill, c_colors=ccols
    contour, post(ii), sqrt(ka), sa,/irregular,/overplot, $
    if (keyword_set(usegcms)) then begin
;    xyouts,sqrt(gcms(0,*))+.15,gcms(1,*),nms,charsize=chsz

    if (keyword_set(dt)) then begin
      dtlabs = replicate(1,31)
      dtlevs =  findgen(31)/20.
      dr =  range(dtdata.dt)
      ddr = dr(1) - dr(0)
      if (ddr lt 0.5) then dtlevs = findgen(31)/50.
      contour,dtdata.dt,sqrt(dtdata.k),dtdata.s,/overplot, levels=dtlevs,$ 
        c_labels=dtlabs, thick=5
      contour,dtdata.dt,sqrt(dtdata.k),dtdata.s,/overplot, levels=[obs],$ 
        c_labels=dtlabs, thick=5, c_linestyle=5


  device, /close,color=0,encapsulated=0
  set_plot, 'x'
  !p.font = -1



“Olympic Mann”

There has been much publicity about Michael Mann’s claims to have been awarded a share of the Nobel Peace Prize. Somewhat overlooked in the excitement about “Nobel” Mann were the accomplishments of “Olympic Mann” at multiple Olympics, celebrated in Josh’s cartoon at left showing “Olympic Mann” in an iconic pose.

In Mann’s lawsuit against National Review, Mann accused them of defamation of a “Nobel peace recipient”. National Review recently honored Mann’s “award” of a Nobel Peace Prize with a full page ad in the Penn State student newspaper (h/t WUWT here.)

With no sacrifice in accuracy, Mann could additionally have accused National Review of defaming an “Olympic gold medalist.”

BBC Radio 4 on Climategate


Karoly and Gergis vs Journal of Climate

On June 10, a few days after the Gergis-Karoly-Neukom error had been identified, I speculated that they would try to re-submit the same results, glossing over the fact that they had changed the methodology from that described in the accepted article. My cynical prediction was that a community unoffended by Gleick or upside-down Mann would not cavil at such conduct.

The emails http://www.climateaudit.info/correspondence/foi/gergis/Part%202a%20Journal%20Correspondence.pdf show that Karoly and Gergis did precisely as predicted, but Journal of Climate editors Chiang and Broccoli didn’t bite. Most surprising perhaps was that Karoly’s initial reaction was agreement with the Climate Audit criticism of ex post correlation screening. However, when Karoly realized that the reconstruction fell apart using the methodology of the accepted article, he was very quick to propose that they abandon the stated methodology and gloss over the changes. In today’s post, I’ll walk through the chronology. Continue reading

Gergis et al Correspondence

Michael Kottek writes in the comment section:

The results of my FOI request to the University of Melbourne can be seen here:


I requested all correspondence between the authors and the journal regarding the paper. The referees reports were exempted as were documents relating to the resubmitted paper.

I also requested correspondence between the authors after the paper was accepted. Once again emails relating to the resubmitted paper were exempted, and personal material redacted.

I note that emails regarding the paper that were received by one author and not forwarded to the others would not have been covered by my request.

Despite the embarrassment of the withdrawn paper, the University is to be commended for their no nonsense approach to this request. As an alumunus, I am pleased that the response is far more sensible than the approach taken by the UEA and UVa.

Steve: Oct 28, 9 pm Eastern
Here is a more detailed commentary which raises questions about Karoly’s claim that they had “independently” discovered the screening error on June 5. [Note: times in the emails are in multiple time zones. In the analysis below, it is my understanding that in June 2012, relative to UTC, Melbourne time was +10, Switzerland +2, Eastern -4, CA blog time -5.]

As CA readers are aware, the issue of screening in Gergis et al 2012 was first raised in a CA post in a May 31 blog post, a discussion that directly quoted the following paragraph of Gergis et al:

Our temperature proxy network was drawn from a broader Australasian domain (90E–140W, 10N–80S) containing 62 monthly–annually resolved climate proxies from approximately 50 sites (see details provided in Neukom and Gergis, 2011)… Only records that were significantly (p<0.05) correlated with the detrended instrumental target over the 1921–1990 period were selected for analysis. This process identified 27 temperature-sensitive predictors for the SONDJF warm season (Figure 1 and Table 1) henceforth referred to as R27.

The CA discussion, and, in particular, the Name and Shame blog post, was referred to on numerous occasions in the internal emails among Gergis, Neukom and others over the next few days, commencing almost immediately with an email from Gergis to Neukom and other coauthors we follows:

We should all be aware that this is unfolding: https://climateaudit.org/2012/05/31/myles-allen-calls-for-name-and-shame

In my original post, I had presumed that Gergis et al had used correlation screening against trending series (which I termed the “Screening Fallacy”), a topic discussed on a number of occasions at critical climate blogs (see references in original post.) Against this, Jim Bouldin and others argued that Gergis et al had employed detrending screening, thereby avoiding the CA criticism. The CA discussion quickly led to Jean S checking the correlations of the available series. On June 5 (blogtime 16:42; 07:42 June 6 in Melbourne), Jean S reported in CA comments here that Gergis’ claim by Gergis to have used detrended correlations was false (asking me and others to check). Jean S’ comment almost immediately (within an hour) attracted online notice from Hu McCulloch (blog 17:39) and Kenneth Fritsch (blog 18:19).

About two hours after Jean S’ post – by now nearly 2 am in Switzerland (June 6 09:46 Melbourne; 01:46 Switzerland; blog 18:46 June 5), Neukom urgently notified his Australian associates of the same problem that Jean S had reported at CA a couple of hours earlier. Neukom had a skype discussion with Gergis, followed up by an email (2Gergis, page 77) to Gergis, Karoly and others. Neukom noted that the mistake was related to the proxy screening (then under discussion at Climate Audit) and thus a “delicate issue”:

As just discussed with joelle on skype, I found a mistake in our paper in journal of climate today. It is related to the proxy screening, so it is a delicate issue. In the paper we write that we do the correlation analysis for the screening based on detrended (instrumental and proxy) data, but in reality we did not use detrended data.

Meanwhile at CA (blog time June 5 20:11; Melbourne 11:11), CA reader HaroldW reported that he had confirmed Jean S’ results. I checked in at CA with a question to Jean S (blog time 20:49 June 5; Melbourne 11:49).

June 7 (Australia time)
The following morning (10 am June 6 blogtime; June 7 01:00 Melbourne), I reported that I had confirmed Jean S’ results, posting the discussion as a fresh post a little later (12:01 June 6 blog time; 03:01 June 7 Melbourne).

In the evening of June 6 (Switzerland; 05:56 Melbourne), Neukom wrote Karoly with his assessment, expressing his desire to discuss matters with Karoly the following day. Karoly wrote back to Neukom (Melbourne 06:48; Switzerland June 6 22:48) urging use of detrended .

I think that it is much better to use the detrended data for the selection of proxies, as you can then say that you have identified the proxies that are responding to the temperature variations on interannual time scales, ie temp-sensitive proxies , without any influence from the trend over the 20th century . This is very important to be able to rebut the criticism is that you only selected proxies that show a large increase over the 20th century ie a hockey stick.

The same argument applies for the Australasian proxy selection. If the selection is done on the proxies without detrending ie the full proxy records over the 20th century, then records with strong trends will be selected and that wi ll effectively force a hockey stick result. Then Stephen Mcintyre criticism is valid. I think that it is really important to use detrended proxy data for the selection, and then choose proxies that exceed a threshold for correlations over the calibration period for either interannual variability or decadal variability for detrended data.

A little later in the Melbourne morning of June 7 (08:03; Switzerland June 7 00:03), Gergis asked Neukom whether he was “250% certain” of the problem. The emails are then surprisingly quiet through the rest of June 7 (Australia).

June 8 (Australia)
Early in the Australian morning of June 8 (06:47), Karoly emailed Neukom and others, referring them to the CA post of about 27 hours earlier (12:01 June 6 blog time; June 7 03:01 Australia; June 6 19:01 Switzerland).

Someone has now tried to reproduce the screening of the 27 selected proxies against the target Australasian temp series and is unable to reproduce the claimed results in the paper. https://climateaudit.org/2012/06/06/gergis-significance/. I suggest that you look at this Stephen Mcintyre post. Given that the error is now identified in the blogosphere, we need to notify the journal of the error and put the manuscript on hold

Although the CA post had cited Jean S’ results of June 5, Karoly disregarded these links back to the original provenance.

Gergis (2G:37, p 112; 2K:31) acknowledged Karoly’s email almost immediately (07:26 Melbourne). Gergis then (08:24 Melbourne; 00:24 Swiss) wrote to Neukom, but did not copy Karoly or other coauthors; Gergis argued to Neukom that they had emails showing that they “became aware of the issue” prior to the “latest blogpost” because they had “contacted authors for permission to release their records”:

Hi Raphi, we have emails that predate this latest blogpost that indicate we became aware of the issue as we contacted authors for permission to release their records

CA readers will recognize that Gergis here is sliding over a couple of issues: they had only asked authors to release records because of the May 31 CA blogpost in which screening had already been made an issue; and secondly, the results in the June 6 CA blogpost merely reported (and linked to) Jean S’ results reported in comments on June 5, a day earlier.

Neukom wrote back (08:26 Melbourne; 00:26 Swiss) warning his coauthors that caution needed to be taken with detrended correlations. A little later (08:42 Melbourne; 00:42 Swiss), Neukom sent Gergis a reconstruction with the (only) eight proxies that passed detrended correlation. Karoly noted (08:54 Melbourne; 00:54 Swiss) that some of the correlations were now flipped.

Throughout the rest of June 8, Karoly and Gergis started notifying others of the problem. First (10:38 Melbourne; 02:38 Swiss), Gergis (2K:34; page 73) notified coauthors Gallant and Phipps (cc Neukom) of the problem. In this first notice, Gergis said that Neukom had identified the problem on the morning of June 6 ( a time that corresponded to Neukom’s original email, which had been received in Melbourne at 09:42 June 6 (01:42 Switzerland):

Following on from my attempt to gain permission to release non publically available records released and submitted online with NOAA over the weekend, on Wednesday [June 6] morning Raphi discovered an error in the Aus2K temperature analysis….

Meanwhile, Stephen Mclntyre and co have located the error overnight (I was alerted through an intimidating email this morning): https://climateaudit.org/2012/06/06/gergis-significance . So instead of this being a unwanted but unfortunately normal part of science, we are likely to have an extremely negative online commentary about our work. Although it was an unfortunate data processing error, it does have implications for the results of the paper. We wish to alert you to this issue before the paper goes into final production.

Gergis had inaccurately notified her co-authors that “McIntyre and co have located the error overnight [June 8]”. In fact, the error had been identified at Climate Audit nearly two days earlier. Although Gergis refers here to a supposedly “intimidating email” (and uses the same phrase to Journal Climate later that day), no such email is included in the FOI emails . Nor did I send her any such email.

At 11:16, Gergis sent Karoly a draft notice letter to the Journal of Climate. Karoly reverted at 11:47 with his edits presumably those shown in the redlined version (2K:35, page 75). This version also reported their discovery of the error as occurring on June 6:

While attempting to release non-publicly available records used in our study with NOAA over the weekend [June 2-3], our team discovered an error in our paper.. .
When we went to recheck this on Wednesday [June 6], we discovered that the records used in the final analysis were not detrended for proxy selection, making this statement incorrect.

Soon afterwards (12:35 Melbourne), Gergis sent a revised notice letter to Journal of Climate. In the revised letter, the discovery date was now said to be Tuesday, June 5 rather than Wednesday June 6 of the earlier letter to coauthors and the draft only an hour earlier. They also inaccurately told Journal of Climate that Climate Audit had identified the error “overnight [June 8]”, more than two days after the actual time of the original report. Their letter stated:

While attempting to release non-publicly available records used in our study with NOAA this week, our team discovered an error in our paper….

When we went to recheck this on Tuesday [June 5], we discovered that the records used in the final analysis were not detrended for proxy selection, making this statement incorrect…

Meanwhile, independently of our team’s detection of this error, prominent climate change blogger Stephen Mclntyre has identified the issue overnight (I was alerted through an intimidating email this morning): http:l/climateaudit.org/2012/06/06/gergis-significance. So instead o(this being a unwanted but unfortunately normal part of science, we are likely to have an extremely negative online commentary about our work and possibly the journal.)

Gergis sent a near identically worded notice to PAGES 2K at 14:19, again saying that they had discovered the error on June 5 (Tuesday), adding a warning to the PAGES 2K consortium that they might have to archive all their data”

In terms of the consortium paper, please run with the current version of the Aus2K temperature reconstruction but please note that it may change in coming weeks…

They are now demanded that the full network of records be made available. Over the past week I have been busy contacting authors of non publically available records that were not used in the final temperature reconstruction to attempt to release their data. Everyone managed to agree on just the C20th portions used for calibration be released, but some still no not want to make their full records available.

This issue has implications for other 2K groups: ANY mention of proxy ‘screening’ or selection criteria is likely to be heavily criticised . Although we attempted to be transparent about our methodology, this has backfired and caused a lot of trouble. I just thought you should be aware that it may not be enough that only the records used in the final analysis are already available. It is possible that every record from every region {those rejected from the analysis and those used in final reconstructions) will need to be made available once the consortium paper is published.elp benefit the broader group.

During the Melbourne afternoon, Karoly worked with University of Melbourne public relations staff on a statement, sending a draft to Gergis and others at 15:57 (2K:38); Gergis reverted at 16:17. This statement adopted the date of “Tuesday 5 June” as the date on which the error was discovered:

While the paper states that “both proxy climate and instrumental data were linearly detrended over the 1921-1990 period”, it was discovered on Tuesday 5 June that the records used in the final analysis were not detrended for proxy selection, making this statement incorrect. Although this is an unfortunate data processing issue, it is likely to have implications for the results reported in the study. The journal has been contacted and the publication of the study has been put on hold.

At 17:56 Melbourne (09:56 Swiss), Karoly sent Neukom a short and long version of their statement. In the long version, they neutrally said that the error was discovered on “Tuesday June 5” (without attribution); no date was mentioned in the short statement. Karoly said that they planned to send a statement to me containing the above paragraph. Neukom reverted immediately (18:18 Melbourne), suggesting that they include the date in the short statement: (2K, 168).

Maybe we can include the date when we discovered the error also in the short statement so that it is clear that we did not just do it as a reaction to the Mclntyre blog?…

And I will try to write down everything that happened in the correct chronological order to be sure l can recall this all correctly. Because I think it may be interesting for some people to see how the error and its discovery developed and when/how we (re-)acted.

Neukom also requested that his work be checked:

I think all the analysis needs to be replicated by someone else (maybe Ailie or Steven) to make sure all other errors I made can be identified and eliminated.

Karoly (18:36) reverted to Neukom that he would put the date in the email to me, but doubted that I would “accept that we didn’t find the issue without his help, but that doesn’t matter”. Karoly additionally asked Neukom to keep “good records” of what happened.

I am about to go home and have some dinner, then I’ll send this to Mdntyre, so that he gets it Friday morning. Melbourne Uni wanted as little detail in the short statement as possible. l’ll put the date in my email to Mdntyre, which he will likely post, as well as the short statement. I doubt that he will accept that we didn’t find the issue without his help, but that doesn’t matter…

Please keep good records of what happened when, and what you did. Also, keep any records of emails you receive from McIntyre or other bloggers. Joelle is being sent hate emails.

If the FOI release is complete, while there are some critical emails, none appear to me to be fairly classified as “hate mail” – the term “hate mail”, as used by climate scientists, appears to include anything that is merely critical. Karoly sat on the notice to me overnight and sent me an email the following morning Melbourne time with the same paragraph. Karoly additionally noted that participants at CA had “also” identified this “data processing issue”:

We would be grateful if you would post the notice below on your ClimateAudit web site. We would like to thank you and the participants at the ClimateAudit blog for your scrutiny of our study, which also identified this data processing issue.

I reported this at the time, with eyebrows more than somewhat quizzically raised at the Gavinesque coincidence that, after months of peer review and after acceptance of their paper, they had supposedly “independently” discovered the error in screening on June 5 – the very day that the precise error was spelled out at CA (though the issue of Gergis screening had already been discussed for a few days.)

The removal of the Gergis paper had been noted in a comment at RC (June 8 15:50 blog time; June 9 03:50 Melbourne). Another RC commenter pointed out to Schmidt that the problem had been discovered at CA:

Gavin – you ought also to mention that the problem was discovered at the Climate Audit blog

Mann appears to have contacted Karoly soon afterwards, as, within 10 minutes of sending this email to me, Karoly forwarded the email to Mann, with a covering note that the comment at RC about removal was correct. Even though Karoly had told Mann about the error, Mann reverted to Karoly that Mann and the other RC authors would falsely tell RC readers that they had “no further information” on the retraction of the paper from the journal website and that he would involve Schmidt and Steig in the plan:

We have simply noted at RC in the comments that the paper does appear to have been retracted from the AMS website, and we have no further information as to why. I will share this w/ Eric and Gavin so they know the status,

Mann also made defamatory remarks about me to Karoly:

Well I’m afraid Mclntyre has probably already leaked this anyway. I probably don’t have to tell you this, but don’t trust him to behave ethically or honestly here, and assume that anything you tell him will be cherry-picked in a way that maximally discredits the study and will be leaked as suits his purposes.

Karoly pointed out to Mann (2K:55 11:19 Melbourne) that there was discussion at CA of the announcement here. Karoly told Mann that they had a “fully-documented” record demonstrating their priority over CLimate Audit:

PS We do have a fully-documented record or who, when and how the data processing issue was identified by a member of the author team independent of, and before, any posts on this issue at CA or other web sites.

Needless to say, no such “fully-documented record” was disclosed to Michael Kotteck.

IPCC Check Kites Gergis

A few days ago, WUWT pointed out that the American Meteorological Society webpage showed that the Gergis et al paper had been officially “withdrawn”. However, readers should know better than to presume that this would have any effect on IPCC use of the reconstruction.

The withdrawal of the Gergis article hasn’t had the slightest impact on IPCC usage of the Gergis reconstruction, which continues to be used in the recently released AR5 Second Order Draft, thanks to academic check kiting reminiscent of Ammann and Wahl. Tim Osborn of CRU is a Lead Author of the AR5 chapter (as he was in AR4) and would be familiar with the technique from AR4.

Continue reading