Climate Audit

I’m in the process of trying to replicate some of the hurricane papers. Obviously this is new territory for me – so forgive me if I’m going over old ground. Emanuel Figure 1 states in its caption that "total Atlantic hurricane power dissipation has more than doubled in the past 30 yr" and David Stockwell’s query about this, in part, stimulated the recent discussion on this blog. I’ve collated hurricane track information from original sources bringing the data set pretty much up to today. Lots of interesting details when you look closely. Some parts of the debate involving Landsea and Emanuel seem to have been going on for over 20 years, long before the issue became prominent.

As a bit of background, here is a Figure from Landsea 1993 online here showing the trend in Atlantic "intense hurricanes" (IH) from a 1993 perspective and the connection to global warming. I don’t think that there’s any need for editorial comment.

Figure 1. Landsea 1993 Figure 7 showing Atlantic intense hurricane count.

Before looking at Emanuel’s Figure 1, I’ll look first at Landsea’s Figure 1 in his 2005 Comment on Emanuel 2005, as I’ve more or less come close to this figure. On the left is Landsea’s Figure 1 and on the right is my replication from archived track information The 2005 and 2006 to-date values are shown in cyan. While 2006 is not complete yet, it looks like it will have low values. In my first attempts to replicate this figure, in making an integral, I applied a factor of 1/4 for quarter-day periods. It appears that the index ignores the quarter-day periods and treats each 6-hour period as a unit. Obviously I’ve got the main features right, but there are some puzzles in the details. I have 1950 PDI as being the highest on the graph, while Landsea’s 1950 value is lower than 2004 values. (Emanuel appears to be even more so.) Landsea Comment on Emanuel notes that Emanuel had adjusted pre-1970 wind speeds following Landsea 1993 – but incorrectly implementing the method; however, Landsea said that the original data should be used and the problem appeared to lie with data in the period 1970-1990. (Data collection methods are not homogeneous.) I have not yet attempted to replicate the adjustment methods. Emanuel mentions adjustment in his Supplementary Information (which is referred to in the article but not available at the Nature website – it is available at his personal website.) To the extent that Emanuel’s results depend on these adjustments, one would have hoped for a more detailed reconciliation of the differences between adjusted and unadjusted results.

Figure 2: Landsea 2005 Figure 1. Derivation of Atlantic power-dissipation index (PDI). a, Emanuel’s bias-correction version2 of PDI for the North Atlantic tropical cyclones for 1949–2004. PDI takes into account frequency, duration and intensity of tropical cyclones by cubing the winds during the lifetime of the systems while they are of at least tropical-storm force (18 m s
1) and summing them up for the year. Values shown are multiplied by 10^-6 in units of m3 s-3. Horizontal line, time-series mean of 10.8; black curve, data after smoothing with two passes of a 1-2-1 filter. b, Three versions of the smoothed PDI for the North Atlantic using: dashed line, Emanuel’s applied bias-removal scheme; dotted line, 1993 version3 of the bias-removal scheme; solid line, original hurricane database. All three versions are identical from 1970 onwards.

Emanuel’s Figure 1 has a different scale as shown below. The caption states that the scale reflects multiplication of PDI by 2.1*10^-12 (as compared with 10^-6 in Landsea’s Figure 1.) In a lower figure (left panel), I show my replication of the Landsea smoothing and can more or less replicate the shape with endpoint-pinning, but not the scale. Any suggestions are welcome.

Emanuel 2005 Figure 1. A measure of the total power dissipated annually by tropical cyclones in the North Atlantic (the power dissipation index, PDI) compared to September sea surface temperature (SST). The PDI has been multiplied by 2.1 * 10^-12 and the SST, obtained from the Hadley Centre Sea Ice and SST data set (HadISST)22, is averaged over a box bounded in latitude by 6N and 18 N, and in longitude by 20W and 60W. Both quantities have been smoothed twice using equation (3), and a constant offset has been added to the temperature data for ease of comparison. Note that total Atlantic hurricane power dissipation has more than doubled in the past 30 yr.

A few posts ago, Willis and bender pointed out that Emanuel had "pinned" the end points in his smooth, resulting in a very distorted smooth. This had been observed previously by Landsea in Landsea 2005. In his Reply, Emanuel stated:

Landsea correctly points out that in applying a smoothing to the time series, I neglected to drop the end-points of the series, so that these end-points remain unsmoothed.

The smoothing described by Emanuel was a (1,2,1) smoothing applied twice. If you do this using the filter-function in R, then you automatically get NA values for the end-points. The only way to replicate Emanuel’s algorithm is by manually re-inserting the endpoiunt values. I presume that the functions would operate similarly in other languages. It would be interesting to know exactly what language and function Emanuel used to perform this operation – in which the endpoints had to be manually removed. It must be something unusual.

Emanuel goes on to say that, if the 2005 results are included, "much" of the upswing is restored – although he did not provide a figure or supplementary information to demonstrate this result. He also discusses the W Pacific in this context, which I’ll get to on another occasion. Emanuel:

As it happens, including the 2004 and 2005 Atlantic storms and correctly dropping the end-points restores much of the recent upswing evident in my original Fig. 1

I thought that it would be interesting to check this result and also to see what happens if 2006 data is included – here I’ve used storm data to date, which is a bit of an under-estimate but it doesn’t seem like there’s much more hurricane activity expected. I’ll re-do this figure if necessary on a later occation – this is indicative for now. The left panel shows the PDI calculation using data up to 2004 with the black dashed line showing my emulation of Emanuel 2005 using the "pinned" smooth; the red shows the (1,2,1) smooth applied twice, losing the last two values. (This is not really a very good method of smoothing.) The solid black line is a simple linear trend for 1949-2004.

While the shape is somewhat similar, note that the smoothed PDI value in 1950 in this emulation is higher than the corresponding value in 2000, while Emanuel has the opposite relationship. (Indeed, using the archived data, the record PDI was in 1950 not 2005!) As noted above, the difference results from Emanuel’s adjustments to pre-1970 wind speeds, which were contested by Landsea. I’ve not attempted to replicate Emanuel’s adjustment so far and will get to it on another occasion with any luck. In other words, to a very considerable extend, Emanuel’s results depend on a contested adjustment – a point made by Landsea. (BTW the ranking of hurricanes by PDI is pretty interesting and I’ll post separately on that. Katrina did not rank in the top 100, but Isabel, which reached Toronto, was in the top 5. )

The second panel adds in the 2005 results. Does this recover "much" of the increase? Using this particular form of smoothing, the closing value is slightly above 1950 values, but only slightly. I think that Emanuel’s characterization of the results is not entirely unbiased. The third panel shows the impact of adding 2006 values with Emanuel’s pin method. 2006 PDI values look like they are going to be rather low in a 50-year perspective – if this were tree rings, Briffa would attribute these low PDI values to some unknown anthropogenic factor, but I guess Emanuel is waiting for the end of the season to arrive. Pinning to 2006 values would result in a very low trend. I’ve experimented a little with lowess smoothing and found that the shape of the smooth can be very sensitive to the f-choice. When f passes from .038462 to .038461, the impact of the 2006 low values impacts tremendously – at highe f-values, there is a high close; with lower f-values, a very low close. In none of the cases is there a significant linear trend.

Emanuel’s original claim about PDI doubling in 30 years appeared to be based not even on a linear trend, but to a considerable extent on the rhetorical effect of endpoint pinning in his Figure 1. What’s the over-under on Emanuel publishing in Nature an updated version of his endpoint-pinned graph using 2006 data? The odds would be astronomical.

PS – A script for these calculations is here http://data.climateaudit.org/scripts/hurricane/emanuel.2005.figure1.txt. I’ve used the elegant R-function tapply for the main calculations which handles "ragged" arrays beautifully. If Emanuel used Fortran, these calculations might take dozens, if not hundreds of lines. In R, the key calculations are done in two lines (and by organizing the data properly the input is in a couple of lines as well.) To restrict to windspeeds above 18 msec-1 takes another couple of lines. Now Fortran may make sense when machine code type calculations are required, but for statistical calculations with small data sets like the hurricane data, R blows the competition away.

hurricane$pdi<-tapply(Track$wind.ms^3,Track$id,sum,na.rm=T) # calculate pdi per storm
chron.pdi<- c(tapply(hurricane$pdi,hurricane$year,sum))# sum pdi per year

Plus there are wonderful statistical packages so you can actually analyze the data as opposed to doing Fortran programs.

Update: Here are two histograms – on the left is a histogram of PDI for all storms in the archive – storms and hurricanes are obviously extreme events and this is demonstrated by the distribution which is strongly skewed. On the right is a histogram of all annual PDIs. In this case, there is much less skewing as the sum of hurricane energies in a given year balances out somewhat. However, the distribution is still strongly skewed and the assumption of a normal distribution, implicit is simple trend analysis, obviously cannot be used.