Emanuel 2005 Figure 1

I’m in the process of trying to replicate some of the hurricane papers. Obviously this is new territory for me – so forgive me if I’m going over old ground. Emanuel Figure 1 states in its caption that "total Atlantic hurricane power dissipation has more than doubled in the past 30 yr" and David Stockwell’s query about this, in part, stimulated the recent discussion on this blog. I’ve collated hurricane track information from original sources bringing the data set pretty much up to today. Lots of interesting details when you look closely. Some parts of the debate involving Landsea and Emanuel seem to have been going on for over 20 years, long before the issue became prominent.

As a bit of background, here is a Figure from Landsea 1993 online here showing the trend in Atlantic "intense hurricanes" (IH) from a 1993 perspective and the connection to global warming. I don’t think that there’s any need for editorial comment.

Figure 1. Landsea 1993 Figure 7 showing Atlantic intense hurricane count.

Before looking at Emanuel’s Figure 1, I’ll look first at Landsea’s Figure 1 in his 2005 Comment on Emanuel 2005, as I’ve more or less come close to this figure. On the left is Landsea’s Figure 1 and on the right is my replication from archived track information The 2005 and 2006 to-date values are shown in cyan. While 2006 is not complete yet, it looks like it will have low values. In my first attempts to replicate this figure, in making an integral, I applied a factor of 1/4 for quarter-day periods. It appears that the index ignores the quarter-day periods and treats each 6-hour period as a unit. Obviously I’ve got the main features right, but there are some puzzles in the details. I have 1950 PDI as being the highest on the graph, while Landsea’s 1950 value is lower than 2004 values. (Emanuel appears to be even more so.) Landsea Comment on Emanuel notes that Emanuel had adjusted pre-1970 wind speeds following Landsea 1993 – but incorrectly implementing the method; however, Landsea said that the original data should be used and the problem appeared to lie with data in the period 1970-1990. (Data collection methods are not homogeneous.) I have not yet attempted to replicate the adjustment methods. Emanuel mentions adjustment in his Supplementary Information (which is referred to in the article but not available at the Nature website – it is available at his personal website.) To the extent that Emanuel’s results depend on these adjustments, one would have hoped for a more detailed reconciliation of the differences between adjusted and unadjusted results.


Figure 2: Landsea 2005 Figure 1. Derivation of Atlantic power-dissipation index (PDI). a, Emanuel’s bias-correction version2 of PDI for the North Atlantic tropical cyclones for 1949–2004. PDI takes into account frequency, duration and intensity of tropical cyclones by cubing the winds during the lifetime of the systems while they are of at least tropical-storm force (18 m s1) and summing them up for the year. Values shown are multiplied by 10^-6 in units of m3 s-3. Horizontal line, time-series mean of 10.8; black curve, data after smoothing with two passes of a 1-2-1 filter. b, Three versions of the smoothed PDI for the North Atlantic using: dashed line, Emanuel’s applied bias-removal scheme; dotted line, 1993 version3 of the bias-removal scheme; solid line, original hurricane database. All three versions are identical from 1970 onwards.

Emanuel’s Figure 1 has a different scale as shown below. The caption states that the scale reflects multiplication of PDI by 2.1*10^-12 (as compared with 10^-6 in Landsea’s Figure 1.) In a lower figure (left panel), I show my replication of the Landsea smoothing and can more or less replicate the shape with endpoint-pinning, but not the scale. Any suggestions are welcome.


Emanuel 2005 Figure 1. A measure of the total power dissipated annually by tropical cyclones in the North Atlantic (the power dissipation index, PDI) compared to September sea surface temperature (SST). The PDI has been multiplied by 2.1 * 10^-12 and the SST, obtained from the Hadley Centre Sea Ice and SST data set (HadISST)22, is averaged over a box bounded in latitude by 6N and 18 N, and in longitude by 20W and 60W. Both quantities have been smoothed twice using equation (3), and a constant offset has been added to the temperature data for ease of comparison. Note that total Atlantic hurricane power dissipation has more than doubled in the past 30 yr.

A few posts ago, Willis and bender pointed out that Emanuel had "pinned" the end points in his smooth, resulting in a very distorted smooth. This had been observed previously by Landsea in Landsea 2005. In his Reply, Emanuel stated:

Landsea correctly points out that in applying a smoothing to the time series, I neglected to drop the end-points of the series, so that these end-points remain unsmoothed.

The smoothing described by Emanuel was a (1,2,1) smoothing applied twice. If you do this using the filter-function in R, then you automatically get NA values for the end-points. The only way to replicate Emanuel’s algorithm is by manually re-inserting the endpoiunt values. I presume that the functions would operate similarly in other languages. It would be interesting to know exactly what language and function Emanuel used to perform this operation – in which the endpoints had to be manually removed. It must be something unusual.

Emanuel goes on to say that, if the 2005 results are included, "much" of the upswing is restored – although he did not provide a figure or supplementary information to demonstrate this result. He also discusses the W Pacific in this context, which I’ll get to on another occasion. Emanuel:

As it happens, including the 2004 and 2005 Atlantic storms and correctly dropping the end-points restores much of the recent upswing evident in my original Fig. 1

I thought that it would be interesting to check this result and also to see what happens if 2006 data is included – here I’ve used storm data to date, which is a bit of an under-estimate but it doesn’t seem like there’s much more hurricane activity expected. I’ll re-do this figure if necessary on a later occation – this is indicative for now. The left panel shows the PDI calculation using data up to 2004 with the black dashed line showing my emulation of Emanuel 2005 using the "pinned" smooth; the red shows the (1,2,1) smooth applied twice, losing the last two values. (This is not really a very good method of smoothing.) The solid black line is a simple linear trend for 1949-2004.

While the shape is somewhat similar, note that the smoothed PDI value in 1950 in this emulation is higher than the corresponding value in 2000, while Emanuel has the opposite relationship. (Indeed, using the archived data, the record PDI was in 1950 not 2005!) As noted above, the difference results from Emanuel’s adjustments to pre-1970 wind speeds, which were contested by Landsea. I’ve not attempted to replicate Emanuel’s adjustment so far and will get to it on another occasion with any luck. In other words, to a very considerable extend, Emanuel’s results depend on a contested adjustment – a point made by Landsea. (BTW the ranking of hurricanes by PDI is pretty interesting and I’ll post separately on that. Katrina did not rank in the top 100, but Isabel, which reached Toronto, was in the top 5. )

The second panel adds in the 2005 results. Does this recover "much" of the increase? Using this particular form of smoothing, the closing value is slightly above 1950 values, but only slightly. I think that Emanuel’s characterization of the results is not entirely unbiased. The third panel shows the impact of adding 2006 values with Emanuel’s pin method. 2006 PDI values look like they are going to be rather low in a 50-year perspective – if this were tree rings, Briffa would attribute these low PDI values to some unknown anthropogenic factor, but I guess Emanuel is waiting for the end of the season to arrive. Pinning to 2006 values would result in a very low trend. I’ve experimented a little with lowess smoothing and found that the shape of the smooth can be very sensitive to the f-choice. When f passes from .038462 to .038461, the impact of the 2006 low values impacts tremendously – at highe f-values, there is a high close; with lower f-values, a very low close. In none of the cases is there a significant linear trend.

Emanuel’s original claim about PDI doubling in 30 years appeared to be based not even on a linear trend, but to a considerable extent on the rhetorical effect of endpoint pinning in his Figure 1. What’s the over-under on Emanuel publishing in Nature an updated version of his endpoint-pinned graph using 2006 data? The odds would be astronomical.

PS – A script for these calculations is here http://data.climateaudit.org/scripts/hurricane/emanuel.2005.figure1.txt. I’ve used the elegant R-function tapply for the main calculations which handles "ragged" arrays beautifully. If Emanuel used Fortran, these calculations might take dozens, if not hundreds of lines. In R, the key calculations are done in two lines (and by organizing the data properly the input is in a couple of lines as well.) To restrict to windspeeds above 18 msec-1 takes another couple of lines. Now Fortran may make sense when machine code type calculations are required, but for statistical calculations with small data sets like the hurricane data, R blows the competition away.

hurricane$pdi<-tapply(Track$wind.ms^3,Track$id,sum,na.rm=T) # calculate pdi per storm
chron.pdi<- c(tapply(hurricane$pdi,hurricane$year,sum))# sum pdi per year

Plus there are wonderful statistical packages so you can actually analyze the data as opposed to doing Fortran programs.

Update: Here are two histograms – on the left is a histogram of PDI for all storms in the archive – storms and hurricanes are obviously extreme events and this is demonstrated by the distribution which is strongly skewed. On the right is a histogram of all annual PDIs. In this case, there is much less skewing as the sum of hurricane energies in a given year balances out somewhat. However, the distribution is still strongly skewed and the assumption of a normal distribution, implicit is simple trend analysis, obviously cannot be used.


17 Comments

  1. Louis Hissink
    Posted Oct 12, 2006 at 6:02 AM | Permalink

    Steve,

    “Hurricane Power Dissipation” – might be useful to post a concise definition of this since my quick reading of your post above left me wondering what on earth that factor actually represents physically.

    At a first approximation I guessed it might be a measure of how quickly a hurricane ceased being one over chronological time (what else!).

    But as hurricanes are essentially a specific and interesting behaviour of turbulence then such a simple formula above seems to be incomplete, based on my own experience in understanding turbulence in water flow in natural drainages related to the deposition of heavy minerals.

  2. Paul Gosling
    Posted Oct 12, 2006 at 6:05 AM | Permalink

    Steve

    I have only half been following the hurricane threads, so may have missed this. Has anyone done separate analyses for El Nino and non El Nino years?

  3. TAC
    Posted Oct 12, 2006 at 6:22 AM | Permalink

    SteveM: Interesting dataset; thank you for posting it.

    I can reproduce your figures from the 1949-2005 Atlantic data you posted, though it seems I am not processing the data quite right (I seem to have the units messed up).

    Anyway, it shouldn’t matter for trends, so I took a look at the 1949-2005 period to check for linear trends. No luck! Even after quite a bit of manipulation (note: the errors are not normal, but a sqrt or cube-root transform seems to yield a reasonable bell shape) and trying a suite of standard parametric and nonparametric tests.

    However, in the interest of exposing possible bias, choosing a starting year of 1949 tends to “skew” things. If one starts somewhat earlier or somewhat later, trends do appear in the series (using a starting point in the 1970s, as Emanual did, seems to be ideal if you want to find significant trends). I am not fully satisfied with my analysis here, but I think that result will stand up.

    To complicate things a bit more, this series is not without time series structure. After transformation (though not before) there seems to be significant autocorrelation at lags 1, 12, and maybe others. Also, there seems to be significant LTP in the series (d=0.2), which casts substantial doubt on my previous comments about the significance of any trends in the series.

  4. David Smith
    Posted Oct 12, 2006 at 7:13 AM | Permalink

    Re #2 It’s been done, but I don’t have a recent link. The appropriate measure is the SOI (Southern Oscillation Index) which is available online and indicates the strength of El Nino and La Nina.

    It would be interesting if there was some statistical way to back out the SOI (El Nino effect) on PDI and see if any trend emerges.

    One small thing that I stumbled over, while trying to replicate Emanuel’s PDIs, is how to handle things called “subtropical storms”. These are hybrid systems that aren’t truly tropical. It appears that Emanuel includes those in his calculations. Subtropical storms may appear in some databases but not others, so be cautious. They don’t have a big impact, because they tend to be weak, plus they are small in number, but they can affect the less-active years PDIs.

  5. Steve McIntyre
    Posted Oct 12, 2006 at 7:55 AM | Permalink

    I’ve amended this post a little. I’ve pretty much matched LAndsea’s Figure 1 scale by using an integral which does not use quarter-day units. I’ve posted up the script.

  6. Steve Sadlov
    Posted Oct 12, 2006 at 9:38 AM | Permalink

    Do these power dissipation models include a factor pertaining to vertical energy flux inpinging on the tropopause?

  7. Steve McIntyre
    Posted Oct 12, 2006 at 9:47 AM | Permalink

    They are simply integrals (sums) of wind speed^3.

  8. Dave Dardinger
    Posted Oct 12, 2006 at 10:27 AM | Permalink

    Steve M,

    At least on my computer, using both Netscape and IE, your part of figure 2 is partly hidden by the sidebar. I’m missing about 1982-2004. It’s not a big deal but would probably be off-putting to some newcomers to the site. Perhaps you could stack the charts rather than put them side by side?

  9. Steve Sadlov
    Posted Oct 12, 2006 at 10:39 AM | Permalink

    RE: #7 – So would it be right to say, the models do not really account for all the power dissipation, since they do not account for all the energy fluxes, correct?

  10. David Smith
    Posted Oct 12, 2006 at 11:48 AM | Permalink

    Re #9 PDI does not account for the areal extent of the winds, either. Hurricanes can vary considerably in area.

    PDI is sort of a back-of-the-envelope estimate, but when you’re dealing with limited data, it’s about all you can do.

  11. TCO
    Posted Oct 13, 2006 at 1:13 AM | Permalink

    Steve, you can’t expect old salts like Mike Mann et al to use new-fangled open-source computer languages, like R. They are old school guys. Substance over process. If it was good enough for Grace Hoppper, it’s good enough for us.

  12. David Smith
    Posted Oct 13, 2006 at 7:54 AM | Permalink

    Here is a link to the SOI (Southern Oscillation Index), of El Nino fame.

    There is lag of a few months between it and El Nino / La Nina.

    What counts are the El Nino / La Nina conditions during hurricane season, not year-round.

  13. Steve McIntyre
    Posted Oct 14, 2006 at 1:20 PM | Permalink

    The following figure is at Emanuel’s website – the units are different from the original diagram (excerpted above). Anyone got any ideas?

  14. Ken Fritsch
    Posted Oct 14, 2006 at 3:47 PM | Permalink

    Re: #8

    At least on my computer, using both Netscape and IE, your part of figure 2 is partly hidden by the sidebar. I’m missing about 1982-2004.

    Copy the obscured graph (picture) to Word and you will see what is behind the right side curtain.

  15. David Smith
    Posted Oct 25, 2006 at 11:43 AM | Permalink

    I’m puzzled yet again. Help wanted.

    I looked at Webster etal (2005), Figure 1. The line labeled “EPAC” is a “running 5-year mean of SST” for an Eastern Pacific box for June-October. It shows a distinct drop in SST around 2000, back to circa 1980 temperatures. Odd, so I wanted to see if that shows up elsewhere.

    So, I looked at Hoyos et al, which is written by (almost) the same team. Figure 1A shows “five-year moving average anomalies” for the same EPAC box and season. It is hard to see through the jumble, but I wanted to confirm the EPAC 2000vs1980 pattern. I looked, but I don’t see it, at least not the same magnitude. There are some other mismatches on the EPAC line, too.

    So, I looked at the other ocean basins in the two papers. Among the oddities:

    * WPAC in Webster rises about 0.5C (low to high) but in Hoyos the rise is maybe 0.25C
    * SPAC in Webster has a 0.5C range (low to high) while in Hoyos it looks like about 0.3C
    * NATL in Webster has a 0.65C range while in Hoyos the rise is 0.4C. (Emanuel in his NATL box shows a 1.1C rise, but that’s another story)
    * several of the shapes don’t seem to wiggle in quite the same way (WPAC for instance)

    Maybe the difference between “running average” and “moving average” creates such curve changes. Maybe the team changed SST databases between papers, though I would hope the SST data correlated reasonably well. Maybe some lines are mislabeled. Maybe my eyeballs are misreading. Maybe i don’t understand Hoyos’ scale.

    Webster’s use of SST was simply a visual presentation and is of no consequence to their paper. Hoyos, though, uses the SST data in a statistical evaluation. The differences in the SST data may make a difference with Hoyos.

    I’ll read the papers in detail later today, to see if there’s an explanation. Meanwhile, if someone can find where I’m screwing up, please post. Thanks.

  16. bender
    Posted Oct 25, 2006 at 12:49 PM | Permalink

    Maybe the difference between “running average” and “moving average” creates such curve changes.

    They mean the same thing.

  17. David Smith
    Posted Oct 25, 2006 at 1:14 PM | Permalink

    Re #16 Yes, in my use they mean the same, but maybe they refer to different methods in Webster and Hoyos.

%d bloggers like this: