UKweatherworld has a thread where people don’t comment but simply contribute references on the Holocene Optimum. I’m going to start doing things like this as a way of keeping track of things. People have contributed data but it’s impossible to keep track of. This thread is going to consist only of scripts and data relevant to replicating the hurricane papers. Any general commentary will be deleted. I’ll put some of the data sets into *.txt files so that they can be called. Please provide references to some of the data sets.
Prior discussion http://www.climateaudit.org/p?790, http://www.climateaudit.org/p?803, http://www.climateaudit.org/p?822.
Emanuel, Nature 2005 url Website
Landsea, Nature 2005 url
Webster publications
Webster, P.J., G. J. Holland, J. A. Curry, and H.-R. Chang, 2005: Changes in Tropical Cyclone Number, Duration, and Intensity in a Warming Environment, Science, 309 (5742), 1844-1846. url
Webster P.J., J.A. Curry , J. Liu, and G. J. Holland, 2006: Response to comment on "Changes in tropical cyclone number, duration, and intensity in a warming environment", Science, 311 (5768), 1713c url
Mann and Emanuel, EOS, 2006. url
Curry, J.A., P.J. Webster and G.J. Holland, 2006: Mixing Politics and Science in Testing the Hypothesis That Greenhouse Warming Is Causing a Global Increase in Hurricane Intensity. Bull. Amer. Met. Soc., 87 (8), 1025-1037 url
Climate Audit 
32 Comments
Here’s a script which gives the location of Kaplan’s SST data. It’s in Netcdf format. Fortunately there’s an R package which reads this format. I can’t say that I know my way around the format but I was able to almost instantly recover the relevant data and make average temperatures for the 6 Webster et al basins.
Script http://data.climateaudit.org/scripts/hurricane/read.kaplan.txt
Data (average for 6 basins from Jan 1856 to Aug 2006 http://data.climateaudit.org/data/hurricane/basin.average.kaplan.txt
Here’s a plot of the 6 basin averages
Here’s the Atlantic PDI data as collated by Willis including David Smith’s update for 2005-1006, as well as SST up to 2004 (I’ll update this.)
Data http://data.climateaudit.org/data/hurricane/pdi.atlantic.txt
See: http://www.climateaudit.org/?p=803#comment-46813
bender adds: The “PDI” values are from Emanuel (2005) and so are the “SST” values (=”RawHadCRUT_SST”). The SSTs must be anomalies since they are in the range [-1,1]. HadISST are sometimes used in these analyses (e.g. that’s what Landsea has used) and these appear to be actual temperatures as
they are in the range [26,30]
Here’s a plot including 2006
Here’s a compilation of Basin Averages of MSU data downloaded in Sept 2006
Data http://data.climateaudit.org/data/hurricane/basin.average.msu.txt
Script http://data.climateaudit.org/scripts/hurricane/make.msu.average.txt
Here’s a plot. The similarity between basins in the satellite data is remarkable. I’ll post up similar data for the CRU and it’s nothing like this.
Well, all the Kaplan SST data sets look very normal in a Histogram plot, and so do the residuals of a linear fit (unsurprisingly). I tested the NATL, NIO, and SWP sets for autocorrelation, and they all show ~0.9 at lag 1 (p
tripped up by the ol’
Darn. 🙂 I’ll do the analysis again tomorrow. It just showed deep auto- and intercorrelation in the Kaplan SST’s, even across data sets representing physically distant locales. If that’s not interesting, let me know.
No posts on this thread without contributing a script or data !!! You may post discussion points on one of the other hurricane threads. This message will self-destruct.
Here are the basin averages using CRU (a fairly recent version, but not the most recent version).
Data http://data.climateaudit.org/data/hurricana/basin.average.hadcrut2.txt
Here is a link to a program that creates timeseries of many atmospheric variables, as well as of sea surface temperature.
Here is a link to a program that creates correlations of atmospheric variables.
Here is a link to the US National Hurricane Center’s hurricane archives.
Here is the script for bender’s calculations. http://data.climateaudit.org/data/hurricane/hurricane_R.txt
Here are data files:
Landfall http://data.climateaudit.org/data/hurricane/landfall.txt
Emanuel PDI http://data.climateaudit.org/data/hurricane/emanualPDI.txt
http://data.climateaudit.org/data/hurricane/allstorms.txt
Here is description of PDI index by David Smith
http://data.climateaudit.org/data/hurricane/Power Dissipation Index.doc
Webster et al in their Response to Comment (Science 311, March 2006) stated that Chan Science 2006 and Emanuel 2005 used the following data sets to construct their PDI:
http://www.weather.unisys.com/hurricane/index.html
which provides information for individual storms in 6 basins for the following periods:
ATL 1851-2005
EPAC – 1949-2005
WPAC 1945-2005
SPAC only 2000-2005 as far as I can tell so far
SIO 1949-2003
NIO 1945-2003
Emanuel 2005 says:
Emanuel’s SI describes the adjustment but does not present the unadjested data. Landsea 2006 presents different results for Atlantic PDI. Is this data archived anywhere?
What collations of the data are available other than what Willis obtained in his email from Emanuel?
Steve, Emanuel has data on hurricane tracks in NetCDF here, but I haven’t had time to look at it yet.
w.
Willis’ calculation of Atlantic PDI is here http://homepage.mac.com/williseschenbach/.Public/Atlantic_Hurricane_PDI.xls
Steve;
Here are a few of the links I contributed to UKweatherworld just searching around NASA. The last link though is not a technical paper; however, it provides a pretty good description of the tropical geology of SE USA from about 7K years ago. It also has some strong references WRT geological and population characteristics and these are reinforced by strong; but, aged studies. If these are of use feel free to add if not it is not an issue.
Dave Cooke
Click to access cp-2-79-2006.pdf
http://earthobservatory.nasa.gov/Study/BorealMigration/boreal_migration3.html
http://globalchange.nasa.gov/KeywordSearch/Metadata.do?Portal=GCMD&KeywordPath=&EntryId=NOAA_NCDC_PALEO_2005-015&MetadataView=Brief&MetadataType=0&homepg
http://globalchange.nasa.gov/KeywordSearch/Keywords.do?Portal=GCMD&KeywordPath=Parameters%7CPALEOCLIMATE&MetadataType=0&lbnode=gcmd3a
Click to access Ackert.pdf
Click to access sdarticle.pdf
http://igloo.gsfc.nasa.gov/wais/pastmeetings/abstracts04/Das.htm
http://igloo.gsfc.nasa.gov/wais/pastmeetings/abstracts99/das.html
Click to access 2003_Vaughan_Etal_Antarctic.pdf
http://search.nasa.gov/search/search?as_sitesearch=igloo.gsfc.nasa.gov/wais/pastmeetings/PPT03&access=p&output=xml_no_dtd&sort=date%3AD%3AL%3Ad1&ie=UTF-8&client=nasa_production&actionType=searchIndex&q=Holocene+Climate&numgm=5&site=nasa_collection&oe=UTF-8&proxystylesheet=nasa_production&ip=209.170.95.133
http://search.nasa.gov/search/search?as_sitesearch=igloo.gsfc.nasa.gov/wais/pastmeetings/abstracts05&access=p&output=xml_no_dtd&sort=date%3AD%3AL%3Ad1&ie=UTF-8&client=nasa_production&actionType=searchIndex&q=Holocene+Climate&numgm=5&site=nasa_collection&oe=UTF-8&proxystylesheet=nasa_production&ip=209.170.95.133
Click to access Ulu_revised80105_3amend_2.pdf
Click to access APPENDIX_K_final_Jan2004.pdf
Steve,
Sorry, my bad you were looking to post Hurricane data.
My humble apologies…
Dave Cooke
Steve;
I would like to submit these links as sources of data that can help explain recent water vapor research that participates in hurricane formation. I did my best to make sure to include links that should terminate in professional studies from a .gov site.
Click to access 83490.pdf
http://www.eol.ucar.edu/projects/refractt/ document link
http://cosmic-io.cosmic.ucar.edu/cdaac/doc/index.html
Click to access CAMEX-5_White_Paper.pdf
http://www.timed.jhuapl.edu/WWW/science/scienceHighlights/index.php#SunCoupling
Dave Cooke
(pls delete)
#3 ->should be …/script*s*/…
#10 ->bender’s calculations should move: …/*scripts*/hurricane/hurricane_R.txt
Steve, your scripts turn out to be a valuable resource for R-novices like myself … wish I could browse that folder.
Thank you!
REF #16
Steve;
Please correct the 3rd attached link from my list for the COSMIC system, it should have been: http://www.cosmic.ucar.edu/~braunj/ The microwave measurement referenced under this link is specifically: http://www.cosmic.ucar.edu/~braunj/front_range/mwrp.html
Note that this is specific to the Denver CO area; however, the characteristics on 7/26, 8/14, 9/21-22, 10/1 2006 seem to be indicitive of an unusual characteristic of a dual direction of freedom from a single phenomon. (It has to be related to the depth of measeurement penetration rather then be a dual vector of change from a single phenomon.)
Thanks,
Dave Cooke
I’m not sure how relevant this is to the Hurricane research papers you are examining, however it is sometimes worthwhile using completely independent data sets to make comparisons, so here are links to the complete Australian Bureau of Meteorology (BoM) Tropical Cyclone Best Track database which covers from 1906 to 2005:
ftp://ftp.bom.gov.au/anon2/home/ncc/cyclone/cyclones_newformat.zip (350 KB ZIP file)
and the corrections and amendments notes:
ftp://ftp.bom.gov.au/anon2/home/ncc/cyclone/amends.txt
The ZIP file contains the full database as an ASCII text file, and also contains a text file explaining the format.
The information is in lines of text for 6 hourly data that includes the BoM season and cyclone number, cyclone name, data and time, lat and lon, and various meteorological factors including central pressure, maximum winds, wind radii, etc.. A spreadsheet can easily be used to separate out whatever parts you may be interested in.
Researchers should note that the BoM uses the WMO standard 10-min-av winds instead of the 1-min-av winds used in the US, and this issue will need to be accounted for if comparing to say US best track data for the same cyclones – this can also be an issue when using independent data for other basins as well.
Further to this, those unfamiliar with the BoM need to know that when making direct comparisons, much of the modern US best track material is largely satellite based, while in addition to satellite data the BoM uses a wide range of material from both land based instrumentation including radar (where appropriate) and shipping reports, and that extensive post-storm analysis is done for many landfalling systems which may all be taken into account in ‘best track’ data.
The total number of cyclones in any season contained in the US and BoM data sets can be different due to both the amount and type of data taken into account, and different definitions of what a cyclone is – the BoM uses 10-min-av winds greater than 33 knots in all 4 quadrants, while the US uses 1-min-av winds greater than 33 knots in any quadrant.
The cyclone intensity information can vary significantly between the BoM and US data – in most cases the BoM data is more accurate due to both local experience and the sheer volume of data used in the analysis.
The full BoM database was undergoing a quality control revision in 1999-2000, however I understand the process is still incomplete (due to funding cuts, if my memory serves me correctly).
I hope this is of use to some of you.
Steve M, See corrections in #18. [Pls delete this post]
Judith, can you explain some nomenclature to me in this hurricane data base http://www.npmoc.navy.mil/jtwc/best_tracks/shindex.html
In the following years 1986, 1997, 1998 and 1999, there are out of sequence storm numbers 1986 has storms 1 to 33 and then 80; similarly for the other three years. Do you know why this is?
Re: #22 – Steve, any JTWC 80+ numbers in best tracks data are those systems that were not identified as storms in real time, but were later found to be storms when re-analysing the data.
This happens from time to time with marginal systems, mainly due to the pressures of getting information out at regular intervals with only a small number of mets working at any particular time, as the poor satellite imagery presentation of many maginal systems combined with lack of other supporting data means it can take a pro-met many hours of careful analysis of multiple satellite images before making a judgement that it is in fact a storm, and time may not be available to do this process properly until after the season when the best tracks are being put together.
Most marginal systems that fall between the cracks in real time are a long way from inhabited areas where other data can help making such a judgement, and in real time they are usually being tracked as ‘invest’ systems that have potential to develop and are assigned 90+ numbers, so if anyone in a boat or plane is intending to be in the vicinity they can usually get enough info to see that the weather could deteriorate in the area.
Details and script for collating hurricane and storm track data is here http://www.climateaudit.org/?p=855.
Re the hurricane data quality issue. I am becoming increasingly convinced that the data outside the NATL is problematic. I suggest a two pronged approach. Prong 1: use exactly the same data as Emanuel to assess what inferences can actually be made from the time series of SST and PDI in NATL and WPAC from 1949-2005 (and ignore data quality issues, just focus on auditing the statistical methods). Prong 2: use the NATL data record (HURDAT) since 1851 to see what can be determined from the longer time series with varying levels of uncertainty as you go back in time. If you agree with this approach, then the data issues become: making sure you have the same data as used by Emanuel, and then sorting through the NATL data quality issues. sorting through multiple datasets, many of dubious quality, could be huge time sink and not really get us anywhere. Lets have some discussion about what data sets we are going to look at, before digging into the various warts associated with a number of different data sets.
#25. I’m not planning to get into hurricante track QC issues – I don’t have the time; I’m already spending more time on this than I should without previously finishing some proxy papers. I’m just going to work through some of the calculations and see if I can repeat them.
Judith Curry has archived track data as used here http://www.eas.gatech.edu/research/data.htm. Excellent.
A discussion of the topic is here http://www.wunderground.com/education/webster.asp
There is extensive information on individual West Pacific storms from old annual reports of the Joint Typhoon operation, now online. An example is here https://metocph.nmci.navy.mil/jtwc/atcr/1959atcr/pdf/section5.pdf. There’s a lot of data on individual storm tracks that could be easily digitized with a little clerical work.
A good general reference on meteorology, including tropical weather, is
here .
It is slightly technical but well-written and understandable.
A well-written article on tropical atmospheric circulation is given here . These Chinese researchers studied Hadley cells and associated features. Nice cross-sections of the ITCZ and circulation cells.
My opinion is that some of the mystery about climate shifts will disappear as we learn more about the behavior of these massive heat removal systems. The Southern and Northern circulations have the same general features but differ in ways (see diagrams) that might affect their ability to remove heat. Perhaps at times one hemisphere’s Hadleys and Ferrels do a better job of removing heat than does the other.
As a side note, if I remember correctly, Tim Ball mentioned that the GCMs do a rather poor job of modeling these important circulations.
A paper on Hadley Cell and tropical/subtropical circulation changes since 1950 is given here .
Sweet site , I hadn’t noticed climateaudit.org before till my friend told me about it.
Keep up the great work! I will be contributing more at climateaudit.org
Looking forward to some good new friends here at climateaudit.org
🙂