On Aug 26, 2006, Judith Curry made the following comment about cyclone data, mentioning that re-processing had been done back to 1983:
The tropical cyclone data really is rather a mess, the NATL is definitely the most reliable, so I am focusing on that data set (with all its warts) until the global satellite data is reprocessed and reanalyzed. This has already been done back to 1983 (paper in review), they should be able to go back to 1977 with alot of hard work, then before 1977 it will be a bit dodgy since it is not clear what kind of shape some of that data is actually in. The problem is that until very recently, people have not been using the hurricane data as a climate data record (mainly used for regional damage estimates etc), so there hasn’t been much incentive until recently to try to get this data in shape.
An enormous ftp archive on all hurricanes from 1983 to date is now online (the directory ftp://eclipse.ncdc.noaa.gov/pub/isccp/b1/hurr which hosts the actual data is dated Sept 18, 2006 so we’re keeping you up-to-date).
The host page is here ; with the ftp location here. The new data set is descibed here . To give you an idea of the size of the archive, the zip file for an individual basin for an individual year range up to 678 MB in size. So there has been no stinting on providing raw data.
Right now, it’s an imposing data set, but certainly more data than I’m interested in or can handle. I don’t know whether there are plans to provide summary data (the Best Tracks data in total is much less than 1 MB in size.)
Climate Audit 
18 Comments
Let the games begin, although an audit of the data surely is the first step. Something that huge is bound to have a typo or two.
The vast bulk of it is probably satelite imagery.
Looks like pure netCDF collections to me…
That’ll show these arrogant amateur wannabe scientsts ….!
Re: #1
From the explanation of the reanalysis of the data we have:
Abstract from Kossin, et al. 2006 states in part:
In summary I would conclude that any auditing of this reanalysis without the aid of a cadre of graduate students just became neigh on to impossible. If I have the correct Kossin et al., 2006 then I would also guess that the data (reanalyzed) has been already used to confirm that an SST increase of 0.7 degrees centigrade over the past 100 years has caused a doubling of hurricanes and tropical cyclones. On the way, apparently, is a paper to connect the increased numbers of hurricanes and TCs directly to AGW.
Is the full Kossin paper available at this time for the general public to read?
Kossin recently stated here
Holland of Webster Curry et al at the same location had the following abstract:
Have you got the two abstracts conflated?
I absolutely blew it in attributing what was abstracted from Holland for the 10/20/2006 AMS seminar to Kossin and your presentations are the correct versions for each. These abstracts would appear to put Holland and Kossin in nearly direct opposition in interpreting past data.
The follow-up should be how can the contrasting views of Holland and Kossin be better understood and definitely removing my previous post (it is misleading and will become a constant reminder of my embarassing error).
#6,
No need to be reminded…You’ve “moved on” from that previous post, lol…
I wrote to Kossin as follows:
I received the following answer which someone might be able to explain for me:
My guess is that he wasn’t sure what you meant by “MB-range.” You have to be careful which abbreviations you use with meteorologists; at first I thought you meant millibars (mb). 😉
Re #8 I think he’s saying that he smoothed the storm tracks. He saw the phrase “existing best tracks” and assumed that your query is mainly about the paths the storms took.
Hurricanes tend to bounce and jump around and the centers are not always easy to find. In reanalysis, the analyzers use the best information available and develop a best-track for the records.
For some reason, he wants smooth tracks. I cannot guess why that would help whatever he’s doing with the data.
His answer took 53 words and was written in Academic Lite, but I guess it sounds better than, “I smoothed the data”.
Re #8: In my work I constantly get requests for data and usually end up playing 20 Questions to find out what the requesters really are after. It’s much better when they tell me what they want to use the data for and I can trim away the non-relevant stuff. Steve, explain what you want to do and ask how likely the data in present form will meet your criteria for quality control. For people willing to share data, this courtesy will encourage them; for those with something to hide, it will smoke them out. Either way you gain some knowledge and don’t inadvertantly put someone off who otherwise could be helpful.
Re: #7
Jonathan, thanks for reminding me that I have moved on. I declare that I have officially moved on, but not before I serve penance by detailing the differences in the views of Greg Holland and Jim Kossin.
FYI related to this dicussion:
Click to access Kossin102006.pdf
All presentations:
http://www.ametsoc.org/atmospolicy/environmentalsssarchives.html
Thanks for the link RPJ. Spurious trends. That’s why we concern ourselves with ergodicity. Spurious correlations. That’s why we concern ourselves with mechanistic modeling. This hurricane trend stuff is crap.
Re: #13
Thanks to RPJ’s links to Kossin’s and Holland’s views on TS intensities and numbers, the underlying data validity and what it means vis a vis SST and global warming, my serving penance has become easier — providing my following summaries are reasonably accurate.
J. Kossin shows how trends in TS intensities can be inflated by using what he describes as data with inconsistencies. He shows after reanalyzing TS intensity data from around the globe and making it “homogenous” that, while the Atlantic basin continues to show an upward trend, the globe as a whole does not. He asks the question why the Atlantic is acting so differently when the SSTs have increased in all areas analyzed and suggests that a more complete view can be obtained through “better physical understanding”.
G. Holland confines his arguments and observations to the Atlantic basin and shows dramatically increasing numbers of tropical cyclones (TC) in that area over the past 100 years (he shows data for the numbers going back over 150 years). He further claims that in this basin and over this time period that SST leads TC increases and explains over 60% of the variance in TC numbers. Holland sees no trend over this time period of the proportion of major hurricanes to TC, but points out that the number of major hurricanes shows the same dramatic increasing trend as the number of TC. Finally Holland finishes with the following statement: “The strong relationship between increases in storm and hurricane numbers and increases in SSTs leads to the inescapable conclusion that the majority of current hurricane activity is a direct result of greenhouse warming.”
I know which view/approach I prefer here and I doubt much that it will be the one that the popular media will be quoting.
Ergodicity in a chaotic terawatt heat engine. Maybe if the climate game were to be replayed it would be the Atlantic basin that would be inhaling and one of the others that would be exhaling.
Trends or transients?
Possibly in the Kossin world, but unlikely in the Holland version. Noticed that Holland is not saying that the “current hurricane activity in the Atlantic basin is a direct result of greehouse warming” but rather “current hurricane activity is a direct result of greenhouse warming” and all based on a presentation confined to Atlantic data. Holland’s concluding statement can be quoted directly by the popular media with no alterations required.
transients, not trends