Myles Allen recently asked that more attention be paid by critics to work on climate sensitivity, rather than paleoclimate. Nic Lewis, a coauthor of O’Donnell et al 2010, has been parsing climate sensitivity calculations for some time and with considerable frustration. Nic Lewis has a very important article at Judy Curry’s here.
One of the seminal sensitivity estimates is Forest et al 2006. Nic reports that he tried for over a year to get data for this study with Forest finally saying that the raw data was now “lost”.
I have been trying for over a year, without success, to obtain from Dr Forest the data used in Forest 2006…. Unfortunately, Dr Forest reports that the raw model data is now lost.
Nic was able to to get data for two predecessor studies and has concluded that the calculations in Forest et al 2006 were done erroneously:
If I am right, then correct processing of the data used in Forest 2006 would lead to the conclusion that equilibrium climate sensitivity (to a doubling of CO2 in the atmosphere) is close to 1°C, not 3°C, implying that likely future warming has been grossly overestimated by the IPCC.
This is important stuff. Nic is very sharp, Forest et al is an important paper and Nic’s conclusions are damning. It’s frustrating that, after all the controversy, climate journals don’t require authors to archive data and that IPCC authors continue to “lose” data.
Climate Audit 
97 Comments
The Enron analogies abound. The more the climate science controversy unfolds, the more it looks like a smartest-guys-in-the-room and a too-smart-for-rules (as in ‘too rich for rules’) situation.
BTW, Ken Green weighs in on ‘postmodern climalogogy’/Patridge today at MasterResource: http://www.masterresource.org/2012/06/postmodern-climatology-paltridge/
Amazing that the wider climate science community can stand for this kind of thing.
Myles said “the point is that if journal editors do their jobs, no one should have to resort to FoI at all”. Maybe he would like to contact Dr. Calais to support Dr. Lewis’ request for Dr. Calais to “do his job”.
Yeah, because storing data is so expensive these days.
Samll planet
Dr. Forest is now with the Department of Meteorology at the Pennsylvania State University.
Before that, he was with MIT, his thesis advisors were Kerry A. Emanuel and Peter Molnar.
http://ploneprod.met.psu.edu/people/cef13/
Mann’s little hang-out , now there is a surprise!
‘Consensus’ climate science is basically a speakeasy, you don’t get through the door without knowing the right people and you don’t ever, ever tell anyone outside what goes on in there.
‘What happens at Penn State, stays at Penn State’.
In my house when I’ve “lost” something I usually find my wife has “tidied up” and it’s in a cupboard somewhere quite inappropriate. Has Dr Forest asked his wife if she’s seen some old papers lying around?
Johnbuk,
When I accuse SWMBO of “tidying up” and misplacing something of mine, she shows me immediately where i left it.
I can relate to both replies, does that mean I can get a job with the IPCC.
Now where did I put that application form.
Climate ‘science’ needs fewer scientists and more database administrators.
Nic Lewis is seeking the backup
Two database sets to align
But the doctor (fearing a crack-up?)
Has “lost” it. The paper’s still fine!
Have his shredders now modeled the sea?
All those inferences seem rather bad
If we can’t see Forest for debris
Is his office one ocean of chad?
===|==============/ Keith DeHavelle
There were three authors of Forest 2006. Does anyone else find it incredulous that neither Stone nor Sokolov kept a copy of the raw data they collectively manipulated into a major climate paper?
Being named as co-author doesn’t mean that you played any significant part in the paper.
It is possible that Forest was the only author to see any of the data and that Stone and Sokolov simply acted as advisors.
Sokolov is an expert on the MIT climate model, so he may well have had something to do with producing the raw model data. I’m unsure – he hasn’t replied when I’ve emailed him.
‘Dr Forest reports that the raw model data is now lost.’
Do that as a student and your essay or dissertation is failed, do it as ‘climate science’ professional and its normal practice.
That the professional in this area cannot meet the standard demanded of undergraduates should cause the science community real concern .
But climate sensitivity claims such a Forest’s only aim to shed central light on the “greatest problem man has ever faced,” leading to the diversion (or not) of trillions of dollars of resources, whereas if a student is allowed to get away with such low quality he could end up damaging the reputation of the college for weeks if not months. Forest has lost the data. Deal with it. Let’s all keep a sense of perspective.
Maybe at PSU its difficult to tell the Forest from the tree rings?
“Nic was able to to get data for two predecessor studies. . .”
Just so we’re clear, the data from those two studies were the only, or at least principal, inputs for Forest 2006?
NM, I see it now in the Curry article.
“However, I have been able to obtain without any difficulty the data used in two related studies that were stated to be based on the Forest 2006 data. It appears that Dr Forest only provided pre-processed data for use in those studies, which is understandable as the raw model dataset is very large.”
What exactly is “very large” these days? If it fits on a 128 Gb USB stick I would not call it large. Even 10 needed USB sticks would not make it a large dataset.
But 2006 is already long ago; computers had less CPU power and less memory. I would expect the dataset not to exceed 100 Gb.
Could Nic disclose the size of the raw model dataset?
It would be very interesting to know the exact size of the database.
Data like these are usually text files of ASCII characters, which compresses into almost nothing.
A little perspective. 128 GB is 32 billion floating point numbers. Alternatively, it’s a database of a billion 128-byte records. There’s no way this “very large” database is even in that order of magnitude. Who has a billion records of proxy data?
Knowing these guys, they’re running this on an IBM 360. In COBOL.
Ohferchrissake, a third-tier internet-sales website manages more data than the whole field of climate science, and if it wants to remain in business it does so 24x7x365 with at least four-nines uptime.
eBay crunches more numbers in a day than everyone Kerry Emanuel’s ever known will do in their lifetimes.
“What exactly is “very large” these days?”
what’s generally meant is “how hard to organise, query, update and purge” A set of several terabytes of structered, well related , uniquely keyed easy to index transaction records wouldn’t be big. A much smaller volume of records that have multiple indirect relations, poor keying, difficult to index, update and delete would be “bigger” in terms of the resources necessary to exploit and manage it.
In terms of a rule of thumb I’d say you could consider over 10 Tb as “very Large” even if it’s well behaved, but it can cut-in a lot quicker if your data sources suck..
400Gb for Forest et al 2008, apparently. I imagine a similar size for the 2006 study. If just annual data had been stored (all that was used) then a database only a fraction of that size would have been involved, I think.
If there is 400GB of ASCII text files, that will compress to about 10%, 40GB.
This will fit comfortably on 5 (five) Dual Layer DVD discs.
More data from a seminal paper down the rabbit hole.
At this point, is anyone surprised?
Wonder if Forest is getting advice from Mann.
First ClimateGate. Now AuditGate?
There is a sure fire way to deal with these scientists who refuse to allow their studies and conclusions to be confirmed (or not) independently by critics. Flush them down the proverbial toilet.
Scientific papers are being used to bootstrap EPA rules. The US government can set standards by which it will accept papers, such as those used as a basis for endangerment findings. The standards must set a requirement that the data is available for critics to scrutinize. That would help put an end to “the dog ate it” excuses. Also, any grants which entail publishing papers on results should require the data be made available to the public, or a claw back provision kicks in. That would put this non-sense to bed once and for all.
Scientific Misconduct (Wikipedia)
“Responsibility of authors and of coauthors:”
“Authors are expected to keep all study data for later examination even after publication. The failure to keep data may be regarded as misconduct.”
Link
http://en.wikipedia.org/wiki/Scientific_misconduct
I guess Connolley hasn’t gotten to that page yet.
NicL, son of SteveM. But that’s a chronology thing. Climate sensitivity is far more central to the official IPCC argument since 1992 that CO2 emissions have become dangerous for mankind than the hockey stick. That “The dog ate my data” is even tried in such a crucial place is extraordinary. I said to Nic last July that his findings then deserved a book for the widest possible audience. I’ve not changed my mind. Ridley? Montford? This really matters.
Richard, perhaps you are the one to write up this story. I agree, it thoroughly deserves writing up. And I think it matches your call.
Dog-ate-my-data-gate: one story of why postmodern Climate Science is not trustworthy.
Interesting idea, Lucy, thanks. Josh has told me I should talk to you about wiki and climate some time too. Nic’s story has some way to run right now and I feel I’m still learning about the latest technical developments, including improvements in e-book readers and open source generally, where Google’s new Android tablet this week is likely to be important – or something very like it. But your comment is noted.
The data will be on MIT’s back-ups from the period. And Stone’s and Sokolow’s.
It’s not even that long ago.
If they needed it, they could get it.
Re: Mooloo (Jun 26 00:46),
It is not enough to know that a backup exists somewhere.
Typically in most organizations these days, there is a mass of information available which may or may not have been organized according to the structure and flow of the process used for a particular analytical run.
Unless there is an indexed roadmap into the data archives which links the files and the data stores actually used in a particular analysis run to the versions of the software and/or the external analytical methods that were actually employed, an auditable information traceability path does not exist, for all practical purposes.
My experience is that most back-ups are mirrors, given the relative low cost of storage these days. That is the data is exactly in the form the scientist stored it in, both in file name and location.
I concede that it will not be findable without the good will of both the university’s sysadmin and the academics who stored it, because of the difficulty of deciphering where it is and how it is labelled. But with that it should not be a problem.
(The alternative is to believe that Forest didn’t clearly organise and label his data and his results from the start. Possible, I suppose. But I wouldn’t trust anyone that slack anyway.)
How to make Money in Climate Science
1. find a major unanswered question.
3. question top scientists to see what they will accept as an answer
3. check pick data and methods to arrive at that answer
4. re-label this technique “training” – it makes it sound intelligent.
5. publish the result.
The results will seem correct to fellow scientists, especially those at the top, so they wont bother to check the math. Everyone will be impressed you have answered the hard question. More so because you will have proven their best guess correct and made them look good in the process. You will advance in your career in science. Fame and fortune will follow.
If anyone does question the results:
6. lose the data and methods
Deleted, not lost. Please use the correct terminology–this is science after all.
There is a simple practice that would allow the requirement for data to be held at a journal to be met, that also answers the problem of whether some data is commercially sensitive or is supplied with a confidentiality clause.
Data should be lodged at the journal in a compressed format, but password protected where it is not immediately available for public consumption.
The source data can then be downloaded by anyone at any time, but only made public when the author(s) reveal the password associated.
This can also be used in a way that would pacify researchers who are concerned that they will lose an immediate competitive advantage; they can chose to unlock the data for interested parties after a suitable interval.
In this way, IP is protected where it needs to be, but author’s are accountable for the accuracy of their work WRT data from day 1
If scientists refuse to publicly disclose the data and methods used by their scientific paper, then the default assumption should be that there is something sriously wrong with the paper and that it should be retracted.
Without the underlying data and methods, a scientific paper is no more than an opinion piece.
Chris
And every government entity making policy should formally adopt as standard operating procedure that science which cannot be checked by the public and other scientists isn’t fit to be relied upon.
Secret evidence should never be used against the people.
Yesterday another case of scientific misconduct came up, in the Netherlands. A (now ex-)professor social psychology had massaged data; also some of his data had been lost, allegedly due to a disk crash.
Gen Goldacre covers this case today in the Guardian:
http://bengoldacre.posterous.com/problems-with-statistics-in-psychology-and-ne
“It’s my view that the information architecture of academia is flawed, in several interesting ways, not least of which are that: negative findings go missing in action; and simple statistical errors are missed by journals, while the to and fro between peer reviewers and authors is often dominated by petty disputes, over the contents of the discussion section of a paper.”
I responded there:
Not only negative findings go missing in action. Complete datasets are suddenly lost.
See for instance this article that appeared on ClimateAudit:
(…)
Such alarming facts on climate research do not get proper attention in the news. Recently a climate warming paper by Gergis et al was published and after a few weeks retracted; the publisher says it had not been published, and it was only put on hold.
The Guardian wrote about the paper: “Australasia has hottest 60 years in a millennium, scientists find”. http://www.guardian.co.uk/science/2012/may/17/australasia-hottest-60-years-study
The Guardian has to my knowledge not retracted this statement.
It’s my view that the information architecture of news media is flawed, in several interesting ways, not least of which are that: negative findings on climate research go missing in action; and cases of misconduct in favor of the “good” cause are missed by news papers.
Its a great shame that Goldacre refuses flatly to apply the same standards on climate science for data control and use of statistics , as he does in other areas .
For reasons he never explained it seems some forms of ‘sudo-science’ are acceptable for him . Perhaps the scientific approach is unimportant to him , when its ‘the cause ‘ in play.
I think it’s more visceral than that. Folks like Goldacre and Singh can’t bear to become allies with deniers. There is genuine fear or hatred or a mixture of the two. I can forgive ’em for it but how is it going to change?
KNR – agreed. Climate “Science” seems to be exactly what he demolished in his book “Bad Science”. How on earth does he square that??
To be fair there isn’t much recent climate change stuff on his website, and he left it out of his book(Bad Science) altogether. I don’t think he became a convert, I think he noticed that the climategate e-mails meant that the game was up.
Silence on climate, when the title is Bad Science, is at the least intriguing.
But it’s the silence of the IPCC that really speaks. When experts started to call into question the 1997 report’s prediction of Himalayan Glaciers melting by 2030, in late 2009, Rajendra Pachauri, IPCC chairperson, was lightning quick to denounce this as ‘voodoo science’. But even if glaciers had been melting that fast (it turns out they weren’t, in case anyone has just come into town) that wouldn’t say anything about whether man’s CO2 emissions caused the problem.
Climate sensitivity, on the other hand, is central to the IPCC’s argument about what can be attributed to man and what can’t. So why the silence from Dr Pachauri now? Is what Nic Lewis is saying about Forest 2006 also voodoo science? And even if it was, does the IPCC not condemn the claimed ‘loss’ of data by the authors, a loss that was only discovered a year after they were asked for by someone wanting to replicate the paper’s calculations.
Why is the IPCC silent about this last point? Pachauri has not been slow to engage in such controversies in the past. Does the IPCC not care about the reproducibility of results they have brought to the attention of policy makers in a blaze of publicity and much handwringing about the future of the planet?
Why the silence?
2007 report that should of course be, in the second para.
Just read this from another field of science:
“According to an Erasmus press release, a scientific integrity committee found that the results in two of Smeesters’ papers were statistically highly unlikely. Smeesters could not produce the raw data behind the findings, and told the committee that he cherry-picked the data to produce a statistically significant result. Those two papers are being retracted, and the university accepted Smeesters’ resignation on June 21.”
http://blogs.discovermagazine.com/notrocketscience/2012/06/26/why-a-new-case-of-misconduct-in-psychology-heralds-interesting-times-for-the-field/
No raw data? Treat it like fraud.
I was just playing with my mp3 collection which I’ve had since the early 2000s. How can we take them seriously when some random Joe – or Dave in this case – shows more regard for their copy of ‘The Electric 80s’ than they do for data they expect should affect humanity?
“…researchers who are concerned that they will lose an immediate competitive advantage…can chose to unlock the data for interested parties after a suitable interval.
And who decides “a suitable interval”?
The journal in question could insist on a policy where they decide what “suitable” or “reasonable” means, and insist on access to the password themselves as a condition of publication.
Authors can then weigh for themselves the prestige of being published in the journal against the time limit imposed on exclusivity (where access is granted at their discretion).
That detail (and others) aside, the point is that the authors must lodge the data contemporaneously. The argument that it is sensitive evaporates, and it would concentrate the mind wonderfully to know that some day you’ll need to make good on the claims you make for the date used in your paper.
There can be no “it’s lost” or “I can’t remember”.
Oops “date” = “data” of course.
LOL. Is this like Disney opening the vaults every now and again to re-re-re-release a film on a new format?
Dr Forest has responded to Nic Lewis on the linked “Climate Etc.” thread.
Chris Forest has responded at Climate etc – he says that it wasn’t feasible to store the data back in 2003 (haven’t we heard something like this before?). He also says it should be possible to recreate the data by running the model again.
It’s been noted there that he is a Lead Author of IPCC chapter 9 on evaluating climate models.
I was managing terabyte-sized geographically distributed databases over a decade ago on commodity PC hardware.
Just because he didn’t know how to do it (a failing I think common to the climate-science field) does not remotely mean it wasn’t feasible.
JEM, not all researchers understand this type of thing.
I have all my original spectral and MacLab data for a 1998 publication stored on mini-optical disks in a draw in the lab I worked in three jobs ago.
The format is no longer supported (but we have one working reader/writer device).
I have the text files copied and pasted into Excel, but like I say, the data is sitting on the disc’s which were the ‘next-big-thing’ in 1997.
I know damn little about data storage and hope and pray that the IT don’t lie about the security of my I-Drive.
I have a few thousand images saved, some of which I have used for cell counts, color densities for assays, which have them been used to measure averages/SD’s for publications.
Some are high resolution images, published:unpublished ratio about 1:100.
I am completely in the hands of my IT people as regards to data storage and security. We all have ONE 2GB flash drive we can use to transport data between computer and home. The security system will not allow us to write to any-other device.
It is a failing of many fields; not merely climatology. Neither is it limited to data stored in high tech media either. High tech occasionally makes the scales of the potential losses bigger, but not necessarily more damaging. I had a professor nearly die in my presence when he found out that a university property manager decided that he would unilaterally, without bothering to even notify our department, move an archaeological collection and all the associated paper work including original illustrations, photographic negatives, electronic media and a whole host of other irreplaceable material to what was known as the “boat shed.” The building had only a partial roof, and was in fact turning into a ruin. Notification of the relocation of the collection arrived six months after the move – after a wet winter. Inspection revealed melted archive boxes, mildewed, unreadcable paper records, completelt spoiled negatives and unreadable floppy disks. Having seen the professor recover from what looked like a near stroke, I have been waiting years now to hear about a mysteriour homicide. I did hear that he explaine to the United States Army Corps of Engineers precisely who, by name, was responsible for the destruction of litteraly irreplaceable data, in case they decided to sue.
In fairness, Feasible-at-all != Feasible-for-him. We’ve all done things that we’d identify as stupid / omitted to do things that seemed obvious in retrospect, or at least I have.
I wouldn’t personally see that as a blanket competence or honesty issue.
That doesn’t let him off the hook as far as the implications for losing it are concerned of course, and the points made below about the process itself seeming to be unfit for purpose are well made.
Maybe I’ll try that excuse next time the tax man shows up for an audit. I’ve always wondered whether or not ‘Club Federal Penitentiary’ is actually as nice as the rumors.
This is, first and foremost, a failure of the scientific-publishing process. It’s broken and I’m not convinced it can be fixed without scrapping the whole concept of the paper journal.
These guys are handcrafting buggy-whips in an era of 3D printers. They are dealing with what are by any modern understanding TINY amounts of data. And they can’t even handle that.
On the one hand, the type of data Forest was using is not tiny–it isn’t tree ring data.
On the other hand, this is a critically important paper.
In about 2005 I was doing a project with GIS data (multiple data layers), field data, remote sensing data, and aerial photos. It would be easy to claim “lost”, especially since our GIS guy was a temp and is now gone, but we just imaged his whole computer onto an external drive, which now sits on the shelf.
Haven’t these guys had to worry about computer crashes? There should be regular backups at any facility.
Worth comparing what happened to Marc Hauser when he lost the data that his experiments were based on. He had to re-run experiments that could be replicated and got a scientific misconduct rap for those he couldn’t.
Something tells me climate scientists would not get such a stiff sentence for such behaviour.
I have to grudgingly support Dr. Forest here. Not all researchers have control over their computer resources. Machines get old and IT maybe gets some more up to date machines. Not all of the old data gets moved to the newer machines. Management often has a tin ear when it comes to
data storage, archiving, etc. Document, simulation, data archival and storage and up-keep is all costly overhead. Sponsors hate to pay for this stuff. They want to know, what are you doing for me now, why is it costing so much, and can we do it with some simpler system. Can you do it in Excel? Having said that, you normally document your key assumptions and save the reports for many years afterwards. I have to believe that any constants or input data for present day sensitivity models have been built on the backs of the older models—-the sensitivity parameters must be located somewhere in one of the GCM models. You need someone who runs the models daily to know where to look.
Having said all that, I do believe that maybe starting now basic model inputs need to be archived, and this needs to be funded along with the rest of the scientific work. This needs to be up-front with the proposal.
I agree with Don Monfort. The following is from Nic’s letter to Dr. Calais:
“My primary interest in Forest 2006 is the statistical inference methods used, which I consider to be flawed. I have still not received any data or code from Dr Forest, despite repeated promises to make data available. However, the issues I raise in this letter are more serious than simple failure to provide materials; they concern apparent alteration of data.” (My bold)
“I have to grudgingly support Dr. Forest here.”
Why didn’t Dr. Forest tell him a year ago that the data was lost?
Lets try not to upset the moderators
Perhaps an experiment in telepathy…..
Focus on this space [ ], see if you read my thoughts about someone, supposedly a trained professional who brings the “My dog ate my homework” excuse to the party. If it’s anatomically possible you aren’t focusing or you’ve channeled someone else.
Honestly, we are supposed to believe that he did not have write commit activities with those datum points, really, really…
In the industrial world I’ve had to organise records to be kept for up to seventy years after the closure of the organisation. and that was for fairly trivial liabilities compared to the sums involved in building the carbon free paradise of our dreams. Us rude mechanicals all thought that the academics were meant to be better than us…
michaelo,
teacher says to trained pro: “I have been asking you for your tardy homework for more than a year? I am losing my patience. Where it at?
trained pro: “My dog ate my homework”
teacher: “Why didn’t you tell me that a year ago?”
trained pro: “We just bought the dog last week.”
So to my workday litany of early arriving; slowly changing etc data types, I can now add LACED -> Late Arriving Canine Excreted Data…. Unfortunately there isn’t a standard handling protocol for this that can easily be described…..
Focus on this space [. ]
To all those studying English: important update on the seminal example on the use of the passive voice to evade responsibility:
1987, Ronald Reagan: “Mistakes were made….”
2012, Chris Forest: “Data was lost….”
“Us rude mechanicals all thought that the academics were meant to be better than us…”
My general impression is that the academy is much like business, only with far less oversight and scrutiny.
“My general impression is that the academy is much like business, only with far less oversight and scrutiny.”
There seem to be two distinct paradigms at work. In the commercial world it’s assumed that everyone will take the p*ss given half a chance. hence contract terms , codes of practice, laws and enforcement agencies. Now we are still hard of learning because every time a new market opens up the same standard chain of events happens.
1)New Market opens up, personal restraint is pronounced adequate
2)P*ss taking occurs
3)Market’s main players set up self-regulation mechanism to reassure punters
4)P*ss taking occurs
5)Government re-inforces self regulation by issuing a voluntary code of practice
6)P*ss taking occurs
7)Government sets up legally binding regulations and an enforcement agency but stresses value of “Light Touch” and “Acting Responsibly”
8)P*ss taking occurs
9)Enforcement agency is beefed up, fines are levied, scapegoats and the unswift are jailed
10Level of urine extraction drops to tolerable levels
11)ex P*ss takers to be found on Fox News whining about creativity being shackled by big government
Academia on the other hand is supposedly driven by the noble search for truth, where man’s (insert old joke here about man embracing woman….)better nature is allowed the freedom to improve the lot of mankind.
the chain of events here seems somewhat shorter
1 Make statement declaring academic objectivity, purity of motive, application of highest standards of ethics, proper compliance with sound practice etc
2 P*ss taking occurs
3 repeat 1 ad nauseam
4 appear on the BBC claiming the debate is over.
I don’t think Forest is necessarily guilty of any mendacious behavior. This episode highlights yet again the important difference between academia and industry. The “product” of an academic project is a publication. That is the measure of success. Hence, if data integrity and permanence are not needed for publication, they are not guaranteed.
The “product” of a similar industrial project is not a publication, but is something on which high stakes financial commitment is based. The stakeholders demand data integrity and permanence.
The persistent problem in the climate science game is that the academic practice is wedded to the high stakes financial impact but without the corresponding higher demand for quality. Hardly anyone in academia and certainly no one in the media “gets it”.
If the IPCC had a clue about quality, or one of the high powered national societies that seem to be so important, or the govt funding agencies that spend all the money …..
Just amazing that people can claim that a problem is the most important problem in the history of the world, but not bother with keeping up with the science that supposedly proves their point. These people need an accountant or a lawyer or someone who has actually spent a week in the private sector and understands the real world. The scope of the incompetence is breathtaking.
Even if we don’t expect Dr. Forest to have clue since he’s only a scientist, billions and billions a year from the taxpayers ought to be enough money to hire someone who does have a a clue.
Forest waited a year, and he stalled
Before saying the data was “lost”
And “mendacious” (if that’s what it’s called)
Has in this field one gigantic cost
Nor did he say he can’t pick it out
That he’s drowning in data en masse
His announcement here fills me with doubt:
He’s mendacious, a fool, or an ass.
===|==============/ Keith DeHavelle
That is just more explaining away.
When in my PhD thesis [many moons ago] I put forward conclusions -and outlined their implications- which fundamentally upset the apple cart in the field, as a matter of course first my supervisors and subsequently my opponent -in a public defense of my thesis- asked me to produce the evidence upon which I had based my conclusions. Not only that, they asked me to demonstrate to their satisfaction that the source for my “raw data” was real and reliable. This is elementary. Elementary.
To publish an ostensibly important paper like Forest 2006 without given readers access to the primary “raw” data for verification purposes is an affront to everyone’s intelligence, the basic tenets of science, and a sorry testimony to the state of affairs in “climate science” in particular.
“why should I give you data , you only want to find something wrong with it ” P Jones
That is climate ‘science’ in action form one of its lead figures.
Tom C, your beliefs are not quite accurate. The end product of an academic study is not a publication, because a publication can cause further consequences.
In engineering, for example, the end product is not a published set of drawings of a bridge. Often it is a bridge; and bridges have been known to collapse.
It is helpful to look back at the data to find out why.
Yes, policy makers may just be interested in why they’ve been so publicly embarrassed by the collapse of something they were once told was settled science. They too might be interested to hear that the crucial data has been ‘lost’.
Agree. But if Forest 2006 was that crucial I would have expected at least ten replication studies. A bridge resting on one pillar must collapse whether or not that pillar consists of sand or concrete.
But this way there was only one set of data to lose. That could be called a misfortune. To lose ten, when the moment of truth came near … Lady Augusta Bracknell might well be inclined to call that carelessness.
If raw data is lost or is not produced, any published paper should be required to be withdrawn.
I agree that the climate sensitivity is critical. The matter of its estimation is more than a matter of data gathering, data processing, and mathematical methods, though these are certainly important. It is also a matter of physics. For example the estimation of the zero-feedback forcing is done differently in different models and may be considerably different, depending on how the line shapes are treated. This variability may account for some of the differing climate sensitivities extracted from the various models. Indeed it is possible that the zero feedback sensitivity of 1.1 C, which is widely acknowledged by ‘skeptics’ and ‘believers’ alike, is also up for grabs. Stay tuned.
I’ve been doing research for over 25 years. I still have all of my data. Even the results of failed experiments and data that was never “solid” enough to publish (some day… maybe…).
Another person I’ve worked closely with is closing down her lab and retiring. She’s been going through old data, including boxes of film/slides from the “old” days prior to digital imaging -even stuff from when she was a grad student.
I could give example after example, but one thing I know is scientists don’t “lose” data (barring the occasional computer crash in mid-experiment or natural disaster). And it practically kills them when they realize it can’t all be kept forever. That data is their life.
Prosecutor: Ladies and gentlemen of the jury I want you to find this accused man guilty and hand down a death penalty because I have studied all the evidence very thoroughly and the only possible conclusion is that he is, in fact, guilty of setting an orphanage on fire during their annual Christmas party.
Defense counsel: Where is this conclusive evidence?
Prosecutor: I lost it, but thats ok because I am an honest man.
Jury: ???
The Federal Rules of Evidence State:
If the facts and data cannot be produced, the expert’s opinion is excluded from the evidence.
Richard Drake,
You mention Pachauri’s silence with regard to several recent events. Is it possible that he no longer has the scope of public performance that he once had – limited from above, perhaps?
Certainly Pachauri’s public performances have been considerably limited in the past year (compared to his previous record), so this could well be part of the reason.
Another part could lie in extending the reach of one of their new, improved “guidelines”: “Blogs and other social media” have been declared to be “unacceptable” sources. So he and/or other powers that be at the IPCC may be using this “shield” in much the same way (back in the day when IPCC reports were supposedly “all based on peer reviewed literature”) that they could conveniently ignore that which did not meet their “gold standard”.
Then again, newspaper and magazine articles have also been thrown into the material non grata bucket; yet the IPCC found it necessary to issue a Press Release in response to Fred Pearce’s recent article.
Oh, well … Never let it be said that the IPCC lacks double standards!
John:
Anything is possible. Pachauri spoke for the IPCC in saying ‘voodoo science’ to deniers of Himalayan glacier melting by 2030. I don’t care who the official spokesperson is today. As Hilary points out, the IPCC still issues press releases in response to articles it doesn’t like. So does it or does it not like the fact that Forest claims to have lost his data, a year after he was first asked for it by Nic Lewis?
Unfortunately many academic scientists do not understand the words quality assurance, or if they do they consider that they simply don’t apply to them. This arademic very, very annoyed tellrogance is usually unseen but is clearly evident when the underpinning scientific endeavour is examined, as it incresingly is in the climate area
Those of us who work in science used in regulatory areas such as drug development find it extraordinary that data can simply not be found. If you want to make an academic very, very annoyed – tell him/her that they should be applying GLP principles to their activities.
Had not this science become the basis for “policy” and regulation, I doubt that many of us would be worried about the housekeeping – but it has.
I marvel that some of the leading lights in the climatology business are comfortable “informing” policy-makers (as they like to put it) based on science which cannot be replicated and which has not otherwise been confirmed.
Maybe legislation which proscribes basing, even in part, policy or legislation on incompletely published “science” might go a long way to reducing our exposure to this nonsense.
“400Gb for Forest et al 2008, apparently. I imagine a similar size for the 2006 study. If just annual data had been stored (all that was used) then a database only a fraction of that size would have been involved, I think.”
Well 400Gb of what is the question?
400GB of a SQL compliant database based on uniquely keyed records,correctly defined relations set up as foreign key constraints etc. compress the database, generate the backup file, squeeze that with a zip utility, keep a copy on disc burn another to read only media. simples
400GB of spreadsheets, images,csv files, datalogger records with incompatible file formats, different time signature conventions, overlapping data sets, bespoke code in more than 1 language etc etc, not quite so simple.
Of course the trick is to do the legwork at the start of the project so that at the end you have the simples scenario. Hey if your univerity doesn’t have a database specialist, you can find one in the phonebook.
I think the 400Gb almost entirely consists of the output from about 2000 simulation runs, with varying parameter settings, of the MIT 2D model. All should have identical data structure and format.
“I think the 400Gb almost entirely consists of the output from about 2000 simulation runs, with varying parameter settings, of the MIT 2D model. All should have identical data structure and format.”
So the bit that *actually* needs to be saved is the approx 400Gb/2001 which should represent the raw data that comprises the model input. About 200Mb. Even if that was a bunch of small messy text files, it wouldn’t be difficult to zip/tar it into a single achive and preserve it.