My point is that there is an expectation that there is enough information in a published paper so that someone who was trained in that field can replicate the data.

I think this is a problem. There is an increasing expectation that journals do not provide the necessary information for replication. Controls, the experiments done; mere detail that a scientist in the field could replicate. I remember Nature showing the creation of knock-out mice with a single sentence; this is a procedure that takes ~2-3 years! Wholly inadequate.

However, it is when something goes wrong that a thorough description of methods pay off. You then have a hope of working out what went wrong, with resorting to quasi-litigation just to find the barest details.

It is also a lot easier if the experiments are fairly simple, with a fairly direct route from the data (which may be shown) through to analysis. However, once you start to get complex analysis (e.g. ratios, mathematical transformations) , the number of ways of perverting a data set becomes large. I am familiar with a whole host of abominations by way of data analysis, all under the murky cover of inadequately described data and methods.

per

World Weather Records
http://www.ncdc.noaa.gov/oa/oldpubs/#WWR

Monthly Climatic Data for the World
http://www7.ncdc.noaa.gov/IPS/MCDWPubs?action=getpublication

Meteorological Data for Individual Years over the Northern Hemisphere Excluding the USSR
(an approximate title)
http://meteo.ru/english/data_b/sp.php?id_article=170

Unfortunately the data for some of these is not actually online. Of course they have only 384 stations in their pool, so it is not clear which ones were selected.

I think the open audit as practiced here is more valuable.

I admit it is messier and more time consuming. However, sometimes value has its price.

Relative to climate science we are now in a period of “buyers remorse” for failure to do our audits earlier. However, since the technology wasn’t available earlier I think the audit community can be forgiven. However, hanging on to old styles is a mark against it.

In addition earlier open review will help good thinking propagate faster. No need to wait for the official review.

If I said that the torque on the bolts holding an anhydrous ammonia line together was “to be reasonably” tightened, and it failed, then I could be sued and jailed for criminal negligence. Not to mention that I would have to specify the gasket, and materials of construction, wetted and non-wetted surfaces.

gb August 25th, 2007 at 5:23 am If these journals were

serious scientific journals

how could they allow such handwaving; when the product of a scientific enquiry is supposed to be repeatable and verifiable.

Oh and by the way,

I don’t believe in the value of an audit but (by) just some person

. One of the nicest things about an audit is that it is auditable. Audits have been thrown out when the auditors did not do a proper audit trail in their work. So, if you think an auditor may not have value in their audit, you need only audit the audit. If they can’t produce the materials for you to properly audit their audit, you should discount any claims made. You can also provide proof that the auditers were wrong and have them change the product or comments. I have done this several times.

Steven Mosher, it is clear that you never have submitted an article to a serious scientific journal. Editors from the more high quality journals reject sometimes 50% of the articles they receive, or even more. My experience is that in general at least one of the reviewers is (highly) critical and it costs me a significant amount of time to satisfy the reviewer(s).In general the quality of my articles has improved significantly because of the comments of the reviewers. But is also clear that the best reviews are written by good scientist with a lot of knowledge about the topic, so therefore I don’t believe in the value of an audit but just some person.

Many scientist agree that the review process is not ideal. But they also know there is no good alternative. It doesn’t prevent bad science but is prevents many bad publications.

I’ve seen this in the software industry where people protect “thier” code as if it were an arm or a leg. When these people are laid-off their first concern is about what will happen to “their” code; they want to take it with them and they can’t so it’s very traumatic.

Good scientists and engineers have learned to keep their work seperate from their egos. You can easily tell who these people are: they ask others to inspect their work. Their focus is on getting the work as correct as possible, and generally their work is of a higher quality as a result.

When you are testing a drug in an animal what is the protocol?

x mg/Kg? or a reasonable amount per Kg?

What would the FDA say if you told them that a reasonable amount per Kg had x effect? Approved for human use? Or suppose in an animal test you told the FDA that you injected a reasonable amount of drug y at some unspecified consistent interval and the results wer a not inconsiderable reduction of an induced tumor. You going to pass that to the FDA reviewer?

Those guys want specifications that can easily be replicated. They want numbaz. Lots and lots of numbaz.

I know that if I give a bolt tightening spec of a “reasonable torque” the FAA would laugh me out of the room. My design review peers would have drilled me a new one.

Or suppose I describe a breaker as tripping at an excessive current but holding at reasonable current? You gonna fly on an aircraft built to such specs? Or suppose I said the minimum air speed for take off was reasonably fast. The FAA going to buy that? Or suppose I say that the captain of the aircraft must abort a take off if he hasn’t reached a reasonable speed while there was still considerable runway left for braking? Want to fly on that one?

What makes “climate science” different?

RE M. Simon #262: “When your Dr. prescribes medicine does he say you should take a reasonable amount for a reasonable length of time? Or does he say two pills four times a day for two weeks? Would you want your Dr. prescribing according to “climate science” methodology?” Ooh – not so good analogy. Even a 10-day bid course is an approximation at best. The dose of medicine is not adjusted for body mass and the pharmacokinetics of a drug in me is not the same it is in you. And despite large clinical trials, the safety of any given drug can vary from person to person. I suppose to apply the standards that are demanded for climate science the drug course in any person would be carefully calculated – maybe after a quick PK study.

Excellent! you made my point even better than I did.

Pay your taxes without coercion? I’ll believe that when you tell me you sent them 30% more than they asked for.

Maybe it’s just my POV coming from the world of biology which has greater complexity and messy data due to so many biological and biochemical interactions and individual variation. Despite this, we have learned a lot about what happens in biological systems and we base decisions on what we do know.

And I pay my taxes cheerfully, without coercion. After all, the taxpayers paid for my graduate education, which was at a land-grant state university (getting an excellent return on their investment, BTW), and it’s the least I can do.