I tried to deal with your concerns in my original post #27 by careful wordsmithing. I used the term “pure science” to distinguish the curiosity-driven tradition of Western science that long pre-dated (but has continued alongside) the practical application of research to worldly problems. Subjects such as paleoclimate reconstruction used to be–along with stellar astronomy, paleontology, linguistics, much of mathematics, etc.–matters of scholarly or “academic” interest, whose broader implications, if any, were restricted to aesthetic or philosophical implications. The people pursuing these subjects formed small and fairly isolated communities whose passions were focused on figuring stuff out and getting credit for it from the other members of their community. No third parties were on the hook for their (or the community’s) misjudgments, and the norms of their scientific fields evolved under these influences.

Now things have changed. As Bill F so aptly put it above, “Since climate science has passed beyond the realm of a “pure science” (where assumably nothing really earth shattering is riding on the conclusions other than bragging rights at the post-conference trip to the hotel bar), and into the realm where governments are being asked to make multi-TRILLION dollar decisions based on it, I don’t think it is too much to ask for climate scientists to start being a little more diligent and lot more open about how their data is collected and evaluated.” Applied science as an input to engineering or economic or military or public health problems is a different matter from pure science. Such applied science sounds more like your experience as a geochemist in the mining business–whether you are right or wrong has fairly immediate and presumably somewhat traceable consequences for other people’s money and careers.

BTW, a good if somewhat Panglossian take on the norms of pure science and their self-correcting character can be found in Michael Polanyi’s old essay The Republic of Science. A longer, and much less Panglossian, but very rigorous and eye-opening account is given by David Hull’s Science as a Process, using the field of systematic taxonomy as his example. Hull was pretty close to the events he describes and he knows where most of the bodies are buried in his chosen community. One interesting point in Hull’s description and analysis is that cutthroat prima donna behavior, some more extreme than the stuff that Steve rightly complains of in climate science, was also endemic in systematic taxonomy. Hull argues that on balance this “bad” behavior served an evolutionary purpose in facilitating the spread of new concepts and techniques; the insights of nice guys were often ignored even when relevant to ongoing controversies. But it is doubtful that that behavior would be functional in the context of an applied problem with external clients.

2)What requirements should law or custom place on Al Gore, Dr. Hansen, the IPCC, or any other organization or person that attempts to scare society into passing expensive laws that are purported to be necessary to avert a planetary catastrophe ?

Dr. Hansen is an employee of NASA. NASA’s standards are very similar to the FAA’s. I’m currently investigating his compliance.

Mr. Gore is a politician.

The IPCC has its own rules. There has been some investigation posted here regarding how well they follow them. I believe they should be held to standards as I mentioned above, or disregarded. Persons participating in the IPCC are doing so by choice, and should therefore be treated in the same manner. Further, I believe the IPCC should not cite as supporting evidence studies which do not implement Quality Assurance of some kind.

Some researchers offer legal testimony as expert witnesses. I have no mercy for sloppy work there.

For the remainder, I recommend awareness of the situation. Some parties have made the consequences of error exceptionally severe, even beyond any aircraft. If nothing else, I appeal to their professionalism. Use the methods that have been found to be necessary in situations where technical errors can get people dead.

3) NSF is not supposed to negotiate away the data, but what if they do so ? Saying that NSF is breaking its own rules isn’t likely to get you anywhere in a court of law, vis-a-vis the obstructive researcher.

That’s not necessarily true. If I recall correctly, the EPA has been successfully sued many times for breaking its own rules. It might not affect the individual researcher, but it could affect the NSF.

This post and following endorsements trouble me. I appreciate that they are thoughtful, but they over-generalise. Like Steve, I’m from a mining industry background. We worked under mandatory requirements of materials storage (like drill core) and prescribed reporting standards and intervals. Full disclosure unless excepted for competitive reasons. Jail for fraud.

As a geochemist I’m classed as a scientist, but in reality the work requires interaction with engineers and preparation of work in readiness to hand a payable discovery to the mining engineers. So, the scientist has to walk the walk and talk the talk of the engineer, to a fair degree. No point handing over a deposit with incomprehensible documentation. So the gulf between science and engineering is context-sensitive. I wish people would not play it up. Really, engineers and others make aircraft that use clever materials like polymers that chemists design. I was even seconded to manage a complex engineering pilot plant that used 10 tonnes a day of chlorine gas at 1050 deg C – in a town. There is large overlap between science and engineering.

My problem is with the standards of science conducted by diverse disciplines. Having worked since the mid-60s and seen the emergence of green activism masquerading as science, green science started from a very low base and is still struggling to get recognition. Much of the climate related green science is in this class. A principal failing is lack of accountability. The wildest of claims have been made with no apparent sensitivity to their consequences. Names need not be mentioned.

The more demanding and high quality branches of science seem to attract people who face the challenge that they must succeed to ensure continuing income. Look at the very low disaster rate among the producers of chemicals, like pharmaceuticals. Their inventor scientists know that their employers will go insolvent unless they produce worthwhile goods at frequent intervals. Yet, there is an active conglomerate of green activism “scientists” trying to stop vital actions like vaccination.

But this is a bit of personal philosophy. The real game is to get full accountability, full disclosure, higher standards of proof, etc as have been the core of CA concerns.

And yes, you are right. Scientists do not get the right answer in the way that mathematicians can. Scientists should be humble enough to know that perceived wisdom is often overturned and that they should strive for a ‘best result’ but never an ‘irrefutable truth’. It is pleasant to read a top paper by a top scientist, with clarity of expression, disclosure of doubt or WIP, neat design, logical conclusions from unstressed data, etc. Few climate science papers I have read yet fit these ideals.

1) I realize that you have studied this area of law

2) I am in complete agreement with you, Mr. Pete, and others as to what SHOULD happen re: disclosure of federally funded data and backdoor 4AR reviewer comments. I’ve stated so in the past.

3) NSF is not supposed to negotiate away the data, but what if they do so ? Saying that NSF is breaking its own rules isn’t likely to get you anywhere in a court of law, vis-a-vis the obstructive researcher.

4) I don’t believe these legal issues are so cut and dried, once the situation gets mucked up. I believe almost all primary grant applicants make disclosure agreements but the same relationship might not exist for all researchers who receive their funds indirectly through another institution. Our discussion was about general rules of disclosure for climatologists, not specific cases.

5) I argued that there is a general professional burden of production for all scientists [regardless of experimental cost], and a separate legal burden for those who accept federal money to do work. I find it hard to understand which of these points is “theoretical”. I guarantee you a judge will see it my way.

6) The professional and legal duties are clearly separable. We should expect published papers to be reproducible, even if they’re privately funded. Privately funded researchers have zero legal duty to disclose source data and code.

I’m shutting up, now.

Steve: Jim, there are layers of issues, but one of the key issues is simply placing sunshine on NSF’s failure to ensure that its grant recipients comply with existing federal policies. As you say, if NSF is negligent, it wouldn’t get anywhere in a court of law with the obstructive researcher, but my hope is that it will get somewhere in the court of public opinion and ultimately force NSF to obey the law and require compliance with US federal policies. Sure, there would still be some non-compliant researchers; it wouldn’t affect the Jan Esper and the Euros, but it would be a start.

2)What requirements should law or custom place on Al Gore, Dr. Hansen, the IPCC, or any other organization or person that attempts to scare society into passing expensive laws that are purported to be necessary to avert a planetary catastrophe?

Anybody who has ever maintained any equipment knows 2 “rules”

Rule 1. Before you start to troubleshoot, be sure that what you have is an actual equipment malfunction. I can’t tell you the number of times I was able to chalk up the problem to “operator error”, or NTF – No Trouble Found.

Rule 2. Accurate measurements must be made, and compared to the nominal values. It doesn’t do you any good to troubleshoot if the test equipment isn’t calibrated. Also, zero must be known and common to all measurements. Only then can you see how far off you are (if any), and repairs or adjustments can be done.

I would not be able to test a radar with missing tech data, a different zero used for each measurement, data that has been “adjusted” after the measurement has been taken, etc.

And my chief of maintenance would have been fired if his comment was “Why do you want to work on the equipment? You just want to find something wrong with it”…

Would you put a loved one on a commercial jetliner designed or built with this level of rigor?

In the case of the FAA processes I referred to, they’re not there to to guard against fabrication. They’re there to guard against mistake and error. The accusation is that the scientists are human. That’s not an insult. It’s just something we’ve learned, with lessons paid in blood.

Scientists don’t design airplanes. They do little experiments.

Airliners are designed by teams of engineers in consultation with MBAs, PhD economists, and the intended purchasers of the planes in order to provide a safe and cost-effective transportation platform that the manufacturer can sell.

The difference is that chemists aren’t saying we need to spend $45 trillion, drastically alter our lives or prosecute skeptics.

You’re mixing up two important points, here.

1a) What requirements should law or custom place on scientists to publish the results of their experiments; 1b) should climatologists be singled out for a higher burden ?

2)What requirements should law or custom place on Al Gore, Dr. Hansen, the IPCC, or any other organization or person that attempts to scare society into passing expensive laws that are purported to be necessary to avert a planetary catastrophe ?

I will repeat, the answer to 1a / 1b is published results should be described sufficiently to allow reproduction of the result, period.

The answer to 2 should be more than good graphics, rhetoric, and hot chicks in tie-died T-shirts.

Fine. Ice cores are expensive to collect. The government has rules requiring disclosure. We don’t enforce the rules.

Be careful not to conflate 2 issues [I'm not saying you are...]. If Dr Thompson is self-financing his research [Of course, he relies on grants...], then the expense of replication is irrelevant. He should have no general duty to go beyond publishing a reproducible description. The data would be his to lock in a safe or flush down the toilet. If the government pays to build a particle accelerator or finance an ice-coring expedition to Antarctica, however, then Uncle Sam should be able to enforce whatever disclosure terms were negotiated with the researcher.

This is a separate duty that has absolutely nothing to do with the expense or public policy considerations of Thompson’s work.
I hope everybody can agree that researchers must be held to their agreements to disclose data and methods [IF an agreement exists]. Please don’t hit me with the FOIA argument b/c it begs the question of “What is the gov’t buying with its $, data or services ?” A lot of grants fund exploratory work but no final paper – so I’d guess in many cases there is a bargain for effort, not results. In those cases the gov’t would have no right to data. [I'm NOT saying it's OK to make such an agreement, just that the parties should be bound to the agreement they actually made.]

Even if there’s no agreement to disclose data, there’s nothing stopping Uncle Sam from using eminent domain to force the sale of the undisclosed data. What’s the fair market value of unused data sitting in a filing cabinet ? $1000 ?

Ahhh… but what if Dr Y cannot reproduce the result? Without access to X’s data and methods, we must either accept X’s result as conflicting — on faith — or we must toss X’s result.

If Dr. Y can’t reproduce the result, he should publicize that fact amongst other researchers.
The controversy would likely draw more investigation into the cores and the analysis techniques, as I stated. In the meantime Dr. X’s result would be suspect; what’s wrong with that ?

We’re not asking for MORE than reproducibility. We are asking for that much however.

I’m glad that we agree. That’s what I said in #39:

The standard for climate scientists, as with all other scientists, should be reproducibility – nothing more.

Asking these paleo guys to provide ID and version of data, as well as what math they used to analyze the data is just forcing them to report a reproducible result. There’s no reason to force them to do more.

Steve: Jim, I’ve reviewed and excerpted US federal law on data archiving. I agree with Pete’s point and do not agree that he is conflating anything. There has been a very clearly stated federal policy requiring data archiving that has been in place since 1991 that applies to recipients of NSF grants (which includes Thompson). This isn’t up to the scientists. NSF isn’t supposed to negotiate with the scientists; NSF is supposed to enforce data archiving policies put in place at a senior level. It doesn’t. I agree with Pete’s point. In this case, I’m afraid that you’re arguing a theoretical point that’s not at issue.

Asking for more than reproducibility is insulting, in my opinion, b/c it’s a backhand way of accusing scientists of fabrication.

In the case of the FAA processes I referred to, they’re not there to to guard against fabrication. They’re there to guard against mistake and error. The accusation is that the scientists are human. That’s not an insult. It’s just something we’ve learned, with lessons paid in blood.

I don’t see why climatologists should be treated differently.

The difference is that chemists aren’t saying we need to spend $45 trillion, drastically alter our lives or prosecute skeptics. “Extraordinary claims require extraordinary proof”. Those are some pretty serious claims. I recognize that not all climatologists are making these claims, but the loudest voices are. If you think it’s not fair, you’re free to take it up with Dr. Hansen.

We save photos of particle collisions b/c it’s insane to force another scientist to build their own particle accelerator. We don’t force chemists to prove through documentation that they ran an experiment, however.

Fine. Ice cores are expensive to collect. The government has rules requiring disclosure. We don’t enforce the rules.

If Dr. X chooses not to disclose all of his data and methods, and Dr. Y goes to the same location and reproduces the result, there’s nothing stopping Dr. Y from publishing all of the data retrieved from his ice cores + the math methods he had to use to repoduce X’s result.

Ahhh… but what if Dr Y cannot reproduce the result? Without access to X’s data and methods, we must either accept X’s result as conflicting — on faith — or we must toss X’s result. This is a science blog, so faith-topics are banned. I take it that “accept on faith” is probably not an acceptable course of action.

…we take them at their word until somebody raises questions of reproducibility.

The question has been raised, endlessly. Look at Craig L’s work, for example. It contradicts other proxy studies. Who is correct? How can we know anything at all if those who sit on one side of the table refuse to turn over their cards?

Eventually, all the cards must be turned face up. It’s time.

To NOT do so is to act as if one has something to hide. Perhaps there’s nothing to hide, but the actions of such a one are indistinguishable from one who DOES have something to hide.

This is why sunshine is such good disinfectant.

Yes, we believe the best of everyone. “Trust but verify” is very good medicine for science.

Asking for more than reproducibility is insulting, in my opinion, b/c it’s a backhand way of accusing scientists of fabrication.

We’re not asking for MORE than reproducibility. We are asking for that much however. It is insane that Steve must go to ridiculous lengths to reverse-engineer others’ methods and data. These are not trade secrets, they are publicly-funded scientific investigations, explicitly designed to inform and influence major investments in our planet.

Thus, without the original data, nor access to their code/selection/etc criteria, it is impossible to replicate.

If this is so, then that would be a publishable result.

I’m not apologizing for any particular scientists, but I do see a difference between 1) actual data collection + analysis / data manipulation, and 2) pulling some unidentified fraction of historical data from the ether + analysis / data manipulation.

Whether any given type of proxy is good or bad, variable or consistent, if “good enough” spatial sampling conditions are described in a publication, the experiment can be reproduced.

If the experiment’s reproducible, the truth can eventually be determined.

It may be that if two ice cores are taken 30 meters apart, you get drastically different results. If so, a person who attempts to reproduce work will discover this and publish. Then the next question will be are the ice cores really different or are there big differences in the way Drs. X and Y are manipulating the data ? Those are the sort of scientific question that can be resolved in a reasonable amt of time.

If Dr. X chooses not to disclose all of his data and methods, and Dr. Y goes to the same location and reproduces the result, there’s nothing stopping Dr. Y from publishing all of the data retrieved from his ice cores + the math methods he had to use to repoduce X’s result.

There are good reasons to ID a particular tree during sampling. You might want to know, for example, if sampling ancient BCPs could lead to eventual damage to the trees. Where you and Steve re-cored some Colorado BCPs and could show directional problems with those trees – that was fantastic.

But even if the tree IDs weren’t reported, it’s enough to core lots of trees in the general area and show that a prior researcher’s data are full of a suspicious number of outliers. Why is it that a random sampling of trees in the area has a std dev of 4.2 units, but researcher X was able to find a group of trees with std dev = 0.12 units ? The obvious implication will be that X didn’t report all of his data.

Asking for more than reproducibility is insulting, in my opinion, b/c it’s a backhand way of accusing scientists of fabrication.

We save photos of particle collisions b/c it’s insane to force another scientist to build their own particle accelerator. We don’t force chemists to prove through documentation that they ran an experiment, however. If they provide a spectrum and say it’s from analysis of compound Z under a repeatably defined procedure, then we take them at their word until somebody raises questions of reproducibility. I don’t see why climatologists should be treated differently. The solution is more scientists trying to replicate experiments.