To Scott Brim, and I quote you:

“It means that a thorough and objective look must be taken at the research work that was done, and at how it was done, before the work can be accepted and relied upon as the foundational basis of other follow-on work efforts — including the addition of such caveats as may apply in performing subsequent work which uses the research product as a basis.”

It is difficult to produce a more succinct delineation of what actual advancement of comprehension and eventual understanding of a matter comprises than your words, especially in what has become such a contententious and regrettably politically and economically loaded situation as this. This is exactly what “science” is, should be, and will in the end be, and acheive.

The present political position has been adopted before the scientific basis or “knowledge” was present, and is now (albeit slightly shakily for the moment, post Climategate and IPCC debacles, ) being uneasily continued . But there we are – politicians have been swayed. A consensus was pronounced. Just as group think occurred with the hockey team, so it occurred with the politicians, of almost all colours.

Sadly the matter is in the hands of the politicians, and thus bureaucracy (as evidenced by recent whitewashes) and importantly the concommittant media. With time and continued pressure from logic and “science” , the politicians will shift, and the rest follows. That it is the hands of politicians means that time must pass. They do shift, but slowly.

As you rightly observe there are wide nuances to the interpretation of the meaning when the public use the word “science”.

Even within the common definition of what the term “science” means as a working philosophy, there are elements of the philosophy that may be interpreted differently depending upon the working context, or at least to be given greater or lesser emphasis depending on the field of study or the type of work being performed — basic research versus product engineering, for example.

As used in the nuclear industry, the term “science” is somewhat equivalent to the phrase “process rigor and discipline, end to end.”

When we draft a new QA checklist such as the one shown above, the initial version is reviewed both by the QA auditors and by the potential future auditees in the line organizations. Many times after a review of the initial draft, the checklist is expanded to cover a finer level of detail or to add new sections. One candidate for addition to the Temperature Reconstruction QA Checklist shown above might be a QA Evaluation Summary/Conclusion section.

In the nuclear industry, when a QA checklist indicates that one or more deficiencies exist for the particular component or the particular document being evaluated, final acceptance of the component or document is held up until all the deficiencies are corrected. That means all of the deficiencies, without exception, because the spec is the spec, period.

However, in performing scientific research, some deficiencies are systematic in nature and are not correctable, per se. This does not mean that such deficiencies should be ignored in coming to some final conclusion as to the overall quality and reliability of the research product.

It means that a thorough and objective look must be taken at the research work that was done, and at how it was done, before the work can be accepted and relied upon as the foundational basis of other follow-on work efforts — including the addition of such caveats as may apply in performing subsequent work which uses the research product as a basis.

The above QA checklist provides a rigorous, structured, and objective approach to making such an objective evaluation.

TEMPERATURE RECONSTRUCTION QUALITY ASSURANCE CHECKLIST

Congratulations on this set of crystal clear and logical series of statements.

While it would be a considerable undertaking to complete your comparison process, and require a very large knowledge base (which sadly I lack in some of the fields involved), I sincerely hope somebody can do it. The results would be invaluable and my guess highly publishable in a main stream journal, WHATEVER THE FINDINGS.

Even if the “climate community” editors didn’t like its findings, and therefore closed ranks against publishing, a version on Climate Audit would attract enormous attention.

Thank you !

Iain

Craig Loehle says, “Yes. Please note that when I did my reconstruction I wished to show that leaving tree rings out leads to a very different picture. I did not claim my result was precise or reliable because the data going into it are neither.”

In the nuclear industry, we have a widely-used tool for assessing the quality and reliability of a component item or a document deliverable, the Quality Assurance Checklist.

Because we are doing things with nuclear-quality rigor and discipline, we have quality assurance checklists for everything under the sun. If I had $1000 for every check-off item on all our various QA checklists, I could afford to build my own nuclear reactor.

Now, the AGW partisans tell me in no uncertain terms that Mann’s hockey stick is universally accepted as confirmed fact within the climate science community; and furthermore, that Mann’s past work on the hockey stick meets all generally-accepted standards for rigor and professionalism in the pursuit of scientific research.

They are particularly vocal when it comes to disputes concerning the Medievel Warm Period, saying that the MWP, if it existed at all, occurred only in a limited number of locations in the Northern Hemisphere.

They say, “If you don’t like Mann’s prior work, then go do your own temperature reconstruction and get it properly peer reviewed by the climate science community.”

What they are really saying is this: In their humble opinion, the Mann hockey stick will not be deposed from its status as the generally-accepted temperature record for the last 2,000 years unless some major new study, one conducted by people with recognized stature in the climate science community, comes to a different conclusion.

Well … Along comes the Loehle-McCulloch temperature reconstruction (2007/2008) which indicates the existence of a well-defined worldwide Medievel Warm Period.

How should an interested and informed observer go about evaluating the Loehle-McCulloch reconstruction, and the approach it uses to reach its conclusions, in comparison with the Mann hockey stick, given that the Loehle-McCulloch team currently has no stature whatsoever within the mainstream climate science community?

Nuclear person that I am, I would like to see a formalized comparison of the two reconstructions performed, using a tool which is essentially a quality assurance checklist.

The “Temperature Reconstruction QA Checklist” would directly compare the two reconstructions, side by side, using a structured topical outline as follows:

TEMPERATURE RECONSTRUCTION QUALITY ASSURANCE CHECKLIST

A) Temperature Reconstruction Process Flow Sheet:
– A process flow sheet referenced by other parts of the QA Checklist which describes, in general terms, each set of steps in the reconstruction and their sequence.

B) Calibration Period Data Sets:
– Enumerated list and description of the calibration period data sets.
– Reference index as to where each calibration period data set is employed within the reconstruction.
– Provenance and availability of each calibration data set; adequacy of document archives.
– An assessment of the quality and reliability of the calibration period data sets, individually and as a whole.
– An assessment of the adequacy of coverage of the calibration period data sets, individually and as a whole.

C) Proxy Data Sets:
– Enumerated list and description of each proxy data set.
– Reference index as to where each proxy data set is employed within the reconstruction.
– Provenance and availability of each proxy data set; adequacy of document archives.
– An assessment of the quality and reliability of the proxy data sets, individually and as a whole.
– An assessment of the adequacy of coverage of the proxy data sets, individually and as a whole.

D) Calibration Period, Process and Methods:
– Enumerated list and description of each calibration method and/or technique.
– Reference index as to where each calibration method or technique is employed within the reconstruction.
– Provenance and availability of each calibration method or technique; adequacy of document archives.
– An assessment of the quality and reliability of calibration process methods and techniques (Process V&V).
– An assessment of the applicability of calibration methods to the specific data sets being employed, both individually and as a whole.

E) Proxy Analysis, Process and Methods:
– Enumerated list and description of each proxy analysis method and/or technique.
– Reference index as to where each proxy analysis method or technique is employed within the reconstruction.
– Provenance and availability of each proxy analysis method or technique; adequacy of document archives.
– An assessment of the quality and reliability of proxy analysis methods and techniques, both individually and as a whole (Process V&V).
– An assessment of the applicability of proxy analysis methods to the specific data sets employed, both individually and as a whole.

F) Appropriate Application of Analytical Tools:
– Enumerated list and description of each analytical tool.
– Reference index as to where each analytical tool is employed within the reconstruction.
– Provenance and availability of each analytical tool; adequacy of document archives.
– An assessment of the quality and reliability of the analytical tools, both individually and as a whole.
– An assessment of the applicability of the analytical tools to the data sets being employed, both individually and as a whole.

This QA checklist would be just the ticket for promoting an organized and objective approach to determining the quality and reliability of the two competing temperature reconstructions — for those who care about such things, anyway.

The work is then in putting these together. This was the novel approach where MBH98 could have added to the science.

For those who say that it the above project is incomplete, could start a competing one. Maybe include a members only part with a repository of the data.

Isn’t it funny how we’ve come to anticipate between little and no editorialising from Steve McIntyre and yet have long fallen silent on demanding the same restraint from the RealClimate gang. The adjective count may be a poor proxy for quality of substance, but our reaction to a high count is an excellent proxy for our expectation of content integrity.

you have convinced me as much as you have convinced yourself that strip bark chronologies are unlikely to produce the relevant information. Even a priori, they don’t look too sensible.

But I would warn against such fast conclusions. Any method to get some information from the distant past is bound to look bizarre or “manifestly” wrong at some level – and for “special” reasons, individual examples how to use the methods may fail. However, some of the methods may turn out to work.

I don’t understand why the PCA became a taboo on the Climate Audit. This is a major argument that the proponents of such methods consider to give the weight to the analyses which is why people should know whether various statements about it are right or wrong.

In particular, I am convinced that the Joliffe statement above is just noise. Of course that sometimes, under some special circumstances, the highly subdominant PCs may be as important as the dominant ones. But the very meaning of the eigenvalues is to separate those that are more important from the others. So the generic situation is that the subleading ones may only be important for a problem if all the previous ones and their linear combinations are shown to be unimportant for the problem.

A reference to authority – and Joliffe would not necessarily be an important one – can’t change this fact of linear algebra.

Best wishes,
Lubos