Armand MacMurray (#57):

You say that this would be “easy to cheaply fulfill by setting up a website containing the various files” — unfortunately, you don’t understand the magnitude of this task. Our project covered about eight years. The work resulted in over 2000 computer files in 140 folders. It also generated about 40 cm thickness of hardcopy, of which little is in digital form. All of this data (which is well catalogued) will hopefully be kept for posterity in case it is ever required for serious research — but I’m not spending my time putting it on the web (EVEN for serious researchers)!

per (#74):

You say “What SM has tried to do is to repeat the exact experiment reported by MBH. Likewise for John A and John Hunter’s data set.”

Rubbish — John A is clearly not prepared to “repeat the exact experiment” — all he wants to do is find a few mistakes in my arithmetic — not to actually go out and do data collection.

John A (#76):

In response to my question “where is there a profession, anywhere in the world, in which the practitioners are expected to provide intimate details of their work to anyone who cares to ask, no matter how unqualified or incompetent that person may be, and at no recompense?” you reply: “Well, you’ve pretty much described most democracies.”

Again, rubbish — ever tried to make an FOI request? Do you even have FOI legislation in the U.K.? (I think you do, but only quite recently.) Is an FOI application free? Do they require no justification? Are there no exclusions?

Finally, you add “Actually I can do better than that. I can actually add value to the reconstruction. Bet you’re curious how, aren’t you?”

NOPE. I remember Richard Courtney giving a similar veiled threat before John Daly launched his final onslaught on our work — it was the usual flannel, although it did waste a bit of my time in responding.

Also, I’m sorry to tell you that our work was not a “reconstruction” (at least in the normally-used sense) — it was basic data collection and analysis.

David H (#81):

One of the particularly distasteful things about climateaudit is the willingness for posters to pretend A has said X when A quite clearly said Y. You now refer to “John Hunter not disclosing his tide gauge data from Port Arthur”. I HAVE indicated where the tide gauge data may be obtained (see #32). However, it would not do you a lot of good, although you could check that I can do a tidal analysis properly (that is, if you or John A know how) — but you certainly wouldn’t get to our final results. What you would need to do is ferret through the documents contained in the 0.5 Gb of 2000 files of data which relate to our 8 years of work. It IS traceable, but would involve a significant amount of work — and I’m just not prepared to put my time and effort into it.

Finally, you say “I’m glad these guys are in climate research and not in medicine or building jumbo jets.”

Now I guess many of you guys consider flying safely is more important than worrying about whether the world is going to get a few degrees warmer. Aeroplane safety is a REALLY serious topic — it affects most of us. So, Steve, how about:

http://www.jumboaudit.org?

Let’s all write to Boeing saying “aeroplane safety is of concern to us all, so can you please point us all to the data and full methodology on the subject of building a Boeing 747, for the purposes of audit and replication” (to steal from John A).

Gees you guys live on another planet …..

If I run a sweet shop here in the UK and I put in my tax return the first thing the tax office do is to put all my numbers into their computer and see if they get the same tax due as I do. This is not an audit but a simple check. If my return adds up they don’t just file it. They look at some key data like return on sales. They have a pretty good idea from other sweet shops what it should be. Suppose they are not happy. Its no good me telling them like John Hunter and Michael Mann do, to go set up their own shop and get their own sales and purchase invoices. They want mine. And if I’ve lost them or refuse to hand them over I get fined.

In the case in point, John Hunter not disclosing his tide gauge data from Port Arthur like Michael Mann’s refusal to disclose the programmes and data enables them to say (if others get different results) you didn’t do it right. The first step and it is only the first step is to check the sums. What is the point of arguing about tide gauges if the sums were wrong anyway?

I’m glad these guys are in climate research and not in medicine or building jumbo jets.

In this sense, Mann and his cohorts are correct in claiming that you (M&M) have produced your own climate reconstructions. What’s dishonest on their part is claiming that since, as you point out, such a sparce reconstruction fails to be statistically meaningful, that this makes your reconstruction invalid. It is, in fact, just as meaningful as theirs and in a reducto ad absurdum sort of way disproves their own reconstruction. In a sense your claim that you haven’t done a reconstruction is what allows them to obsfucate their own method’s lack of validity. OTOH, this is all water over the dam now. The cat’s out of the bag, the fat lady’s sung and all that sort of thing. You are the only ones left standing on the field of battle when it comes to MBH98/99 style climate proxy reconstructions. That’s why they’re desperate to “move on”. The trouble is that there aren’t that many new proxie series to add to the pile so that it’s not that easy to hide behind their new bulworks. You’ve ranged them and now it’s just a matter of deciding where the target rich areas are.

If there’s a prize for most mixed metaphors in one paragraph then this gets my vote…

I’m glad to see your point 5 as I was going to post the same thing if you hadn’t. This is what people like John H and others like him, who actually have done scientific work, are missing. The multi-proxy studies, unlike much other scientific work, is essentially historical rather than practical. Let me give an example of both sorts. Let’s switch to astrophysics. If a scientist were to do a practical study of supernovas he’d collect data on a large number of them and then he’d divide them into different sorts depending on objective characteristics; maximum absolute brightness, brightness decay rate, various spectral features, etc. Then he’d publish, declaring how his new taxonomy of supernovas is superior to past ones. In this case auditing his work would involve getting the raw data he worked from and any methods he used to calculate the listed characteristics. A replicator would instead gather data on a different set of supernovas and apply his methods to this data and see if the same results were found. Having the original data would be nice though as it’d let the replicator see if he was doing things right, important if he found different results.

The historical sort of science is interested in finding new characteristics from/of an existing historical database. To be sure the researcher may add to this database in the process but that’s almost incidental. The important things are the new characteristics found. To audit this sort of science is essentially to replicate it as well, as once you’ve verified that the methods used when applied to the data do indeed produce the characteristics claimed you’ve done everything possible and only can discuss what the characteristics mean rather than that they exist.

Switching back to climate proxy reconstruction, we have existing data; tree rings, ice cores, thermometer readings, etc. An article may introduce new data to the field, in which case it should be, (but often apparently isn’t), archived. The results in this case is the production of a set of graphs which reproduce one or more climate characteristics; temperature, rainfall, sea level, ice coverage, etc. Auditing in this case is checking out the data for potential pitfalls and checking to see that the methods used indeed give the results claimed. But there’s not much difference between such auditing and replication itself. What would you do in producing a new replication except pick a different grouping from the existing proxies?

In this sense, Mann and his cohorts are correct in claiming that you (M&M) have produced your own climate reconstructions. What’s dishonest on their part is claiming that since, as you point out, such a sparce reconstruction fails to be statistically meaningful, that this makes your reconstruction invalid. It is, in fact, just as meaningful as theirs and in a reducto ad absurdum sort of way disproves their own reconstruction. In a sense your claim that you haven’t done a reconstruction is what allows them to obsfucate their own method’s lack of validity. OTOH, this is all water over the dam now. The cat’s out of the bag, the fat lady’s sung and all that sort of thing. You are the only ones left standing on the field of battle when it comes to MBH98/99 style climate proxy reconstructions. That’s why they’re desperate to “move on”. The trouble is that there aren’t that many new proxie series to add to the pile so that it’s not that easy to hide behind their new bulworks. You’ve ranged them and now it’s just a matter of deciding where the target rich areas are.

John A (#40): The mental leap from “science must be replicable” to “every scientist must be prepared to provide all his/her data and workings for checking by anyone (whatever his/her qualifications or competency), at absolutely no recompense for the time and effort spent in so doing” never fails to amaze me. It seems to have caught on here as a contrarian ploy, though.

It’s interesting what you regard as a “contrarian ploy”. I’m asking for access to the data and models you used to reconstruct sea-level rise from the “isle of the Dead” sea level marker. I’m not asking because I have a political agenda or because I particuarly dislike you. I’m asking that you simply allow access to data that you have already archived. Straightforward n’est-ce pas?

Tell me, John A, where is there a profession, anywhere in the world, in which the practitioners are expected to provide intimate details of their work to anyone who cares to ask, no matter how unqualified or incompetent that person may be, and at no recompense?

Well, you’ve pretty much described most democracies. I don’t know that I’m asking for “intimate details” as if I were enquiring about your sex life, but the normal process of data archiving that you have already told us six months ago that you were carful to do.

So I am being asked to “list, with citations to publicly accessible archives, the data used, and the full description of your methodology together with the data files that reproduce your results” to someone who (a) will not reveal his qualifications, (b) will not reveal his identity, and (c) has shown himself to be eminently capable of doing a hatchet job on any scientific work he doesn’t happen to like.

I will reveal my identity when I get academic tenure and/or security of employment. Until then, for my family’s sake, my identity will remain private. If I’ve shown myself to be eminently capable of doing a hatchet job on any scientific work I don’t appear to like, then perhaps those scientific works are not as robust to examination by even people lke me. Besides I never realised that I had to like a piece of work in advance in order to examine it. I’m pretty sure that you weren’t madly in love with John Daly’s work on the isle of the Dead when you examined it, were you?

You finish with the wonderful “if your study holds up, you’ll be vindicated and I’ll be humiliated”. Pray, tell me why you think the blessing of the anonymous and (at least until you provide evidence to the contrary) unqualified and incompetent “John A” is better than the support of a number of expert sea-level scientists who have already reviewed our work, at several different stages?

Well, hopefully the same was also true of Hwang Woo Suk.

You appear to think that “replication” just involves checking that you “understand that the data (I) used is correctly cited and (my) methodology is transparently correct and replicable”, which is a bit like thinking you can replicate a chemist’s experiment by just checking his calculations.

Not really. I don’t expect handholding, but even so this is a historical reconstruction here, not an attempt to repeat a genetics experiment.

So, John A “¢’‚¬? here is what you need to do to replicate our work:

(a) Read our papers “¢’‚¬? presumably you already done this “¢’‚¬? they give you most of the information you need.

(b) Get hold of Thomas Lempriere’s original data “¢’‚¬? we give the reference to this in our first paper.

(c) Digitise (b), correct or remove obviously erroneous data, and obtain the mean tidal level for 1841 and 1842 “¢’‚¬? compare this with the values given in our first paper.

(d) Install a tide gauge at Port Arthur and collect data for two or three years.

(e) Connect the levels of your tide gauge and the tidal benchmark on the Isle of the Dead using one or more surveying technique(s).

(f) Make due allowances for Glacial Isostatic Adjustment using an appropriate model.

(g) Do an comprehensive error budget of the data obtained from (a)-(f).

(h) Estimate the change of sea level at Port Arthur from 1841/1842 to the present “¢’‚¬? compare this with the results given in our second paper.

( I can’t be much more transparent than that can I?)

Now that would be REPLICATION “¢’‚¬? not grandstanding.

Actually I can do better than that. I can actually add value to the reconstruction. Bet you’re curious how, aren’t you?

Me singing the praises of the IPCC? How could anyone praise an organisation which hijacks the credibility of a bunch of scientists to make political points?

I’m sure the IPCC has plenty of good scientists working for it but then the damn politicians step in and make an edit here, an edit there. Before you know it the report has been “adjusted” to be useless.

It’s not the nature of disagreements that we have to have a tit-for-tat conversation. What SHOULD happen (as happened, for example, with our discussion of how much CO2 contributes to the “greenhouse effect”) is that if I say something wrong you point out what it is and why, and if I am indeed wrong I will admit it and change what I am saying. I have already adjusted my position before once you pointed some actual scientific results out to me which I didn’t know about. However, all too often the response to a legitimate challenge is some whinging about not having to make data available or “consensus” or some other bogus concept which is not the actual nuts and bolts of what we are trying to discuss.

Can’t you see that if one of us says “the results of this study are questionable, I think it should be audited” and the response is “why should I give you my data if you’re just going to show what I’ve done wrong?” that tells us a lot about the confidence of that scientist’s conclusions. How dishonest is it, to hope that any flaws in your study are never revealed, in order to keep one’s reputation intact? It may be good for your career but it’s hardly good for mankind. The “irony” of the situation is that it’s the AGW-doomsday-people who claim to have the best interests of mankind in mind, yet they are often the ones who would withhold potentially useful knowledge from the rest of us for selfish reasons.

the real nail in the coffin was that when people looked at the paper, obtained the methods used and tried to replicate the experiment in other labs they failed.

I am bemused. What SM has tried to do is to repeat the exact experiment reported by MBH. Likewise for John A and John Hunter’s data set. It is also worth pointing out that there is a long tradition of people falsifying results, and direct data audits show this.

Lets say I reported arithmetic mean of 500g, you get my data, look at it and say a median is more appropriate and that is 600g, someone else says that no, arithmetic mean is correct, but that due to my rounding errors the actual value is 522g. Where does that get us.

I have always considered the analysis of data to be important. It is obvious, even from the examples you suggest, that these discrepancies could make the difference between a result being statistically significant; or not.
cheers
per