I know that many of you want to hear our take on Mann (and on ourselves). I’ll get to that. I want to set down some notes on the other speakers while my notes are relatively fresh in my mind; my notes on the morning session are not great, but I’ll give a gist of what happened with Pollack and Schrag.
Both confirmed that they could not estimate temperature 1000 years ago within a half degree. Schrag emphasized that that using proxies to estimate average temperature is a “very difficult statistical problem” and policymakers are “asking more than the community can provide”. That might well be the take-home message from the panel.
Pollack
Pollack presented first. He didn’t leave a handout for the audience. Pollack does boreholes. I missed the first part of his presentation as Ross and I were re-editing our PPT. (Both of our nerves were frayed by this point in getting ready and Ross had the patience of Job.) We didn’t miss too much and we came in when he was showing a graphic with borehole estimates not back as far as MWP. Pollack emphasized that these were “very noisy” records over and over and that it was hard to extract a signal. I believe that he said that they cut it off before the MWP because of noise problems. Christy asked about groundwater bias. Pollack: “obvious perturbations” e.g. fractures were removed. (It would be interesting to ask mining people about this, about micro-fractures).
To show the linkage between ground and surface temperature (“coupling”), Pollack showed a graphic showing seasonal relation of ground temperature to surface temperature at Fargo ND — only to 4 m. It would be interesting to see some data a little deeper.
Roberts asked about effect of deforestation and land use. Pollack said that the boreholes were mostly rural. Since the boreholes are virtually all opportunistic holes from mineral exploration, I agree that their locations are non-urban, but it was ironic that one of his examples was labelled from Val d’Or, which is a mining town in northern Quebec with bars and even hockey teams. (The owner of their Junior A team is a friend of mine.)
In passing, it would be interesting to get an opinion from a mining person as to whether any biases could be introduced by using opportunistic holes. For example, holes drilled in mineral exploration are hopefully not a random sample of the earth’s surface (although when you’re getting results from a failed drill program, it sometimes seems like it.) For example, gold exploration virtually always takes place (or should take place) in “shear zones” — is any bias introduced by that? If you have to allow for fractures, then you should presumably have to allow for not just big fractures, but little fractures and micro-fractures. You’d want to get a specialist opinion from a mining geologist on that, or an opinion that the fractures don’t matter, in which case, why adjust for the big fractures?
The discussion of ground water worries me. I didn’t quite focus on how ground water mattered or how they dealt with it, but there’s ground water everywhere in hard-rock mines. Pumping water from underground is a major operating issue at underground hard-rock mines. If you leave a mine in northern Canada unattended, it will fairly quickly fill up with water (not in the sense that people will drown if the pumps fail, but in a period of months rather than centuries.) Many of the early steam engines were invented by mining engineers for pumping mines — I think that James Watt had mining customers in Cornwall. I don’t know whether coal mines have similar issues with ground water; the geology is different. Anyway, if ground water is a problem for boreholes, I can’t see where the problem would begin and end for boreholes from hardrock mineral exploration. Anyway, that’s just musing.
To show borehole results prior to the Little Ice Age, Pollack showed either the following graphic from Dahl-Jensen et al [Science 1998] (or a similar one for a shorter period), apologizing for the muddiness.
Ironically, the same article had a perfectly clean version showing a pronounced MWP, which we subsequently showed the Committee as a 5-second comment on boreholes.
I think that Cuffey asked Pollack if he could estimate climate 1000 years ago to within 0.5 deg C. Pollack answered: No.
Schrag
Schrag had been advertised to us as being very outspoken in his positions, but didn’t make very strong claims about proxies, the opposite in fact. He was there presumably as a coral specialist, but hardly said anything at all about corals, except that they did not provide a long enough record to be relevant to millennial issues.
He said that the big issue is whether we can detect AGW. He showed a graph of temperature changes through geological time including Eocene-Cretaceous (which was much, much warmer). The graph was not dissimilar to one that I presented last spring. He said that the “scale of natural variability far surpasses anything that we’ve seen in recent times”. The corollary that he pointed out was that, because the range in “recent times” was so limited (and here he might even have been thinking on a millennial scale, I’m not sure), it was unfair to have high expectations from modellers.
As to whether the present warming was outside the range of natural variability, he primarily (entirely) relied on Lonnie Thompson’s tropical glaciers as proof that it was. He showed pictures of receding Quelccaya glacier, saying that this hasn’t happened in “1500 years” and is “unprecedented”. He went on to say that similar recession is happening in high-altitude equatorial glaciers and is a “global phenomenon”.
He said that estimation of changing global temperatures should use “stable points” of the earth’s climate (e.g. the western Pacific warm pool). He said that records in the warm pool don’t go back 100 years and therefore a multiproxy approach was needed. He argued that high-altitude tropical glaciers were another “stable point”.
Christy said that he thought that Quelccaya was only 1500 years old. Schrag said that that was merely the length of time that a seasonal signal could be distinguished and that it was older. He mentioned that radiocarbon-dead material had been found underneath the receding glacier and I think that he said that some organics dated 5400 BP had exposed by receding material. [As an editorial comment, we’ve had some discussion on Quelccaya here in the past and Hans Erren has an interesting graphic on Quelccaya accumulation, that was one of the very first things that I was involved with. The seasonal signal is distinguished long before 1500 years. I wonder whether people like Schrag issue errata to panels or not. I don’t see that the presence of radiocarbon-dead material underneath means anything I can think of lot’s of ways for that to happen. I don’t know what has been published on the Quelccaya organics — I’ll take a look when I get a chance. I’ve commented here on organics in Kilimanjaro where I find the evidence very sketchy and not very conducive to accurate dating.]
Schrag pointed out that it would be “nice if the proxies were thermometers” but they aren’t. He said that “many details are swept under the carpet” in making an average [SM – guess what – I agreee with that], but “you have to realize how difficult” it is to make an estimate of average temperature, pointing out that there were disputes even in the instrumental period. He said that it is a “very difficult statistical problem” and policymakers are “asking more than the community can provide”. [Again some SM editorializing: in my opinion, this is a pretty important comment and one that is very germane to the panel’s deliberations.]
On corals, Schrag said that there were no 1000 year corals as they grow fast and die young; and there are organisms that attack them after they die.
Christy asked if there are sediments that preserve centennial information. Schrag” you would need 10 cm/1000 years. They exist but not many places. Suggested that Lloyd Keigwin had located good places in North Atlantic. [SM editorial note: Keigwin’s Sargasso Sea sediments have a pronounced MWP, but not the one near Nova Scotia.]
He was asked about survival of corals in the warm Eocene. He said that corals would not go extinct in a warmer world, but suggested that they would survive as individuals and not form attractive reefs.
Cuffey: do we know the temperature 1000 years ago to within 0.5 degree, yes or no? Schrag: No.
Climate Audit 
82 Comments
The coral expert declaiming on high altitude glaciers. More tea, Alice?
=============================================================
Cobblers. The effect of warming (if warming is deleterious to corals) would be to have them migrate polewards. If corals are meant to be so sensitive to warming then the Eocene should have killed them off.
What about the coldwater corals? Much more slower growing etc.
They should have had Keigwin in front of the Panel as someone who knows how to properly do a single proxy reconstruction of past climate, and avoid experimenter bias in doing so.
I really hope those BBC bods were taping this.
Steve – Watt’s engines were used to pump water from mines in Britain including coal mines if that’s helpful.
I thought it was Cornish tin mines ?
Re #5 OT but I think you might be thinking of Trevithick? http://en.wikipedia.org/wiki/Trevithick
RE the bore-holes. Heat progates from the surface to the underground via diffusion. It’s hard to understand how they can extract much information from the temperature profile since the diffusion depends on the detailed properties of thermal diffusivity of the earth. Water has a different thermal diffusivity than rock, fractured rock is going to be different still,… Plus heat comes up from the interior of the earth as anyone who has gone down in a deep mine knows. That’s bound to mess up the calculation.
The speed of thermal diffusion is very slow and the effect of a temperature change at the surface is attenuated exponentially with depth. That’s why people had root cellars before the days of refrigeration. A cellar, even 2m down, has a nearly constant temperature all year round.
This sounds like tree rings as proxies all over again, lots of variables affect the measured temperature distribution with no control or understanding of them.
Re #7
I’ve always wondered whether its possible to do a thermal profile of an ice mass like in the interior of Antarctica. In that particular case, you can guarantee the homogeneity of the “rock” you were surveying.
Re #6, Peter, no, Trevithick was later. As your link says, he would have grown up with Watt’s steam engines pumping mines around him.
How about a borehole near where the nuke wastes are being intombed? No ground water there for as long back as we’d be interested in!
#7. You don’t have to go all that deep before the heat of the earth becomes an issue. Deep (2K) mines are HOT. I think that for some mines, air conditioning is the biggest energy cost. I can’t imagine that thermal heat flux doesn’t become an issue in these calculations. I’m not trying to de-construct these calculations right now – I’m merely observing that (IMO) no rational panel would have left Pollack’s talk thinking that boreholes were a magic bullet.
Schrag’s comments seem to be on reef corals. There are species of deep-sea coral (which can dwell anywhere from about 50 to 5000 m below sea level) that live for well over 1000 years. This seems to have been first pointed out for paleoclimatology by
Also, even when corals are fossilized, they are still useful, because they can often be dated by other means (Th/U or C). This can extend pretty far back, e.g.
In principle, it ought to be possible to construct detailed chronologies using overlap techniques similar to those used to construct tree-ring chronologies, but this does not seem to have been done.
If anyone is interested, a nice online review, albeit a bit dated, is the following.
A potentially-important recent advance is the use of SIMS (secondary-ion mass spectrometry) on gorgonian corals—which live at all depths, from the poles to the equator. See e.g.
A practical issue is that SIMS is very slow and expensive; moreover, there are not many good SIMS instruments in the world (and they tend to be booked).
As to how accurate coral-based paleothermometers are, scepticism is obviously in order. Corals are biological: their metabolisms are affected non-linearly (even non-monotonically) by many factors, which are in turn sometimes local to the microenvironment.
Information on how borehole temperature profiles are related to historical surface temps:
Basic methodology from a the Environmental Sciences Resource Center in Antigonish, NS, Canada. I don’t know for sure but they probably have a hockey team. One with pucks, skates and such.
Optimal surface temperature reconstructions using terrestrial borehole data authored by Mann, Rutherford, Bradley, Hughes and Keimig. Journal of Geophysical Research from April 2003.
All this because:
Re: 12, One major problem with SIMS is that it gives too much data. It is so sensituve that there is always the issue of separating what was in the sample from what contaminated the sample on the way to the lab.
I am uncertain how you could sample coral under cleanroom conditions in order to reduce sample contamination. Using SIMS on samples from nature would mean very large magnitude error bars.
Re #14. Clean-room conditions are certainly helpful. Your comment makes little sense though: it matters zero whether or not the sample is from nature. Did you read the reference cited?
Oh dear ….. John A said (#8):
> I’ve always wondered whether its possible to do a thermal profile
> of an ice mass like in the interior of Antarctica. In that
> particular case, you can guarantee the homogeneity of the
> “rock” you were surveying.
What a wonderful and novel idea. You should tell the glaciologists — they are really really dumb and have never even contemplated measuring the “thermal profile of an ice mass like in the interior of Antarctica”.
John, if you have really “always wondered whether …..” it just might be a good idea to start reading the scientific literature …..
Re #8,
John, it is actually done, besides isotope ratio changes, see:
http://waiscores.dri.edu/Amsci/taylorintro.html
and http://polarmet.mps.ohio-state.edu/Icecore/fjlRussia.html
Re #8. Ice core bore holes are used to estimate past temperatures and are an important constraint on temperatures during the last ice age. Because the high snow accumulation (~25 cm / yr in places) helps to trap and extend thermal anomalies, it is possible to reconstruct much greater time periods in ice than in rock.
In fact, the Dahl-Jensen plot shown above is one such reconstruction from an ice borehole. They assert the following about local temperature changes at the GRIP bore hole site in central Greenland:
They also reported similar fluctations with ~1.5 times the amplitude from the Dye-3 ice core site nearer the edge of Greenland.
Unfortunately, borehole thermometry in ice is a sensitive measurement that neccesarily has to be made in harsh conditions at least several years after all drilling has stopped, and hence is rarely every done, so only a few such reconstructions exist.
Re #16
With respect, but my answer would be “Yes John A they have computed a thermal profile of the Antarctic ice sheet and here are some papers on it, etc, with links”.
Comments #17 & #18 are good examples of polite and helpful answers to John A’s question.
Re 19:
Sorry, Louis — I just get very frustrated by the amount of discussion on this blog which is based on virtually no knowledge of the actual state of the science.
I would probably have been more “polite and helpful” if only this blog was a vehicle of open inquiry rather than being mainly a criticism of anything that appears to support the idea of anthropogenic global warming.
RE 20:
The problem, as I’ve come to see it, is that almost everything that puts the “A’ in GW seems tainted. The bulk of it is tainted by bad proxies (and the corresponding statistical analysis) and political agenda. And even with the other science, there doesn’t seem to be enough data to be able to know one way or another. Isn’t this what is debated endlessly on blogs and journals and labs? Until we have conclusive data nothing is settled.
Not only that, but it’s been very difficult to reconcile the idea that the earth has had climate variability over its entire history with the idea that 100 years of human activity (a mere moment of the earth’s history) can act as such a lever as to move all of “climate” off it’s moorings. The thing that is very difficult to reconcile is the idea that a warmer earth is a bad thing. The historical evidence seems to suggest that a relatively warmer earth would generally be a good thing (yes, I’m aware of the theoretically potential threat that the ice might all melt and cause the sea level to rise–and that could be a bad thing).
Re 20: John Hunter, you say “I would probably have been more “polite and helpful” if only …”
Curiously, near as I can tell the “if only” never seems to happen in your life. Here’s a novel idea. Why not just be polite and helpful no matter what the other guy does?
Because I’ll guarantee you that posts like #20 just make people either laugh at you or curse you, and I don’t think that’s quite the effect you’re looking for.
w.
Re 22, I listed John Hunter’s post as #20. In fact it was #16. Mea culpa.
w.
How would we tell? And besides, this blog is about bad statistics, hiding data and methodology from scrutiny, scientific misconduct in climate science – all subjects which should interest a professional scientist like yourself.
Speaking of which, can you please point us all to the data and full methodology of “Pugh, Hunter et al” on the subject of global sea level rise and the Isle of the Dead, for the purposes of audit and replication, or is this precious information still embargoed lest it fall into the hands of terrorists?
Thanks to the others for the icecore temperature links, which were most helpful.
I don’t see a whole lot of “science” in some of the areas being discussed here. I see a bunch of interacting authors publishing papers that are peer reviewed by each other–and then claiming scientific consensus. Moreover, I reserve the right to ask questions or make comments on issues that I don’t fully understand. That’s how I learn. We can’t all be as knowledgable as you.
I get frustrated with the amount that is published in “peer revewed” climate science journals which is based on virtually no knowledge of the actual state of statistical knowledge. Ammann and Wahl 2006 is proof positive.
Re #26
So isn’t it time you stopped faffing around with climate journals and started submitting to whatever journals the real statisticians read ?
Steve,
Have you considered getting a group of high-powered statisticians and mathematicians together to make a statement about the statistics used (or abused) in climate science?
Re: #28
Two or three statisticians, a couple of paleoclimatologists, a math geek with expertise in PCA and RegEM, and a scientific ethicist.
A high powered statistics and mathematics team will need a name….for the T-shirts, obviously.
#27 – I haven’t submitted anything for a while. I’ll need to turn off the blog for a while, but there always seems to be something to do. On balance, I think that it’s more important to stay on top of things like the NAS panel as opposed to working up new articles.
#28 – I’ve spent enough of my life organizing things. I don’t want to organize for a while. I’ll cooperate with others, but don’t want to do it myself. Besides I don’t know the people anyway – I’ve been in business.
Just for humour, it is probably appropriate to mention that the abuse, and sometimes torture, of statistics is not confined to climate science.
There is a greek statistician who spends his time showing that most epidemiology is wrong (does not reproduce; conflicts with subsequent studies).
There was a recent paper showing that (if I recall correctly) ~10% of papers published in the BMJ had statistical errors in them; and remember, each BMJ paper gets a separate statistical review.
You can look at numberwatch (www.numberwatch.co.uk) for a fairly interesting coverage of maths as applied to science.
cheers
per
John A (#24):
You ask “can you please point us all to the data and full methodology of ‘Pugh, Hunter et al’ on the subject of global sea level rise and the Isle of the Dead, for the purposes of audit and replication”
John, this stuff has been done to death on climateaudit. I’ve discussed my views of “audit and replication” elsewhere (e.g. #72, “IPCC 1[1990] – Comment #1”, 24 June 2005).
If you really want the data from our study, can add value to it, are competent to analyse it and are not going to just waste my time, then I will gladly collaborate with you. However, much of the data is already in publicly-available archives (e.g. the normal repository for Australian sea level data, which is the National Tidal Centre; sources of other data are as indicated in our publications). Also, if you want (and are able) to do further laborious searches of archives, or repeat some of our sea level and surveying observations, then I am only too glad to help. But if all you want to do is sit in front of your computer and throw mud, then forget it.
You also ask “is this precious information still embargoed lest it fall into the hands of terrorists?”.
I would ask the same concerning your “precious” identity ……
Re: #32
John,
One has to be “competent” to analyze your data before they allowed to have it for replication purposes? Competent by who’s standards? Yours I suppose. I don’t remember this being a part of the scientific process when I was in school. I’m completely flabergasted that you would make such a statement. This really makes be wonder how good your analysis is if it can’t stand up to the scrutiny of other scientists. Unbelivable.
Re: #32
One other thing: What does John A’s identity have to do with your unwillingness to submit your paper to replication? That’s a red herring if ever there was one.
Re #32, John Hunter, you should be ashamed for calling yourself a “scientist”. You say:
This is a joke. Science is about transparency. It is not about only letting your data be seen by those who in the infallible judgement of John Hunter are “competent” to look at it.
He is not asking you to “collaborate” with him, John, because to do that, a person would have to be a serious masochist with a strong stomach, a lack of integrity, and an immunity to insult. He simply wants to know if your results are replicable, which is the core of science.
You do remember “science”, don’t you, John, that thing where one guy does the work, then lets everyone else see if they can either replicate it or show it is incorrect? But then of course, you don’t want anyone looking to see if your work contains gaping holes …
w.
The NAS panel seems to have caued a churning of denizens of Middleearth – both Hunter and Lambert at the same time. We are twice blessed.
So Steve,
Which one of them is the Ballhog? [sic]
Wave’s bag of garlic frantically around while searching for pine stakes!
Re #35, Wills, if “Science is about transparency” then when isn’t it transparent who the number two here is?
Also, if a twelve year old child logs on to CA as, I don’t know, ‘Bill’ and asks a question like, ‘I wnt yr numbers I wnt 2 replic8 them’ you think John Hunter should be preparred to waste his time with him or her? And with every such request? Of course not. So, what are your criteria for a scientist to comply with such a demand for his or her data?
John Hunter:
No. I don’t want to throw mud, I want to understand that the data you used is correctly cited and your methodology is transparently correct and replicable. The added value is that your study is audited and replicable by skeptics – how precious is that? What I will do is publish in steps my efforts to replicate your results.
If the data and methodology stands up, then I will say so on this blog for everyone to read. I do not make such a promise lightly but knowing full well Steve will hold me to my promise.
So please list, with citations to publicly accessible archives, the data used, and the full description of your methodology together with the data files that reproduce your results. If the data are not currently accessible for some reason then give those reasons.
If you would rather not post these things to the blog (which I can understand), then send them to climateaudit AT gmail.com and we’ll correspond that way. It’s a win-win. If your study holds up, you’ll be vindicated and I’ll be humiliated.
You can’t say fairer than that, can you?
I don’t know anything about John Hunter’s paper. But when I see this:
that must mean that some of the data is not !
The crux of science is its reproducibility. All sorts of people want to publish information, and have the kudos of a peer-reviewed paper, but don’t want to publish the structure of their chemical/ the gene sequence/ the protein structure/ etc. This has to be stamped out, because it is antithetical to science.
If John H won’t publish his full data, and demands to be named as a collaborator for passing out information that he has already published on, that makes a fairly clear statement about the sort of scientist he is. If he is refusing when someone only wants to replicate his study; well what can you say ?
yours
per
You’ll need Bristlecone stakes, Louis.
======================================
Re #42
Kim,
No, just realised the pine in match boxes are over engineered for the purpose. Come to think, tooth picks might be too radical too.
Dang,
Louis
John A (#40): The mental leap from “science must be replicable” to “every scientist must be prepared to provide all his/her data and workings for checking by anyone (whatever his/her qualifications or competency), at absolutely no recompense for the time and effort spent in so doing” never fails to amaze me. It seems to have caught on here as a contrarian ploy, though.
Tell me, John A, where is there a profession, anywhere in the world, in which the practitioners are expected to provide intimate details of their work to anyone who cares to ask, no matter how unqualified or incompetent that person may be, and at no recompense?
So I am being asked to “list, with citations to publicly accessible archives, the data used, and the full description of your methodology together with the data files that reproduce your results” to someone who (a) will not reveal his qualifications, (b) will not reveal his identity, and (c) has shown himself to be eminently capable of doing a hatchet job on any scientific work he doesn’t happen to like.
You finish with the wonderful “if your study holds up, you’ll be vindicated and I’ll be humiliated”. Pray, tell me why you think the blessing of the anonymous and (at least until you provide evidence to the contrary) unqualified and incompetent “John A” is better than the support of a number of expert sea-level scientists who have already reviewed our work, at several different stages?
You appear to think that “replication” just involves checking that you “understand that the data (I) used is correctly cited and (my) methodology is transparently correct and replicable”, which is a bit like thinking you can replicate a chemist’s experiment by just checking his calculations.
So, John A — here is what you need to do to replicate our work:
(a) Read our papers — presumably you already done this — they give you most of the information you need.
(b) Get hold of Thomas Lempriere’s original data — we give the reference to this in our first paper.
(c) Digitise (b), correct or remove obviously erroneous data, and obtain the mean tidal level for 1841 and 1842 — compare this with the values given in our first paper.
(d) Install a tide gauge at Port Arthur and collect data for two or three years.
(e) Connect the levels of your tide gauge and the tidal benchmark on the Isle of the Dead using one or more surveying technique(s).
(f) Make due allowances for Glacial Isostatic Adjustment using an appropriate model.
(g) Do an comprehensive error budget of the data obtained from (a)-(f).
(h) Estimate the change of sea level at Port Arthur from 1841/1842 to the present — compare this with the results given in our second paper.
( I can’t be much more transparent than that can I?)
Now that would be REPLICATION — not grandstanding.
Re #41. Where can I see all of your data per? Will you please spoon feed (web addresses, emails, copious amounts of your time), your data so I can replicate it? Actually I demsnd you do that! Additionally, I have to tell you at some point I will start a website and while I wont accuse you of much, I will let correspondents accuse you of f****, I assume you’ll still cooperate?
best
Peter
Re #44. Cracking post John 🙂
re #44
Easy John,
You are a public servant.
John H., the devil is in the (g) tales.
=======================================
Hunter: you use the same “logic” that my kids did when I caught them doing something wrong. I am not qualified (at least now) to replicate your work. However, what harm is there in archiving all your data and methods? If as stated in 47, you are a public servant, you have an obligation to archive the data. I’ll bet it is already archived somewhere (I hope you keep records!), and all you have to do is provide a link. How hard is that? Like so many of your ilk, you obviously have something to hide. There is no excuse for not giving John A., or even me, access to your data.
It seems clear that the data/methods John H used have been largely archived. When John A begins his audit of John H’s work if/when he finds missing data or methodology I am sure John H will be happy to provide him with it so he can complete his anonymous audit. Except that John A isn’t going to do an audit is he. As so often there is plenty of talk, but little substance behind it. Perhaps the anonymous John A is a politician?
jae (#49): It is clear that you are “not qualified to replicate (my) work” — so I’m not sure why you (like so many others on here) think you are qualified to contribute to a discussion on replication and archiving. You certainly don’t understand what is a reasonable way for a scientist to work.
You say “I’ll bet it is already archived somewhere (I hope you keep records!), and all you have to do is provide a link”. If you put all the files on your computer on the Internet, do you think anyone else would understand them, even if you had kept reasonable notes in each folder? I keep pretty scrupulous notes, but even if I come back and look at work that is several years old, I still find it hard to put everything back together — I can, but it takes time. Which leads me to a few points which I hope may sink in:
1. I do keep comprehensive notes so that, if I have to, I can recontruct what I had done. But that does take significant time and effort.
2. My data and notes would not make much sense to anyone else — unless they spent a lot of time sorting out what I had done. The published papers are effectively a shorthand version of my notes, in a form that is clear and understandable (I hope!) to other scientists.
3. I can see no justification for anyone to think that they have an automatic right to my data and notes and for me to spend my own time and effort interpreting them for other people — for no proven benefit to the science.
Re. #47 and #49: Louis is wrong: I am not a public servant. jae is wrong: I do not have an “obligation to archive the data”. However, I do take reasonable steps to assure that my work is traceable and that all original data is recoverable.
Hey, Hunter: Your scientific education seems to be lacking something: the foundation of science is replication. How is anyone going to replicate your work, if you don’t provide the data? How can I believe what you have published? Take your word for it? What if you inadvertently made a big mistake? You don’t have to archive every note, just the data relevant to the study. I doubt that this is that hard or complicated to do. After all, some climatologists do archive their data. Maybe they are more organized, or something. Those who refuse to archive their data are looking pretty bad to me…
jae (#53): You ask: “How is anyone going to replicate your work, if you don’t provide the data?”. I thought I made that clear in posting #44 — if you want to REPLICATE my work you have to collect the data again, I’m afraid. That’s what chemists do when they replicate an experiment.
But you just don’t seem to understand. I’ve given JA quite detailed instructions on how to replicate my work (#44). If he wants to be lazy, and NOT collect new data, then I’ve told him where our original data is. What else do you want — your nappies changed for you?
Re #51 and #53: John, I hear and (I think) understand your pleas. However, there is something I think that you are missing. You (and other climate scientists) are publishing papers that have been very influential in IPCC and other forums (fora?) where policy is being decided. In effect you are making substantive public statements. It is surely not surprising that people seek some confirmation that the statements are sound, and so they come under close scrutiny.
Leave science for a moment. In the commercial world, you would be expected to ensure that your substantive public statements are “true, plain and fair”. You would deal with this by demonstrating that there are processes in place to verify that the statements that you make are in fact “true, plain and fair” and are backed up by readily accessible data archives that can be produced as a defence, should (for example) court action ensue for some reason.
Turning to science. As is widely publicised, replication is at the core of scientific method. I accept that in the commercial world a proponent may commission an independent review, and publish that independent review in support of his methods rather than necessarily publish all of the detail (for example, where proprietary IP is an issue). And that approach may be acceptable in science as well. However, as I understand good scientific practice, the obligation on the scientist publishing is that he must make available his data and methods to permit replication. As one quote (from http://www.psy.plym.ac.uk/year1/scimeth) says “in my book it isn’t science without public disclosure of methods and results.”
It does seem to me that you have a choice. If you want to be taken seriously, and your pronouncements accepted as good science, you must conform to good scientific practice. If not…..
mtb (#55). Thank you for saying “If you want to be taken seriously, and your pronouncements accepted as good science, you must conform to good scientific practice.”
I think I do.
Re:#51
Come now, John, no one is asking for any significant time/effort on your part. You ask
Yes, I do. That’s all most of us (I think) are asking for. You’re not required to spoon-feed anyone, just supply the data/code used to generate your paper. Any “replicator” should be prepared to spend a fair amount of his/her own time fitting it all together and working things out. The various email/phone requests Steve has recounted here on the website all seem quite reasonable and should require no significant time/effort in response. If hundreds of biologists (just speaking from personal experience here) can do it, why not you?
You note:
Again, no one is asking you to spend any significant time/effort to interpret your data/notes. Of course, by having published a paper, you entered into an implicit contract to supply all relevant data/notes to anyone who asks (which is easy to cheaply fulfill by setting up a website containing the various files, as discussed above), so that seems justification enough for the first part of your statement.
The well-proven benefit for science is, of course, that the free access to data/materials/techniques helps speed up the adoption of robust/interesting/generally useful results, along with the abandonment of incorrect or non-robust results. Imagine if PCR had remained a closely-held secret, or even a technique forbidden to be revealed to non-Ph.D. holders! Since, as shown above, there is no significant cost to the scientist by doing so, what possible harm could come from making the data/code/methods used in published research fully available on the internet?
Re #54, John H.
You don’t seem to get the point. You’ve done no scientific work, zero, without allowing for replication. Replicating your work requires the use of your data. If your methods look good and crunching your data pans out then someone might go the final mile and gather new data and do a full replication. What sort of idiot would go to all that trouble if your own data doesn’t give the results you say it does. It is the duty of any true scientist to make replication as easy as possible. Indeed to invite it.
Please tell me you aren’t employed in science.
I have to disagree with a lot of the posters on here. Science does not work by one person collecting data and then 20 others reanalysing that data. That is not replication. For example. If I go out and catch 1000 cod in the N. Atlantic and weigh them and find the mean weight is 500g. I can then publish a paper saying that. If you come along, demand my methods, my data and re-calculate the mean and get 500g that is not a replicated study. That is just confirming that I can work a balence. It adds nothting to the science and nothing to the ‘bank’ of knowledge. You need to take my methods, maybe modify them a bit, go collect some of your own fish and find your own data. THAT is replication.
Re: #59, what you say is true only if there are no errors in your work.
If there are errors, and we recalculate and get a mean weight of 300 grams, then your study has not been replicated …
Which does add to the science and to the ‘bank’ of knowledge.
Once that’s done, then we can do as you say, try different methods, etc. But first we have to be able to get your answers to your questions from your data, that is to say, we have to replicate your study.
w.
Re #59, Paul Gosling
But if I get 600g, it shows that you can’t handle basic statistics. That does add to science by removing erroneous information. (Which seems the appropriate comparison for this site.)
The crux of the matter is that there’s only one accepted way to find the mean weight of 1000 cod. However, there are myriad ways to attempt to pull a signal out of a set of proxies.
If your claim was that you had found a particularly good way to find the mean weight of 1000 cod, it would be very hard to tell how valid your method is without a proper description (i.e. source code) for it. For all we know, your method might work great with your 1000 cod but not with a different 1000 cod. If we try your method on our 1000 cod and find it doesn’t work, it helps to know the weights of each of your 1000 cod to see what the reason for this is (whether it’s your mistake, or ours, or a peculiarity of your method, etc.).
In short, if a large part of the work in your study is statistical, it’s very hard to examine and test your results without the data and methods used to produce those statistics. I think most scientists would agree that the more details of your methods and data are disclosed, the more you contribute to science. It seems to me there are two points to publish a paper. One, to expose your results, and two, to add your data and methods to the body of knowledge of science for posterity. To think that it’s sufficient to do only the former is not ideal.
I find it shocking that so many “scientists” apparently don’t agree with the need for replication, or even to understand and accept that it is a basic element of scientific method.
The essence is that if you make a scientific statement, you must be able to demonstrate that the scientific statement is well-founded, based on proper procedure etc. Replication is the most effective way to achieve that. Avoidance of replication serves only to cast doubt on the scientific statement, on the basis that “he must have something to hide”.
Peer review is supposed to ensure that scientific statements are robust, well-founded, and supported by replication. However, numerous instance cited on this site indicate that not all peer-reviewers share this view of their responsibilities. Their failure to accept (or even understand apparently) this requirement tends to debase science – an issue that credible scientists must find deeply disconcerting.
Re 60
That is a ludicrous position.
If every time I or any other scientist for that matter wanted to examine a question the initial requirement was to collect all the data for all the previous studies and reanalyse the data before I set about my own work we would never get anywhere. I look at what others have done, how they have done it and I may send them an email to ask about areas which are unclear. (and No people are not always totally forthcoming with methodological details, especially if it is something they are developing themselves). I then go and repeat a similar experiment and see if I can replicate their findings. What you are all talking about is auditing of results. That is what Steve M is trying to do with the multiproxy studies, there is a clue in the title of this blog. It is different to replication in the sense you are talking about and different to the type of replication John A was demanding of John H’s work.
Post #59 is excellent, as is #64. They both make clear that climateAUDIT isn’t about replication but just about a self appointed ‘audit’. Science is, in part, about replication, but it’s also about ideas, observations, gathering data, theories and the rest. This place does none of that – it’s single purpose is to ‘get’ (as in destroy, break, bust, rubbish their work and character) certain scientists.
Re: 60, 64
You do not have to replicate every study before believing it. However, there are a great number of issues where someone reports an experiment, and it is so important that it is necessary to replicate that experiment before proceeding to do further work. After all, if you cannot replicate the experiment, how can you proceed to do further work based on the experiment ?
I think that you are fundamentally wrong about audit. This site is not seeking to audit MBH’s work, save in a very limited sense. The audit extends only so far as to get the data and methods to be able to replicate the work; and this is a very restricted form of audit.
As you know, in pharma research at GLP, audit actually requires all the lab notebooks (and a sight more…); so not only are the methods reproducible, but you have to provide a great deal more information for that level of audit.
Post #65 is PH at his best. Yes, science is about replication, but if you are attempting to do replication on the sainted M, B or H, then you are part of an evil conspiracy to …
cheers
per
Re #60
Lets look at the cold fusion arguments. Cold fusion in a test tube was reported, and like MBH98 some people thought it brilliant, others that it must be wrong. While there was some analysis of the original data (I believe), the real nail in the coffin was that when people looked at the paper, obtained the methods used and tried to replicate the experiment in other labs they failed. That was true scientific replication, get the methods go and try and repeat the experiment. It is repeated experiments which validate or otherwise the original. The other approach, which seems to be advocated by some here is for one person to do an experiment and lots of other people reanalyse the data. If we go back to the cod example. Lets say I reported arithmetic mean of 500g, you get my data, look at it and say a median is more appropriate and that is 600g, someone else says that no, arithmetic mean is correct, but that due to my rounding errors the actual value is 522g. Where does that get us. If instead you had gone and collected 1000 fish measured them and reported the mean (say 533), but said the median was a much better value (say 602) and someone else had gone and done more new research we would be in a much better position. We would have true replicated work, not pseudo replication that John A is so keen of.
I will admit that multiproxy studies are different because it is the analysis of the data which is the experiment, so the data and methods do need to be released, but John A was haranguing John H about his research and how it should be replicated. It was that and normal scientific replication I was originally commenting on.
What do you do if they won’t reveal their methods? To date, I don’t believe that Mann ever fully described his methods in MBH98. (You can ask Steve for confirmation; I think he may have finally guessed what Mann did to get the results).
It tells us that, if your study relied on the 500g mean being correct, then its conclusions could be wrong. It also tells us to be careful not to make the same mistake you did.
The point with the proxies is, you re-run the same code on different data and you get different results. Then you wonder why – is it a problem with the method, or the data? Without having the original data to check against (and see if you get the reported results from the reported method), how do you know?
This is devolving into a joke. Peter’s comment #65 is ludicruous. He’s just applauding everything that anyone says that is anti-climateaudit. I no longer have any doubts that he is a troll, pure and simple. I also have no doubts that any scientists arguing against transparency of methods and data is doing so for any other reason than they can not argue on the substance of the science itself.
Paul, a couple of issues that I think that you are not giving sufficient weight.
First, for “best practices”, I’m not using an abstract benchmark for standards of archiving data and code, but standards in effect already at econometrics journals, which deal with data sets and programs of similar size and scope as millennial paleoclimate. So we’re not talking something impossible. You have to archive data and cade as a condition of review.
Second, it’s been shown in econometrics that if authors are required to archive data and code at the time of publication, there is little inconvenience (and perhaps even benefit to the authors.) Difficulties come after the fact, such as Crowley losing his original data. If he’d dealt with it at the time, it wouldn’t have been a problem.
Third, one of the issues that concerns me about these little cherry picked series is their sensitivity/lack of robustness to proxy selection. If you want to study that, you have to be able to replicate the study.
Fourth, in the U.S., there have already been decisions made on many of the issues in terms of high-level policies for funding and journals often have policies. In many cases, I’m simply asking that people live up to existing policies. It’s one thing for recipients of funding not to live up to some abstract standard and another thing for them to be non-compliant with objective policies.
Fifth, the multiproxy studies that I criticise do not involve the collection of any NEW data. They involve the application of sometimes “new” statistical methods to existing data. Mann didn’t go out and catch any fish himself, nor did Crowley or Moberg. So the processing is important. Understanding the statistical properties of the procedure is important. For example, the mean of all the MBH proxies does not yield a hockey stick; so the processing matters.
Re #68 “This is devolving into a joke. Peter’s comment #65 is ludicruous. He’s just applauding everything that anyone says that is anti-climateaudit.”. Err, a teeny weeny bit unfair. After all I don’t hear you singing in praise of the IPCC very often, or indeed, anyone who accepts the consensus…
But, yes, it is surely in the nature of disagreement that we, well, disagree?
As I have already said
My argument is with those who see anyone involved in climate science as some malevolent force because they do not conform to the ‘norms’ of scietific practice when it comes to archiving data. For example from John A.
Clearly Hunter et al 03 (from the abstract) is not a multiproxy study. So normal methods reporting and replication apply. John A is just trolling. Extensive data archiving may well be the norm in econometrics but take it from me it is not in many other fields. In my own (soil science) some journals require data to be archived, others, regarded as reputable, publish articles where means are quoted with no error terms at all, or at best a LSD. Something I find incredible.
That wasn’t haranguing, that was asking. You’ll know when I’m haranguing anyway…
The request was genuine. He puts his results and methodology on the table and we go away and audit them, and report back in steps good, bad or indifferent. I will even given John Hunter first look so he has an opportunity to rebut.
I don’t like grandstanding, and if John Hunter would rather answer questions by e-mail then its fine by me.
The reason I ask, is that there are several disconnects between the historical record and Pugh, Hunter’s reconstruction that I’d like to comprehend. If their result is reproducible without too much fuss then I’ll report that result without too much fuss.
I’d rather talk to the man himself rather than second guess his reasoning. I’m pretty sure we don’t need a Parliamentary Committee to ask him to show his work.
Replication in the mining game is to repeat the assay by taking another representative aliquot of the original sample and re-analysing it according to the methods previously described for the original assay.
Equally rechecking the computational methods (based on a prevailing theory) is also replication in the sense that a particular result was arrived at not by chemical analysis, but by application of some mathematical theory to the data. Mineral ore reserves are specific examples of this.
It is replication in both instances, one of a physical measurement, the other of an abstract measurement.
Subtle? no, just depends on whether you actually understand what the problem is. Steve et al do.
I am bemused. What SM has tried to do is to repeat the exact experiment reported by MBH. Likewise for John A and John Hunter’s data set. It is also worth pointing out that there is a long tradition of people falsifying results, and direct data audits show this.
I have always considered the analysis of data to be important. It is obvious, even from the examples you suggest, that these discrepancies could make the difference between a result being statistically significant; or not.
cheers
per
Peter, yes, what I said was unfair but that’s beacuse I’m very annoyed with certain peoples’ attitudes.
Me singing the praises of the IPCC? How could anyone praise an organisation which hijacks the credibility of a bunch of scientists to make political points?
I’m sure the IPCC has plenty of good scientists working for it but then the damn politicians step in and make an edit here, an edit there. Before you know it the report has been “adjusted” to be useless.
It’s not the nature of disagreements that we have to have a tit-for-tat conversation. What SHOULD happen (as happened, for example, with our discussion of how much CO2 contributes to the “greenhouse effect”) is that if I say something wrong you point out what it is and why, and if I am indeed wrong I will admit it and change what I am saying. I have already adjusted my position before once you pointed some actual scientific results out to me which I didn’t know about. However, all too often the response to a legitimate challenge is some whinging about not having to make data available or “consensus” or some other bogus concept which is not the actual nuts and bolts of what we are trying to discuss.
Can’t you see that if one of us says “the results of this study are questionable, I think it should be audited” and the response is “why should I give you my data if you’re just going to show what I’ve done wrong?” that tells us a lot about the confidence of that scientist’s conclusions. How dishonest is it, to hope that any flaws in your study are never revealed, in order to keep one’s reputation intact? It may be good for your career but it’s hardly good for mankind. The “irony” of the situation is that it’s the AGW-doomsday-people who claim to have the best interests of mankind in mind, yet they are often the ones who would withhold potentially useful knowledge from the rest of us for selfish reasons.
John Hunter:
It’s interesting what you regard as a “contrarian ploy”. I’m asking for access to the data and models you used to reconstruct sea-level rise from the “isle of the Dead” sea level marker. I’m not asking because I have a political agenda or because I particuarly dislike you. I’m asking that you simply allow access to data that you have already archived. Straightforward n’est-ce pas?
Well, you’ve pretty much described most democracies. I don’t know that I’m asking for “intimate details” as if I were enquiring about your sex life, but the normal process of data archiving that you have already told us six months ago that you were carful to do.
I will reveal my identity when I get academic tenure and/or security of employment. Until then, for my family’s sake, my identity will remain private. If I’ve shown myself to be eminently capable of doing a hatchet job on any scientific work I don’t appear to like, then perhaps those scientific works are not as robust to examination by even people lke me. Besides I never realised that I had to like a piece of work in advance in order to examine it. I’m pretty sure that you weren’t madly in love with John Daly’s work on the isle of the Dead when you examined it, were you?
Well, hopefully the same was also true of Hwang Woo Suk.
Not really. I don’t expect handholding, but even so this is a historical reconstruction here, not an attempt to repeat a genetics experiment.
Actually I can do better than that. I can actually add value to the reconstruction. Bet you’re curious how, aren’t you?
#71. In some cases, it’s helpful to come at things with a fresh pair of eyes. AGU has excellent policies for requiring data archiving, but they don’t enforce them. Same with the U.S. government. If soil science journals have what I would regard as inadequate practices, that doesn’t justify similar poor practices in paleoclimate journals. Paleoclimate is being applied to policy, so I think that high standards should be invoked. The other approach that I have on these thing is not merely accepting the status quo: if paleoclimate journals think that their practices are OK, I cannot imagine that the people who are paying for it will be satisfied once the issue is brought to their attention. ONe way or another, I anticipate that the Hockey Team will have to produce everything. There’s an old saying: you don’t save a dog any pain by cutting off its tail one inch at a time. ONe of the things that attracts attention to this debate is that the Hockey Team fights every inch of the way – every scrap of data is contested. My impression is that this strategy is backfiring on them by creating separate side issues that are distinct from the results themselves.
For what it’s worth, von Storch completely endorsed our position on data and methods and was severely critical of Phil Jones, expressing disbelief that a reputable scientist could say what he did to Warwick Hughes (and vS confirmed that the quote was correct.)
Steve M,
I’m glad to see your point 5 as I was going to post the same thing if you hadn’t. This is what people like John H and others like him, who actually have done scientific work, are missing. The multi-proxy studies, unlike much other scientific work, is essentially historical rather than practical. Let me give an example of both sorts. Let’s switch to astrophysics. If a scientist were to do a practical study of supernovas he’d collect data on a large number of them and then he’d divide them into different sorts depending on objective characteristics; maximum absolute brightness, brightness decay rate, various spectral features, etc. Then he’d publish, declaring how his new taxonomy of supernovas is superior to past ones. In this case auditing his work would involve getting the raw data he worked from and any methods he used to calculate the listed characteristics. A replicator would instead gather data on a different set of supernovas and apply his methods to this data and see if the same results were found. Having the original data would be nice though as it’d let the replicator see if he was doing things right, important if he found different results.
The historical sort of science is interested in finding new characteristics from/of an existing historical database. To be sure the researcher may add to this database in the process but that’s almost incidental. The important things are the new characteristics found. To audit this sort of science is essentially to replicate it as well, as once you’ve verified that the methods used when applied to the data do indeed produce the characteristics claimed you’ve done everything possible and only can discuss what the characteristics mean rather than that they exist.
Switching back to climate proxy reconstruction, we have existing data; tree rings, ice cores, thermometer readings, etc. An article may introduce new data to the field, in which case it should be, (but often apparently isn’t), archived. The results in this case is the production of a set of graphs which reproduce one or more climate characteristics; temperature, rainfall, sea level, ice coverage, etc. Auditing in this case is checking out the data for potential pitfalls and checking to see that the methods used indeed give the results claimed. But there’s not much difference between such auditing and replication itself. What would you do in producing a new replication except pick a different grouping from the existing proxies?
In this sense, Mann and his cohorts are correct in claiming that you (M&M) have produced your own climate reconstructions. What’s dishonest on their part is claiming that since, as you point out, such a sparce reconstruction fails to be statistically meaningful, that this makes your reconstruction invalid. It is, in fact, just as meaningful as theirs and in a reducto ad absurdum sort of way disproves their own reconstruction. In a sense your claim that you haven’t done a reconstruction is what allows them to obsfucate their own method’s lack of validity. OTOH, this is all water over the dam now. The cat’s out of the bag, the fat lady’s sung and all that sort of thing. You are the only ones left standing on the field of battle when it comes to MBH98/99 style climate proxy reconstructions. That’s why they’re desperate to “move on”. The trouble is that there aren’t that many new proxie series to add to the pile so that it’s not that easy to hide behind their new bulworks. You’ve ranged them and now it’s just a matter of deciding where the target rich areas are.
If there’s a prize for most mixed metaphors in one paragraph then this gets my vote…
Let’s be honest whether we say audit, replicate or repeat what we want to do is check if what some one says is true at least numerically and mathematicaly.
If I run a sweet shop here in the UK and I put in my tax return the first thing the tax office do is to put all my numbers into their computer and see if they get the same tax due as I do. This is not an audit but a simple check. If my return adds up they don’t just file it. They look at some key data like return on sales. They have a pretty good idea from other sweet shops what it should be. Suppose they are not happy. Its no good me telling them like John Hunter and Michael Mann do, to go set up their own shop and get their own sales and purchase invoices. They want mine. And if I’ve lost them or refuse to hand them over I get fined.
In the case in point, John Hunter not disclosing his tide gauge data from Port Arthur like Michael Mann’s refusal to disclose the programmes and data enables them to say (if others get different results) you didn’t do it right. The first step and it is only the first step is to check the sums. What is the point of arguing about tide gauges if the sums were wrong anyway?
I’m glad these guys are in climate research and not in medicine or building jumbo jets.
Sorry, I’ve got a bit behind on this due to a power failure, but at least it has let other people take the heat for a while! (Thanks Paul G and Peter H!). Here are a few comments:
Armand MacMurray (#57):
You say that this would be “easy to cheaply fulfill by setting up a website containing the various files” — unfortunately, you don’t understand the magnitude of this task. Our project covered about eight years. The work resulted in over 2000 computer files in 140 folders. It also generated about 40 cm thickness of hardcopy, of which little is in digital form. All of this data (which is well catalogued) will hopefully be kept for posterity in case it is ever required for serious research — but I’m not spending my time putting it on the web (EVEN for serious researchers)!
per (#74):
You say “What SM has tried to do is to repeat the exact experiment reported by MBH. Likewise for John A and John Hunter’s data set.”
Rubbish — John A is clearly not prepared to “repeat the exact experiment” — all he wants to do is find a few mistakes in my arithmetic — not to actually go out and do data collection.
John A (#76):
In response to my question “where is there a profession, anywhere in the world, in which the practitioners are expected to provide intimate details of their work to anyone who cares to ask, no matter how unqualified or incompetent that person may be, and at no recompense?” you reply: “Well, you’ve pretty much described most democracies.”
Again, rubbish — ever tried to make an FOI request? Do you even have FOI legislation in the U.K.? (I think you do, but only quite recently.) Is an FOI application free? Do they require no justification? Are there no exclusions?
Finally, you add “Actually I can do better than that. I can actually add value to the reconstruction. Bet you’re curious how, aren’t you?”
NOPE. I remember Richard Courtney giving a similar veiled threat before John Daly launched his final onslaught on our work — it was the usual flannel, although it did waste a bit of my time in responding.
Also, I’m sorry to tell you that our work was not a “reconstruction” (at least in the normally-used sense) — it was basic data collection and analysis.
David H (#81):
One of the particularly distasteful things about climateaudit is the willingness for posters to pretend A has said X when A quite clearly said Y. You now refer to “John Hunter not disclosing his tide gauge data from Port Arthur”. I HAVE indicated where the tide gauge data may be obtained (see #32). However, it would not do you a lot of good, although you could check that I can do a tidal analysis properly (that is, if you or John A know how) — but you certainly wouldn’t get to our final results. What you would need to do is ferret through the documents contained in the 0.5 Gb of 2000 files of data which relate to our 8 years of work. It IS traceable, but would involve a significant amount of work — and I’m just not prepared to put my time and effort into it.
Finally, you say “I’m glad these guys are in climate research and not in medicine or building jumbo jets.”
Now I guess many of you guys consider flying safely is more important than worrying about whether the world is going to get a few degrees warmer. Aeroplane safety is a REALLY serious topic — it affects most of us. So, Steve, how about:
http://www.jumboaudit.org?
Let’s all write to Boeing saying “aeroplane safety is of concern to us all, so can you please point us all to the data and full methodology on the subject of building a Boeing 747, for the purposes of audit and replication” (to steal from John A).
Gees you guys live on another planet …..
One Trackback
[…] diagrams – the Dahl-Jensen borehole from Greenland showing a pronounced MWP discussed here and an Antarctica borehole which didn’t show a MWP. It turned out (and I confirmed this with […]