Here are Judith Curry’s Comments on the Wegman Report. I appreciate these sorts of contributions and am obviously relying on such contributions (Willis, bender, etc.) more and more.
Here are my comments on the Wegman report. I am not going to comment on any technical aspects related to the hockey stick debate nor make comments on the behaviour of any individual scientists or “auditors”. Rather, I focus only on Wegman’s recommendations.
Recommendation 1. Especially when massive amounts of public monies and human lives are at stake, academic work should have a more intense level of scrutiny and review.
***100% agreement. Peer review is a valid procedure for weeding out papers that have obvious flaws, are topically unsuitable to the journal, don’t provide anything new (essentially duplicative of prior research). Unless a reviewer is very close to the topic being reviewed and is already conducting a related investigation, it is unlikely that deep flaws (not obvious one) would be uncovered in the peer review process. Cliques do exist in any field, and continued peer review from within a clique can generate a false sense of “consensus”. However, inferring cliques from long lists of coauthors is misleading. For example, I have my name on about half dozen papers where the number of coauthors numbers into double digits. On one of these papers, I have never met half of the coauthors, and on each of the other papers some of the coauthors I have never met personally (including 2 of these papers on which I am first author).
Two anecdotes regarding peer review. The Hoyos et al. paper skated through Science’s review process with only minor comments. The media, however, apparently conducted an exhaustive peer review of the paper during the week prior to publication (owing to the large publicity of the WHCC paper). I talked to reporters who mentioned the three mathematicians that they sent it to for review, I received a plethora of emails from scientists mentioning that they received the paper to review and had a question, etc. The Hoyos paper seems to have survived pretty much unscathed, and the media did a good job in the peer review of a paper that they thought was highly relevant. The other anecdote is my BAMS article. When this was submitted (Nov 05), the hurricane media wars were especially intense. I requested that the paper not be reviewed by anyone (from either side) that was involved in the media debate, and requested that it be sent to 2 climate researchers and 2 hurricane researchers for review. The initial review process was a little bit bloody, but the review process on the second version in mid Feb totally broke down owing to the infamous Feb 2 WSJ (front page) article of brain fossilization fame. Two of the reviewers were so hostile towards me as a result of the WSJ article that they could not even focus on the paper. The review process broke down, and I negotiated with the editor as to what the final manuscript should look like. The lesson from this is that good papers can get screwed in the peer review process, and highly relevant papers will receive a substantial amount of scrutiny once published.
It is especially the case that authors of policy-related documents like the IPCC report, Climate Change 2001: The Scientific Basis, should not be the same people as those that constructed the academic papers.
***Reports like the IPCC are not simply assessment reports, but rather they are synthesis reports. Assessment is certainly an element of synthesis, but synthesis is arguably a higher-order activity. It is hard to imagine some model for activities such as the IPCC where the authors of the primary academic papers are not involved in such documents; not only is their expertise valuable, but it is hard to imagine people very far outside the field of expertise that would have the motivation to devote the time and energy to this endeavor, even if they were paid to do it. More of a focus on assessment by outsiders should be included in efforts like the IPCC and the CCSP, but it is the combination of assessment and synthesis that is the most powerful and of greatest value to policy makers.
Recommendation 2. We believe that federally funded research agencies should develop a more comprehensive and concise policy on disclosure. Some consideration should be granted to data collectors to have exclusive use of their data for one or two years, prior to publication. But data collected under federal support should be made publicly available.
***100% agreement. Data, plus some reasonable version of the metadata should be stored in a permanent data archive (NSF and NASA fund numerous such data archives). This should be a requirement, and scientists that do not do this should not receive further funding from that agency. Requirements for making the actual code available seem somewhat less defensible (this is a complex issue and should be considered further), although the method used should be completely transparent and reproducible. Scientists whose methods/codes/etc. are not shared should become less relevant in the scientific debate (this is of course not the case when the same scientists are in charge of assessment reports).
Recommendation 3. With clinical trials for drugs and devices to be approved for human use by the FDA, review and consultation with statisticians is expected. Indeed, it is standard practice to include statisticians in the application-for-approval process. We judge this to be a good policy when public health and also when substantial amounts of monies are involved, for example, when there are major policy decisions to be made based on statistical assessments. In such cases, evaluation by statisticians should be standard practice. This evaluation phase should be a mandatory part of all grant applications and funded accordingly.
***I agree with the first part of this statement, but not the last sentence. Statisticians should be involved in the climate assessment reports. For example, the assessment led by Jerry North under the auspices of the NAS/NRC Climate Research Committee (CRC) did include some scientists with statistical expertise, but arguably it would have been a good idea for the CRC to have contacted Wegman’s NAS Committee on Applied Statistics for suggested members of the assessment committee. NAS/NRC is very good about interacting with boards and other committees and disciplines, although after 3 years of serving on the CRC (I just rotated off), I had never heard mention of Wegman’s committee. I will send a message to the NAS staffer at the CRC about this, although I suspect that they have already connected the dots.
It is not always clear in advance what research will be policy relevant. The Emanuel and Webster et al. studies (completed early summer 2005) became policy relevant as a result of Katrina. With regards to routine statistical evaluation of each paleoclimate research project, the funding for that community is miniscule. Now that the awareness of this community has been raised in terms of the importance of the statistical analysis (and they can count on being audited by climateaudit), it will be interesting to see if there is an increase in the rigor of statistical analysis by the paleoclimate community.
Recommendation 4. Emphasis should be placed on the Federal funding of research related to fundamental understanding of the mechanisms of climate change. Funding should focus on interdisciplinary teams and avoid narrowly focused discipline research.
***Climate is inherently a very multi- and interdisciplinary field. To make progress in this field requires observations, understanding of individual physical processes, and the cumulative integration of these physical process in the context of climate variability. Climate models are the embodiment of our integral understanding of the climate system. A critical element in the establishment of any theory is whether it has predictive value. The complexity in the greenhouse warming issue is that if we wait until we are convinced as to whether the models have predictive value, then we may have missed a window of opportunity for action if the predictions of the models are actually correct. The funding of interdisciplinary projects is rather problematic in the funding agencies, particularly NSF, where the focus is disciplinary research (NSF would disagree with me, but I am very prepared to argue this point with them should they ever find they want to listen to me on this subject). The big “integration” activities related to climate and particularly climate modelling tend to occur at the govt labs such as NASA GISS, GFDL, and NCAR with some sort of “block funding”. The bottom line is that the greater involvement of statisticians in climate research would be a good thing, and Wegman’s NRC committee might be able to influence this in some way. I note that NCAR does have a Geophysical Statistics program http://www.image.ucar.edu/GSP/, our field is paying attention to such issues.
JC’s summary statement: As summarized in my BAMS article hypothesis must pass three tests if it is to be elevated to a theory:
1. Survive scrutiny and debate, including attacks by skeptics.
2. Be the best existing explanation (physical and statistical) for the particular phenomenon.
3. Demonstrate predictive capability.
Yes, we all know this, but I think we need periodic reminding of this so that we don’t end up getting all hung up on the minutiae and over react to a single flaw in a paper and infer that the larger hypothesis has been refuted. Skepticism about whether an argument has been made convincingly and the identification of flaws does not imply that hypothesis has been refuted and the converse must therefore be true. There is a difference between science and trial law. Rejection of a hypothesis requires falsification (innocent until proven guilty), although elevation of a hypothesis to theory (complete exoneration and acceptance) must pass more stringent tests. A defense lawyer’s approach is to try to poke a single hole in the prosecution’s case, so the client can get off without conviction. The defense lawyer’s approach to “discrediting” scientific hypotheses doesn’t cut much mustard with scientists in terms of actually falsifying a hypothesis, but the appropriate scientific response should be to do further work to see if the concern raised can be addressed, this is part of the “survive scrutiny and debate, including attacks by skeptics” test. Note, I am sticking to science qua science here, not addressing the fast tracking of hypotheses to policy issue.
Climate Audit 
58 Comments
Housekeeping Transfer:
Re #323 The following statement:
“The complexity in the greenhouse warming issue is that if we wait until we are convinced as to whether the models have predictive value, then we may have missed a window of opportunity for action if the predictions of the models are actually correct.”
may contain some implicit policy analysis that goes beyond the science. Whether or not we have missed a window of opportunity for action depends on the optimal action assuming the predictions of the models are actually correct. While it might seem obvious to many scientists that the optimal action is to eliminate, or at least severely curtail, growth in CO2 emissions under those circumstances this may not be the case.
First, continued growth in CO2 emissions may have other benefits. For example, faster economic growth is itself of value for many reasons, including allowing us to afford more R&D on new energy technologies. Bringing fossil fuel technologies to more developing countries in particular also has many benefits for health and welfare more generally. Economic growth is also the best known method of reducing fertility rates, and there is very good evidence that CO2 emissions are more strongly related to population growth than to economics growth. The key reason is that economic growth tends to be associated with reduced CO2 output per unit of GDP. Thus, allowing higher growth rates of CO2 emissions in the short run may, by allowing faster rates of economic growth, actually lead to a sharper long decline in CO2 emissions in the future.
The aerial fertilizer effect of added CO2 in the atmosphere is a second reason that eliminating or severely curtailing growth in CO2 emissions may not be an optimal policy even if the GCMs are accurate. The increased productivity of agriculture from the aerial fertilizer effect may be quite important in enabling the earth to feed the likely population in the next 40 years (until lower fertility rates spread to more countries). These benefits of CO2 need to be offset against the possible climate-related costs.
Third, if there is a substantial chance of natural climate change, an optimal response to anthropogenic climate change might be to focus on adaptation and mitigation of losses rather than redcuing the growth in CO2 emissions. Adaptation and mitigation strategies could protect against climate change regardless of its source. This could give us much greater returns on our investment in protecting ourselves from climate shocks.
Fourth, it is likely that continued technological change will make it much less costly to reduce CO2 emissions in the future, or to extract and sequester CO2 should that prove necessary. Making expensive reductions today may not make much sense if we have good reason to expect that the same reductions could be made in, perhaps only 20 years, are far lower cost.
Fifth, the supposition that “the models are actually correct” is in fact a bit of a “leep of faith” at the moment. Given the funds now being spent on climate research and the much better climate data collection tools we now have in pace, it is likely that we will learn a great deal about climate in the next decade or so. The large uncertainty that still exists regarding the mechanisms that control climate means that any investment aimed at affecting future climate is very risky. Accordingly, the expected return needed to justify such investments therefore has to be much greater.
Other considerations are no doubt relevant. My point here was really to observe that indeed the climate issue involves many disciplines. In particular, there is also expertise outside of the scientific areas (and statistics) that is relevant once you start talking about policy. Too much of the debate about policy in this area has been conducted by people who have not in fact thought very carefully about what optimal policies might look like. It seems to be assumed that the “obvious” response of reducing emission growth follows logically soley from identifying the problem.
Comment by Peter Hartley “¢’¬? 8 October 2006 @ 1:56 pm | Edit This
#
Re: # 326
Peter Hartley, a rendition like yours is needed periodically to keep all that transpires on this subject in perspective with a view to the positive as well as negative aspects of it.
Comment by Ken Fritsch “¢’¬? 8 October 2006 @ 2:09 pm | Edit This
#
#323 Judith,
I don’t agree with you about IPCC. IPPCC is, first and foremost, a political entity. If you participate in that process, you accept the fact that the Summary for Policy makers will be written by a small group of people who have not written the bulk of the report, and, furthermore, the report as well as the SPM will be accepted by a VOTE by government representatives. The SPM is voted “line by line”. Participating in that process means that you accept that the science will go through a political vote.
Also, when you say:
The complexity in the greenhouse warming issue is that if we wait until we are convinced as to whether the models have predictive value, then we may have missed a window of opportunity for action if the predictions of the models are actually correct.
That statement is representative of what has gone wrong in the climate debate on policy. Are you a specialist on public policy? You will have to admit that you are not. You are, I am sure, a very talented and rigorous scientist in your field. But for you to determine what is a “window of opportunity” on climate policy, and how is one to deal with imperfect models: you are now outside of your area of expertise. There are people with the expertise on how to make public policy based on imperfect knowledge. It is not up to climate scientists to decide when the science is good enough. Just tell us what you know and what you don’t know, and the other experts will decide how to act. My personal impression is that some climate scientists have taken it upon themselves to “save the planet”. They have decided that there is enough science, that there IS a consensus, and they also decided on what should be done. Furthermore, they use their status as scientists to make these pronouncements. I would argue first, that they don’t have to, and, second, that they should not do it.
If some in the climate science community believe that AGW is real and is dangerous, they should nevertheless refrain from commenting on policy. Stick to the facts. I don’t see why and how other intelligent people could not draw from these facts the necessary conclusions about public policy. Again, this is my personnal impression, but one gets the feeling that some of you think you should overstate your case in order to be listened to. But by doing so, you leave aside the scientific objectivity. It’s as if you don’t trust the “non-experts” to understand the implications of your work.
As much as, as scientific experts, you don’t like “non-experts” to come and criticize your work (or not “show respect”), you should also recognize that, when it comes to public policy, climate experts are no experts any more.
Why can I say that? Let’s say there is a nice analogy with commercial applications of R&D. I am myself a scientist, and have been in academia for some time (about 10 years), then I moved to industry (in fact started my own business). When you’re a university professor, you can believe whatever you want about the commercial potential of your work. You can talk to industry people about how marvelous your inventions are. And if they’re not interested, you may feel they’re not smart enough to recognize the tremendous potential of your work. But the fact is, they know much more than you do about what makes a good commercial product, and it’s much more than just the technology. If there is commercial potential, they will spot it, even if they don’t understand the first thing about the technology.
I’m sorry to say so, but being a University professor just gives you a very narrow view of the world. You may think you’re perfectly aware of this (I mean you’re not THAT dumb!) but you still live in an ivory tower. Your perspective is what it is, but there are other points of view. In a matter as important as climate policy, the scientists should remain very humble. Unfortunately, that’s not what we are seeing.
Comment by Francois Ouellette “¢’¬? 8 October 2006 @ 6:13 pm | Edit This
#
Re #329 You are barking at the wrong horse. For the record, I have staked out a position of strictly avoiding any policy advocacy or recommendations regarding specific policies related to global warming or hurricanes. The farthest that I have gone out on a limb in this area is signing onto an op-ed led by Kerry Emanuel (and including Landsea and Klotzbach as well as a number of other scientists conducting research on this topic) urging decision makers not to get distracted by whether the increased hurricane activity is caused by global warming or not, but urging decision makers to re-evaluate their policies, that were developed in an era of quiet hurricane activity, on coastal engineering, building construction practices, insurance, land use, emergency management, and disaster relief policies in vulnerable regions.
I fully recognize that global warming is a political issue with huge policy implications, and I am engaging actively in the policy process by interacting with decision makers (governors, congressmen, mayors, emergency managers, evangelical groups, insurance companies, energy companies, retail stores). These interactions most emphatically do not include advocating for specific policies, and RP Jr even criticized me over at Prometheus for not taking advantage of a private one hour meeting with Jeb Bush to discuss policy. In any event, its a little hard to characterize me as sitting in an ivory tower. I did not seek out these interactions; rather these decision makers all realize the socioeconomic implications of increased hurricane activity. These socioeconomic implications are nationwide since they influence energy prices, insurance prices, and our taxes through disaster relief, to name a few. I have no intention of discussing this topic further on climateaudit, I will reserve any further remarks on this topic for Prometheus, my positions on this issue are clearly described on various prometheus posts
It took me a while to decide what climateaudit was about: a group of “trial lawyers” looking to poke holes in greenhouse warming theory motivated by trying to preserve a fossil fuel based economy, or a bonafide group of scientific skeptics focusing on the applications of statistics motivated by the need for a better assessment of uncertainty in policy relevant science. I am convinced that SteveM and a number of the other principals are in the bonafide skeptic category. However, there is a substantial amount of noise on the site that gives the impression of “trial lawyers”. Knee jerk reflexive characterizations of a scientists’ policy views or meddling simply because they accept greenhouse warming as a theory do not help.
And to “bonafide skeptics” in the group, I hope that you can engage in a meaningful diaglogue about Wegman’s recommendations, outside of the context of the “pissing match” with Mike Mann.
Comment by Judith Curry “¢’¬? 9 October 2006 @ 4:26 am | Edit This
#
And to “bonafide skeptics” in the group, I hope that you can engage in a meaningful diaglogue about Wegman’s recommendations, outside of the context of the “pissing match” with Mike Mann.
I don’t know how I’m “categorized” and I do not usually want to get involved in these “policy” discussions, but here are my few cents.
I do not like Wegman’s Recommendation 3. I agree with the point (there should be more statisticians involved in climate research), but IMO requiring “statistician included”-label to climate grant applications is wrong medicine. In the worst case you would end up having a lot of statisticians being present in applications, but maybe little or no actual involnment in the research itself. Also if you need to include statisticians, why not the physicists? Or computer engineers? Or so on…
What I’d like to see more is voluntary co-operation between statisticians (mathematicians/physicist/engineers etc.) with true “climate researchers”. IMO, this requires two things:
1) Climate researchers need to admit that their science is already on the edge of their mathematical understanding/training. Also that there is nothing “very special and unique” in climate research… similar porblems are studied in other fields.
2) Statisticians etc. should realize that there are indeed very interesting problems in climate research they can contribute.
If there is some “governmental” intervention needed, then IMO it should come in forms of rewarding co-operation between different groups, e.g., by giving grants to “multidisciplinary research projects”. But IMO the most important thing is to somehow encourage “both sides” to get involved with each other. A good start is to have (like I think Wegman is now having somewhere) special “climate”-sessions in statistics conferences and maybe special “advanced statistics in climate” sessions in climate conferences AND actually somehow get people from “the other side” to take part in those sessions. So somehow to get Judith to attend a statistics conference and Wegman to attend a “climate conference”. This is the way to get to know people you want to work with and make real, beneficial to all, connections.
Comment by Jean S “¢’¬? 9 October 2006 @ 5:38 am | Edit This
#
#338, Judith
“It took me a while to decide what climateaudit was about: a group of “trial lawyers” looking to poke holes in greenhouse warming theory motivated by trying to preserve a fossil fuel based economy, or a bonafide group of scientific skeptics focusing on the applications of statistics motivated by the need for a better assessment of uncertainty in policy relevant science. I am convinced that SteveM and a number of the other principals are in the bonafide skeptic category. However, there is a substantial amount of noise on the site that gives the impression of “trial lawyers”. Knee jerk reflexive characterizations of a scientists’ policy views or meddling simply because they accept greenhouse warming as a theory do not help.”
Do you also think that some of us are “flat earthers’ or perhaps maybe “Holocaust deniers’ also? For the record I (and I suspect the vast majority of the other skeptics on this blog) do not seek “to preserve a fossil fuel based economy” – far from it. What I seek (as I also suspect the vast majority of the other skeptics on this blog seek) is the truth. The truth of whether or not global warming is caused by man (i.e. us) or not? We pay taxes. We are therefore entitled to question just exactly what those taxes are being spent on. We have a right to question just exactly what value for money we are getting from this expenditure. Climate research is no exception.
I worked in the nuclear industry for a long time and I had no problem with environmentalists questioning what I did. They as i do now had a right to question what was going on. Why was nuclear power being subsidised? Why was little research being down on alternative forms of energy? I agreed with their arguments in regard to how the economics of nuclear power where distorted in its favour. I didn’t agree with them on safety however and they were just as alarmist back in the 70s/80/90s about nuclear safety as some climatologists are now about global warming (this isn’t a cue by the way for yet another off thread discussion on nuclear power). I could also recount in detail my experience of the poor science involved in the leukemia clusters debate of the early 90s but I won’t because that would also be off thread.
KevinUK
Comment by KevinUK “¢’¬? 9 October 2006 @ 6:26 am | Edit This
#
#339 Jean, there is a broad spectrum of statistical, mathematical, physical, and chemical expertise in the climate community, in various subfields as well as among individual scientists; this spectrum includes some people with highly sophisticated understanding and application of statistics. The main problem IMO is a general lack of appreciation for the importance of addressing uncertainties; this problem arguably emerged as a result of weather forecasts being “useful”, so as a field we stopped worrying about issues such as the uncertainties in the radiosonde temperature profiles, since these profiles were “usefully” be assimilated into numerical weather prediction models and improving the forecasts. Analysis of climate data is hugely complex from the statistical point of view since you have a disparate collection of measurements that are not intercalibrated and of unknown accuracy, substantial sampling issues, and very large spatial and temporal domains. If we can’t draw conclusions that are statistically significant from these data (many climate researchers don’t even bother about trying to quantify the uncertainty because it is so difficult to figure out how this might be done in many (but not all) cases), how should we proceed? Pretend the uncertainties don’t exist? (bad idea) Rely on theory and models, evaluated against observations where feasible ? (this is what we mostly do) Not do anything since the data isn’t good enough and the models aren’t reliable enough? (bad idea, scientists will keep trying to figure all this out, and society thinks this science is “relevant”.) Raising the expectations for discussions of uncertainty and improving the statistical analysis of data and model output will help move climate research forward. Mechanisms to encourage such collaboration will hopefully be discussed.
Comment by Judith Curry “¢’¬? 9 October 2006 @ 7:11 am | Edit This
#
#338, Judith, My remarks about policy in #326 were not “motivated by trying to preserve a fossil fuel based economy”. I was motivated solely by a desire to provoke a discussion about what sorts of considerations mat be relevant to making good policy in this area. More particularly, I was attempting to make the point that policy analysis requires a consideration of the relative costs and benefits of different types of policies. Identifying a possible problem does not immediately tell us what the appropriate response should be. Even where scientific understanding is necessary to good policy, such understanding is not sufficient. Many other considerations come into play.
Comment by Peter Hartley “¢’¬? 9 October 2006 @ 7:16 am | Edit This
#
#342 Peter, my previous comments were not directed at your post. You are absolutely correct that identifying a problem in no way equates with identifying a solution. The problem arises when a group of people decide that they don’t like one of the solutions and then try to deny that the problem exists. The policy responses to global warming need to address the short term and longer term risks (risks implies uncertainty), and any policies must be practically feasible, cost effective, and politically viable. Policies result from the political process, and the influence of individual scientists or groups of scientists on this process that accept the greenhouse warming theory and some of whom even actively advocate for emissions reductions is not very large IMO, otherwise the Kyoto Protocol wouldn’t be as dead as a doornail.
Comment by Judith Curry “¢’¬? 9 October 2006 @ 7:26 am | Edit This
#
Re #341
The main problem IMO is a general lack of appreciation for the importance of addressing uncertainties
I could not agree more. Denial is pervasive. Let us instead accept these uncertainties, model them where we can, and have them embedded as part of the decision-making process.
Comment by bender “¢’¬? 9 October 2006 @ 7:33 am | Edit This
#
#341 Bender, exactly on target, this is what we need to do
Comment by Judith Curry “¢’¬? 9 October 2006 @ 7:54 am | Edit This
#
We know that the climate has varied considerably in the past (the MWP, LIA and the 28 plus ice age cycles among them.)
We know that CO2 levels have varied considerably in the past (such as being 80% in the early earth, 1.0% 500 million years ago, 0.038% today). Earth’s historical climate seems to have not varied much at all with those levels. In fact, the entire Earth froze over numerous times when CO2 levels were much higher than today.
What we are saying is that there is A LOT of evidence that points in the opposite direction of AGW. Given all these other factors that must be considered, AGW is a theory which needs A LOT of proof.
If we are going to shut down our coal-generated electricity industry and park our cars forever in some garage, we need A LOT of proof.
But what we see is A LOT of alarmism. We see A LOT of “the poles are going to melt”, the sea level will rise 50 metres, the deserts will expand and the amazon will be gone. We see A LOT of sloppy science when one looks into the data and the statistical analysis. We see someone trying to push a “belief” rather than prove a scientific point.
Hurricanes are a really good example. When one uses the whole dataset (instead of just selecting 1970 to 2005 or something), one sees that there is just considerable variability in tropical storms (which does not appear to be correlated with CO2 levels at all.)
The Scientific revolution demanded reproducible proof. That is what lifted mankind into the technological civilization that we have today, what changed medicine from “bleeding” into the truly effective medical treatments we have today.
Why is climatology and global warming science exempt from the scientific process?
Most of us do not work in the fossil fuel industry. We are just people who are interested in science and at some point in the last few years, we came across the global warming debate and our interest was tweaked, mainly by the unproven alarmism that was evident.
Comment by Jeff Weffer “¢’¬? 9 October 2006 @ 8:24 am | Edit This
#
Re #344 and #345 I agree with these comments too. That is one reason I find the predilection in the scientific literature for producing “scenarios” without probabilities so frustrating. It is almost impossible to think about sensible policies without having probabilities attached to the various possible outcomes.
Comment by Peter Hartley “¢’¬? 9 October 2006 @ 8:46 am | Edit This
#
otherwise the Kyoto Protocol wouldn’t be as dead as a doornail.
As a side comment, IMO that’s very US-centric comment. Here, only the direct costs of Kyoto for the national economy have been estimated to be around 1 billion euros (1.25 billion dollars) during the five year period (2006-2010). In a country of five million, that’s 250$/person.
Comment by Jean S “¢’¬? 9 October 2006 @ 9:10 am | Edit This
#
#341/#344: I do also agree. However, calculating exact uncertainties is sometimes very hard, so I don’t really blame climate scientistic for not doing that. Many times it is not done, for instance, in engineering, either. But what I find troubling (from my little reading) is the lack of attempt to do anything “compensatory”. For instance, if you are doing GCM analysis, the very least you could do is to make meaningful control simulations, e.g., twist your model parameters using only observed temperature measurements, and then check the model performance form precipitation records. I really can not understand how anyone can take seriously model temperature predictions, if the same models can not even reproduce interannual precipitation variability in the control simulations as have been the case in a few studies I’ve checked.
Comment by Jean S “¢’¬? 9 October 2006 @ 9:29 am | Edit This
#
Judith: For what it’s worth, I agree with about every thing you say. Call me anything you want, except please don’t call me a trial lawyer.
Comment by jae “¢’¬? 9 October 2006 @ 9:30 am | Edit This
#
#346 Jeff: Thank you for putting into words what I have been thinking. When you ask:
Why is climatology and global warming science exempt from the scientific process?
I am reminded of recent efforts to suspend habeas corpus in the United States. Are things really so desperate that we need to discard the very things we cherish in order to protect them?
Comment by TAC “¢’¬? 9 October 2006 @ 9:33 am | Edit This
#
“It took me a while to decide what climateaudit was about: a group of “trial lawyers” looking to poke holes in greenhouse warming theory motivated by trying to preserve a fossil fuel based economy”
Why would anyone want to preserve a Fossil Fuel based economy, that’s like wanting to preserve a certain form of accounting.
ALL of the Oil companies would love to find the next great thing when it comes to energy. Can you imagine coming up with the energy source that will be used world wide, and getting a patent on it? That’s money that even Bill Gates or even the robber baron trainmen of the last century never imagined. Imagine having a car that could get 200 mpg (not that it is even possible for a road car), if you could you would sell them faster than you could make them. The money would just roll in.
No one has any sort of desire to keep the Fossil fuel economy going. The oil companies don’t care what it is they sell that you put in your tank, just that they sell whatever it is to you. It could be liquid Hamsters, if all of sudden cars ran of liquefied hamsters, then you can bet that the oil companies are going to have liquefied hamster at the pump. Electric companies don’t care how they make electricity, just that they do. They would just assume use Hydro. Here in the states they are not allowed to, but look at Canada. How many hydro plants do they have. Their electric companies use Hydro in the name (and yet people still complain about hydro power but anyways). Electric companies just want to put electricity at your wall socket so that you can use it, and pay them for it. They don’t care how it’s generated.
A comment like:
–“It took me a while to decide what climateaudit was about: a group of “trial lawyers” looking to poke holes in greenhouse warming theory motivated by trying to preserve a fossil fuel based economy”
says a lot more about the orator than about the people she is trying to describe.
Hey John A does that essay count for a bonus on this months check. One of the Ferraris is down, I need a few extra maids, and I’m looking at a nice beach house in St Tropez. Another 10 Mill or so would come in handy.
Comment by ET SidViscous “¢’¬? 9 October 2006 @ 9:49 am | Edit This
#
#338 Judith,
Where has the thick skin gone ?! Must’ve been the “ivory tower” button…always works with academics…
I’m truly sorry if you feel offended by my comments. I was specifically commenting on your previous post here. I haven’t read all your other posts on other blogs.
If you think I am a “trial lawyer looking to poke holes in greenhouse warming theory motivated by trying to preserve a fossil fuel based economy”, isn’t that a knee-jerk characterization of my views? It’s kind of funny because I biked to work for most of my carreer!
So let me try again to state my own point of view. You have the right to not like it.
I still find that the climate scientists speak a kind of double language. There is the prudent and rigorous language of journal publications (with the exception of Jim Hansen). And the rest. You may claim to refrain from policy recommendations, yet you talk about a “window of opportunity”. The policy recommendation is implicit, and it is any time a climate scientist talks to the media. And now it has even contaminated the scientific journals as well (e.g. there was a paper earlier this year in Nature on the extinction of frog species in S. America, where the ABSTRACT mentionned the need for emissions cuts!).
I fully agree with what you’re saying on dealing with uncertainties. In fact, this represents my own position on the subject. But then again: double language! Because you (meaning climate scientists in general) let the media propagate the message that there is no uncertainty in climate science. Just like Michael Mann’s recent letter to Nature, complaining that his message was distorted by the media. How many times have I read that “the science is settled”, that there is no doubt. And then if someone raises objections, characterize him/her as a paid contrarian.
Why not “poke holes” in greenhouse warming theory? What’s wrong with that? Maybe the theory IS wrong, after all. As a scientist, I’ve always felt that I should get up every day asking myself “what if I’m wrong?”. I have repeated experiments dozens of times just to make sure, just to get slightly better data. I’ve sent students back to the lab with the same message: you data isn’t quite good enough! I have rejected quite a few papers, because the experimental results weren’t up to scratch. This paid me well, because sometimes it’s in the little detail that doesn’t quite fit that you learn something new and important. As a scientist, you know that yourself. So when I see papers like MBH98, and quite a few others that I’ve read on climate that really look like sloppy pieces of work, but just try to make themselves interesting by claiming it’s the warmest in a “millyun years”, I cringe! And when I see that “eleven climate scientists” take the time to write a letter to refute a paper correlating the sun’s activity to the climate, and use dubious arguments casting doubts on the author’s scientific integrity, I cringe again!
That you complain about industry-paid contrarians, fair enough. But their arguments are so easy to refute, and as much as you may believe that climate scientists have little influence on policy, I don’t that think the “paid contrarians” have that much influence either. But when those intimidation tactics extend to serious scientists who happen to explore phenomena that might disprove your darling theory, then something is definitely wrong. The implicit bias then rears its ugly face!
Again, I would feel better about your proclaimed neutrality and scientific objectivity if you would openly admit, for example, that Michael Mann’s actions to protect his little turf were reprehensible. It’s not just a “pissing contest”. You can’t claim to remain objective and policy neutral, and not say a word when others in your field dangerously drift towards activism. It’s not enough to say “I didn’t do it!”. In that sense, I am much closer to, say, Hans von Storch’s position.
Comment by Francois Ouellette “¢’¬? 9 October 2006 @ 9:59 am | Edit This
#
Re #349
calculating exact uncertainties is sometimes very hard, so I don’t really blame climate scientistic for not doing that. Many times it is not done, for instance, in engineering, either. But what I find troubling (from my little reading) is the lack of attempt to do anything “compensatory”.
Understood & agreed. The last & ultimate check on uncertain data in any analysis is cautious interpretation. I think we can agree that alarmism is the opposite of that.
Alarmism may have a role to play in the mass media, but not in PNAS, or Nature, or Science or GRL. I would argue that climate scientists need their own specialized journal – “Alarmist Climatology” – so that the alarmism could be compartmentalized within, if not expunged from, the scientific literature. Sound the alarm if you want. Just don’t call it science.
#346 Jeff
Yet another succinct and excellent post from you summing up the problem with the heavily politicised issue of global warming. As a “flat earther’ (as some in the pro AGW community would have me) I think it’s about time that those who live off the back of global warming alarmism (funded by the tax payer) finally admit that the whole climate science industry is a pretence invented by politicians to serve their own political agendas. There are far more important (genuine) problems that need to be addressed on this planet that we all share (e.g. famine, poverty, disease, war etc) than global warming. IMO the vast resources that go into climate research need to be re-directed to solving these real global problems (many of the anthropogenic) and not on perpetuating the eco-theologically inspired myth of anthropogenic global warming.
KevinUK
Comment by KevinUK “¢’¬? 9 October 2006 @ 10:38 am | Edit This
#
Comment by KevinUK “¢’¬? 9 October 2006 @ 11:40 am | Edit This
#
It took me a while to decide what climateaudit was about: a group of “trial lawyers” looking to poke holes in greenhouse warming theory motivated by trying to preserve a fossil fuel based economy, or a bonafide group of scientific skeptics focusing on the applications of statistics motivated by the need for a better assessment of uncertainty in policy relevant science.
If a person with your academic credentials can make an insinuation such as this, I would think that you might understand how some skeptics view (incorrectly) climate scientists. Neither group has a monopoly on insinuation, in my view, and that is why concentrating on content is so important in this discussion.
Comment by Ken Fritsch “¢’¬? 9 October 2006 @ 12:29 pm | Edit This
motivated by trying to preserve a fossil fuel based economy
I’d like to go on record as being all for preserving a fossil fuel based economy, unless and until it is convincingly demonstrated that its costs outweigh its benefits. It and its efficincies have been the engine which has driven a vast amount of wealth creation and its attendent benefits such as several scores of millions of lives saved and made bearable. When something better and more efficient comes along I’ll try and preserve it too.
However as one who has put several lawyer’s kids through college I can assure you that is not the direction from which I approach the issue.:)
On the other hand, I do agree with nearly all of your comments in regard to Wegman’s recomendations
Judith, I read your comments again. I am bugged by this part:
You have indicated (I think) that the climate models contain immense uncertainty. They still don’t include correct information on all the parameters that should be considered (e.g. Solar forcings). If they are “are the embodiment of our integral understanding of the climate system.” we are in deep do-do, if we depend on them. If they define the “window of opportunity,” we are betting “all in” on a pair of duces. Think about it; if you had to bet your fortune on climate models, would you do it? I really doubt it. This whole “precautionary principle” has been discredited hundreds of times, yet it still attracts attention by the doomsdayers and the public, because it seems to offer a way to approach the unknown. I’ll bet you don’t run your life according to the “precautionary principle,” or you wouldn’t even go to work. I get a kick out of environmentalists that spout this principle, and still ride a bike to work!
In fact, if we followed the “precautionary principle” all the time, civilization would fall back into the dark ages. Assuming the worst, can’t have nuclear power. Assuming the worst, can’t have DDT or ANY man-made chemicals; assuming the worst, can’t mess with the human genome; assuming the worst, can’t sail the seas; assuming the worst, can’t burn fossil fuels. LOL.
Dr. Wegman’s suggestion that the FDA is the best model for statistical work astonished me. Of all the agencies he could have named, why the FDA? It’s been in the pocket of the industry it affects, cheerfully and publicly. And it’s been getting worse rapidly. Look at any of the public health sites for more, but this summed it up nicely.
http://content.nejm.org/cgi/content/extract/353/10/969
Volume 353:969-972 September 8, 2005 Number 10
FDA Standards “¢’¬? Good Enough for Government Work?
Jerry Avorn, M.D.
“… there is one area of biomedicine in which the government allows “¢’¬? even defends “¢’¬? a minimal standard that would be unacceptable anywhere else in research. . . .”
Did anyone here attend JSM (the Joint Statistical Meetings) in August in Seattle? Website description: “the largest gathering of statisticians held in North America. It is held jointly with the American Statistical Association, the International Biometric Society (ENAR and WNAR), the Institute of Mathematical Statistics, and the Statistical Society of Canada. Attended by over 5000 people ….”
Dr. Wegman’s statistical organization is just one of several. I worry it may be one too close to the government; looking at his publications, Dr. Wegman’s work has included a lot on data mining from very large databases. During the hearings, I recall, a Rep. asked for more about social network analysis, and the reply (from NAS, I think?) was that nothing more about that could be said in public. So, that work’s not going to be audited, eh?
(Has everyone read The Day of the RFIDs by Edward M. Lerner? downloadable, 85 cents, here: http://www.fictionwise.com/ebooks/eBook39993.htm)
People who worry about trial lawyers trying to preserve the fossil fuel economy need to worry about what Kyoto would do to the price of Aluminum foil.
I think some people haven’t read the whole quote by Judith Curry and are being unfair to what she said.
(my emphasis)
She has written previously about her decision to contribute to blogs and decribed her trepidation that the experience would not be a constructive one. (“It took me a while to decide”)
However, having taken the plunge she is saying that she has found that many contributors and especially the principal here are in the constructive category and have moved the scientific debate on but there are some here who approach the debate in alternative and rather more adversarial ways. This seems an eminently defensible comment.
Uncertainty is the critical point. Climate science is riddled by uncertainties. Those uncertainties haven’t stopped some practitioners from proclaiming the science settled and a consensus. As policy is based on risk analysis, doesn’t it behoove climate scientists to speak clearly about the extent of these uncertainties? Given the complexity and seriousness of these matters it is unacceptable for the others to stay in their warm ivory towers and say nothing about this. Einstein didn’t shirk from making public comments about the consequences of developing of nuclear weapons. If climate scientists don’t speak out who will, the objective neutral media?
#8
How about climate science?
Science, like religion, has a long and troubled history of “false prophets”. I have faith that “Revelation,” if not “Salvation,” is to be found in rigorous science, not in any scientist.
[Does this earn me my first “yellow card” for religious metaphor?]
I understand the necessity of something like the IPCC report, after all it is a good guide. However, what I don’t understand is the chapter for “press and policy makers”! Either the journalist/politician is smart enough to understand the document, OR he is ignorant. In the latter case, he shouldn’t vote on anything of it and in the first case, he obviously knows enough to make a good judgment.
However, these press chapters obviously are a form of political statement by the authors of the IPCC and not scientific per se.
IMO, this report should be considered a report and a good introduction for interested people, but NEVER should it be used as a guide to politics and politicians.
I think the science community has long forgotten the cases in the early 20th century and late 19th century, when biologists and race scientists wanted to engineer a new society. They also could only succeed, because politicians were willing to follow their scientific recommendations, leading to the euthanasia of thousands of immigrants and blacks.
You might think what this has to do with the current situation and climate policies. Well, you can never tell with what wicked policies the political caste comes up (remember eminent domain?).
So long, until the day comes, when all our steps are prepared for us by friendly government officials as to be “carbon-neutral”.
First time commenting here; a useful and revealing site for a non-scientist like myself to follow progress in the field. Can’t comment on the MBH debate because my ignorance does not allow proper analysis. Can, howver, have an opinion on the Wegman findings.
If I was preparing a report, or paper, on any subject, and found that there was a need for substantial statistical analysis, or a large dependency on stats., I would be well advised to show the material to a statistician and ask for feedback, before going into print. This way, if I had done something the wrong way, or used an inappropriate tool, I could avoid embarrassment and ridicule and be more confident of the validity of my work. For all I know, this is what climate scientists already do (though the comments on this and other sites would suggest otherwise). So, in response to that recommendation, my suggestion would simply be that, where appropriate, a statistician should be asked to peer-review publications with respect to the use of statistics. Even better, if this process was undertaken before papers were put into final draft form. Is this expecting too much?
Thanks for an interesting site and regards,
I think there is an interesting issue here and I’m not clear that we are adressing it properly. Wegman says, “you all are not using main stream statisticians”. Mann says, he we have all kinds of statisticians, have courses in it, have North and Bloomfield, who specialize in statistics for climate, have special conferences on it, etc. I wonder what is the difference between what WEgman wants and what is being done. For instance, are the climate statisticians lightweights? Are they refugees from the more hardcore mathematical community, the way Mann is a refugee from physics?
Hi Judith,
A bit OT, I’m afraid, but I assume that, as the rest of us mortals, you watch TV and read newspapers so you must also be subject to the daily bombardment of catastrophic global warming news. The ice caps are melting (even faster than predicted), the polar bears are starving, penguins are losing their habitat, sea levels are rising, Pacific islands are sinking, CO2-induced GW is causing the drought in Spain, the floods in Central Europe, last summer’s heat wave in Britain, last winter’s cold snaps in Russia, hurricanes in the US, tsunamis in the Indian Ocean,…you name it.
As a leading climate scientist, how does all of this hysteria make you feel? Do you think that you climate experts are doing enough to moderate this exposure of the population to a constant stream of non-scientific claims related to climate change? Don’t you think that a mass campaign like this on a single environment topic is unprecedented and in the end, if predictions fail, could do a lot of harm to the public credibility of Science?
Kind regards,
Mikel
#11: Welcome!
Yes, IMO, that’s the true key in the long run. Unfortunately, it won’t happen until there are enough people from, e.g., the statistics community, who have publications in climate journals. Why? Because that’s the way peer-review works: editors send out papers to peers working in the similar areas. If the paper is about hurricanes, it goes to hurricane research people. Although an editor realizes that a paper contains a lot of statistics that should be better checked by a statistician, who would she/he send the paper? If you pick randomly a name from a statistics faculty, he/she is very, very likely to say “No thanks”. Why? Because everyone is busy and most people do not feel comfortable reviewing papers which they do not understand as a whole even if you were given an explicit task to review “the use of statistics” in the paper.
This is an additional reason why I said (#1) that it is crucial to make connections between the statistics community and the climate research community.
Von Storch said that main stream statisticians were often not very useful in climate problems as they often approached things too much from an independent identically-distributed perspective and they were more trouble than it was worth. Sort of like having Rasmus on your side at RC.
I don’t think that the perspectives of Ross and I are irrelevant – climate statistics are autocorrelated and, as I’ve often observed, the statistical problems are probably more akin to econometric statistics than i.i.d. guys. Some of it should also be through better and more relevant education for climate scientists. An econometrics course might be more relevant than a garden variety statistics course. You’d certainly learn to more wary of things like stepwise regression between multicollinear time series (which Rob Wilson has employed in his most recent article, argggh).
TCO #12,
Bias exists within experimenters not only in selection of samples, but selection of methods. Whilst it is important to include specialist climate statisticians, it can also be very useful to get someone from outside the field to look over the work. This can bring a different perspective which is often a great way of routing out bias.
My argument would be to primarily use climate statisticians, but for those climate statisticians to present their ideas to statisticians outside of the climate science community, through the language of statistics and in applied statistics types of journals. Effectively peer review of the climatology statisticians work by independent statisticians. This already happens in a number of fields; I’ve linked to this journal before on this blog as potentially an ideal place for this to happen. Note on the journal definition: provides a forum of communication between applied statisticians… business, computing, economics, ecology, education, management, medicine, operational research and sociology. No climate science. Why?
That would be my preference, others views may be different…
One point on Nychka and Bloomfield – these were not good appointments as outside statisticians. Nychka was a consultant and close associate on Wahl and Ammann – he was a decent guy but had no business being on the NAS Panel in view of his conflicts. I objected to NAS in writing to his appointment and suggested that there were many more appropriate statisticians that could have been appointed, I got totally blown off by NAS on this request. Cicerone said that there were “too many” such suggestions to accommodate. I am very doubtful that this is an accurate statement. I’ll bet that there were very few such suggestions other than mine.
Nychka’s conflicts interfered with his performance on the panel. He noticed Mann’s untrue statement about verification r2, but didn’t say anything or do anything. Hw should have said something and An unconflicted statistician might very well have acted differently.
Steve, agreed. But I’m interested in the broader issue: Wegman says that the field is not well connected enought to main stream statistics. Mann et al can cite guys in the community that have a Ph.D. in statistics. So what’s the clarification? Does Wegman think those guys are no good?
I could be wrong, but I thought I gave a pretty clear example of how statisticians from different fields get together to review each others work in #16; I noted that climate science does not appear in that list.
Did Wegman say they were no good? Does the existence of the above journal imply other statisticians are no good? Of course not! This is a weak strawman statement. Nobody is saying “these people are no good”. Much more along the lines of, including more independent viewpoints should enhance the science with only a small incremental cost.
Which is why I think they need to look at signal processing experts (sorry, I’m biased). IID is great in the classroom, but in practical applications, it takes a walk. SP folks are better described as “statistical applications experts.” The first time a radar engineer, for example, assumes his environment is full of IID signals, and the missle hits the ship anyway (simulations, of course), he quickly realizes another viewpoint is necessary. It is not a surprise that many of the algorithms the climate analysts use were developed by SP folks.
What’s really troubling about VS’ statement, however, is that many of the applications we’ve seen in climate analysis start off with an IID basis.
Mark
Assuming “unconflicted” also implies “not worried about his career.” I know some in this field that are truly afraid of speaking their minds simply because of the damage it could do to their careers if it were discovered they were “skeptics.” Sad that this happens…
Mark
There’s lots of invocation of signal-processing metaphors but it seems to me that this is only a metaphor for proxies – which might not contain a “signal” or contain a different signal. It quickly goes to the assumption of signal plus noise, which is not an assumption in econometrics.
Agreed, but this is another issue altogether, IMO. Identifying what the recovered “signals” are is quite different than uncovering said “signals” in the first place. To date, no climate scientist has been able to prove that the “signals” they are uncovering are even valid representations of the assumed forcings.
Step 1: prove various forcings.
Step 2: use statistical analysis, i.e. signal processing techniques, to uncover each of the forcings in the form of “signals.”
Step 3: prove that each “signal” is indeed representative of one of the above named forcings.
They haven’t even gotten past Step 1 yet.
I don’t know enough about econometrics to speak with any validity, but these techniques (PCA, ICA, RegEm) inherently separate out “signals” and “noise,” even if that’s not how they are viewed. This is an interesting viewpoint to ponder…
Mark
Uh, the above is referenced primarily to the use of proxies to determine temperature, particularly tree-ring proxies.
Mark
I believe Dr. Mann, during a 2004/05 Senate hearing, suggested that one Dr. Soon, from Harvard Univ. shouldn’t be allowed to publish climate papers any more. Dr. Mann made the suggestion after Dr. Soon had published a paper summarizing 250+/- papers on the existance of the LIA and midieval warm period. The existence of one or both would summarilly destroy Dr. Mann’s hockey stick graph. I believe Dr. Mann has made similar comments regarding other scientists. Yet, he appears to remain a prominent figure in the AGW debate. Dr. Mann’s statements are more in tune with an environmental activist rather than a scientist and reflects the current state of polical “debate” in the US. AGW skeptics are still often refered to as being in the hand of the fossil fuel industry. It may be unpopular to ask, but what circles has Dr. Mann been seen in since his 1998 paper.
Here’s how statisticians can help. This is from the “Chandra” affair. Chandra was a professor at Memorial University (Newfoundland). Almost his entire career was built on fraudulent papers, clinical studies that had never been made. Here’s from the CBC’s piece on the subject:
This eventually led to Chandra’s downfall. Well, not exactly a downfall. He quietly resigned to take an “early” retirement, and is now, apparently, making a fortune in India selling his multivitamins…
Talking about fraud, Dr. Curry (respect, respect…) says:
I wonder if she is aware that including multiple co-authors is one way to ensure the credibility of one’s fraudulent paper. Hwang Woo-suk (the Korean researcher on stem cells) had many co-authors who had never participated in his work. Co-authorship is a common currency in the research world: you let me be a co-author on your paper, I’ll do you the same favor, boosting both researchers’ publication list.
Re #27: So fraudulence is correlated with co-author quantity? Is that how things work in physics? Maybe you could list as examples a few recent physics papers with lots of co-authors so we can know which ones are frauds. Did Chandra have a lot of co-authors, BTW?
re 29
The field mentioned in 27 is stem cell biology not physics. Fraud was proven. Co-authors who put their name on papers with which they had only a trivial connection was an aspect of this
Re #27 Large number of coauthors in the climate field usually indicates some sort of large team that contributed to a field experiment or something. On the papers that I was first author on, i tried without success to keep the numbers of coauthors down, but coauthors were insisting that some colleague or other had made a significant contribution to what went into a diagram, etc. There is also some quid pro quo stuff going on as well. People like to have their names on papers since they get “credit” for this at their institution. As a strategy for trying to show consensus with a large number of coauthors, this strategy actually backfires. You then severely limit the pool from which reviewers can be drawn, to a much smaller pool of scientists that have no conflict of interest with any of the authors, and therefore you are less likely to get “insider” favorable reviews. This problem is even worse for grant proposals for large projects with many investigators; you can quickly eliminate anyone from the pool of reviewers who has detailed expertise on the topic owing to conflict of interest problems. The “insider clique” disease is one to be concerned about, but a simple examination of coathor lists may or may not provide any insight into this in a particular case
Re #15 Steve, I agree with von Storch’s general point, but not with the specific application of this to MM. The real value of MM (and your subsequent work and climateaudit) is that you are also trying to dig into the science issues in your critique, so we are starting to get the desired integration of the physics/chemistry with the statistics. I am trying to promote this approach over on the hurricane thread.
#30 Dr. Curry,
I was not in any way insinuating that your papers are fraudulent. This was just a general comment on the dangers of co-authorship. You describe very well how it happens.
I don’t necessarily believe in the “clique” theory either. On the other hand, I think that the current anonymous peer-review system has more perverse effects. You never know who is reviewing your papers. If you make a negative review, you don’t really know if the author is not going to find out one way or another (as you said, the pool or reviewers is always small and most often includes people you know, and are either potential collaborators, or worse, competitors). So you’ve got to watch your back all the time. The easiest way out, if you want to avoid absurd wars where you make a nasty review to someone because you suspect them of doing the same to you, is to make complacent reviews, especially with people you know. Add to this that the whole publication system is overloaded, and there are just too many papers to review (I could easily get one or two a week, if not more, and even now, not having published anything for 10 years, they still send me those damn manuscripts…), and in the long run you end up with a system where peer review is less and less efficient as a quality-control system.
So it’s not a deliberate “clique”. But the result is more or less the same.
One last word about fraud, for Steve B.: it is much more prevalent than most believe. When I started digging, it’s amazing what I found! The thing is that the institutions where it happens aren’t too proud about it, so they keep very quiet when it happens. And I think the scientific community does not like this aspect to be publicized too much. Whenever a fraud is revealed, the scientific establishment is always quick to insist that it’s an exceptional case. The public image of scientists is a precious commodity that they don’t want to be tarnished.
#32: “I was not in any way insinuating that your papers are fraudulent.”
Not Curry’s papers, but perhaps papers by other climate sicentists? I fail to see the point of the back to back posts on fraud re co-authorship and re statisticians. How exactly are they constructive to this discussion? You say that statisticians can help, then you provide an example of outright fraud. So statisticians could be called on to detect the fraudulent papers in climate science? Which papers would those be?
As for co-authorship, some posters complain that not enough scientists are consulted, and you complain that too many scientists could be a sign of fraud.
I agree that editors, etc should be ever vigilant, but it seems you expect to find fraud in the climate field.
Boris, you say …
I would say:
w.
#33 Boris,
The back to back comments are only the result of my own twisted line of thought. After posting on the Chandra affair (which I think was relevant to the ongoing discussion), I re-read Dr. Curry’s post, and her comment on co-authorship reminded me of the link with fraud. I have recently done a lot of research on scientific fraud, hence the link. I apologize if anyone felt that I was accusing her, or climate science in general, of fraud. I was not. Actually, fraud seems to be much more prevalent in the bio-medical field, which in itself is kind of scary…
I nevertheless think that modern science (in general), especially because of the nature of the “publish or perish” system, is plagued by a number of diseases that range from : irrelevant work (a lot of it!…) —> sloppy work —> tweaked or cherry picked results —> outright falsification or even pure invention or results, with a lot of variants in between. MBH98 was, in my opinion, somewhere between the “sloppy work” and “tweaked results” category, not a minor evil, considering its use in policy discussions.
There was a survey on “scientific midconduct” published in Nature sometime in 2005. 0.3% of the respondents admitted of fraud, 1.4% of plagiarism, and more than 5% said they had at least once rejected results that contradicted previous findings. Multiply this by the number of papers published every month, you get an idea…
I think Wegman’s recommendation 3 to require the inclusion of statisticians in grant proposals where their advice and participation is indicated, is an excellent idea. For several years I had the privilege of sitting in on biomedical peer review meetings of federal grant applications as an independent scientific editorial contractor and synthesizing the comments and criticisms of the reviewers into summary statements for the record and for the benefit of the applicants. I recall some instances when reviewers were enthusiastic about a proposal and actually modified the submitted budget to include funding for a statistician when applicants had not requested it, because they thought it was essential to the success of the project.
A handy acronym all might agree on:
“CITOKATE or… Criticism is the Only Known Antidote to Error” — David Brin
As mentioned here:
http://falsepositives.blogspot.com/2005/03/citokate-or-criticism-is-only-known.html
Spence:
Yeah, I saw what you wrote earlier about having climate statisticians validate their methods with the broader stats community, even with mathematical statisticians. I think this is good, but not sure how to get the community to drive this. Also, would like to know if Wegman thinks this.
I’m still interested if Wegman thinks the climate statisticians are no good. Or if they really are no (or not very) good. If you reread my question, I did not say that Wegman had said this. I asked a question. Be open to questions, my friend…
Francois,
Fair enough. I agree with your larger point on fraud, but I don’t think statisticians or less co-authorship will help. After all, statisticians can be frauds as well.
In the end, outright fraud is a huge embarassment, but if fraud is assumed in every case–I think that might be rather worse in terms of delays, etc. If a paper is a fraud it will be found out.
#39 Boris,
Of course we can’t assume fraud in every case. But fraud, when it happens, is not so easily detected. And when it is, it doesn’t mean that proper action is taken. Go read the Chandra piece that I linked to, and you’ll understand.
The point is not that more statisticians or less co-authorship will help. The whole system encourages co-authorship, as Dr. Curry explained: it just gives you one more paper, and a good publication record is the key to your career.
The scientific publication system has many flaws. But it is still touted as a guarantee of scientific quality. In my opinion you can’t rely on peer review to assess a paper’s correctness or honesty. It sure weeds out the obviously bad papers, but that’s about it. Past that, it’s up to the reader to scrutinize it and decide whether it’s good or not.
What it means in this whole climate debate is that the argument that a research that has past peer review is necessarily true, is just not a valid argument. One has to go deeper, like Steve M. has done with MBH. Sure they’re not all that bad, but each must be discussed on its own merits, as is done here with those hurricane papers, for example. But blogs are not a solution in general. Discussing statistical issues is relatively easy: statistics are easily done with a standard home computer, and good knowledge of the statistical techniques, which many people learn in various fields. Most of science requires much more than that: experimental apparatus, very specialized skills etc. You couldn’t do that on a blog!
I rarely, if ever, make technical contributions here, even though I have a Ph.D. in physics. Climate science is huge, and I don’t feel I have the slightest competence to make a scientific comment. It may not seem like it to Dr. Curry, but I have a lot of respect for the work of climate scientists. I learn more every day, reading papers, but I will never have her level of competence. I DO comment on the way the whole field has been hijacked by the political debate, and how it affects the QUALITY of the research. I have no respect for intimidation tactics used to stop other researchers from proposing alternative explanations, or for the kind of propaganda that you read on RC, hidden behind a veneer of objective science.
I may not be an expert on climate, but I can read a scientific paper with a critical eye, having written and reviewed many. A lot is always hidden in between the lines. I have a relatively good grasp of what are the weak points of the AGW theory. When researchers publish a paper with really puzzling results that might put a lot of what they believe in question, you can feel their panic. It’s never written as plainly as “we have no f** idea what’s going on”, just much more subtly than that. That’s why one has to go to the source, not trust press releases, or second hand reports.
Yes. Peer review is just a front end filter, and after that truly rigorous attempts at falsification will begin. Without peer review process there would be too much noise for science to work.
This is why Steve M. (IMO) mentioned “unconflicted” statisticians. There’s a lot less chance of fraud if you have nothing to gain (or lose) from doing a review. Such objective opinions are hard to find.
I have no problem with peer review being only a front end filter (with publications at least). I just think that scientists in general need to see it for what it’s worth. Mann, et. al., seem to think publication is a holy grail of some sorts. The regularly make comments regarding this, as well. If you’re not published via peer review, your ideas are wrong, but if you are, they’re bulletproof. Once these guys pass peer review, they see any attempt at falsification as heretical, and thusly refuse to admit error even when the obvious is found (such as redefining the definition of the expected value of a random vector, grrr).
Mark
#42 I think that scientists are perfectly aware that it’s just a front end filter. But it’s just not the way they want the public to perceive it! Double language!
Peer review IS useful to filter out the noise. Once a paper has been published in the litterature, a real discussion can begin. That’s why I think that the very valuable work done here by Steve M. and others should, at some point, be summed up in a paper and submitted for publication, if it’s to serve a useful purpose for other scientists in the field. On the other hand, if this is just a hobby for Steve M., why should he bother!
Submission to Journal of Negative Results in Climate Science
Abstract: I couldn’t reproduce MBH99 CIs.
#35 Francois O,
“There was a survey on “scientific midconduct” published in Nature sometime in 2005. 0.3% of the respondents admitted of fraud, 1.4% of plagiarism, and more than 5% said they had at least once rejected results that contradicted previous findings. Multiply this by the number of papers published every month, you get an idea…”
I’m tempted to add that I suspect that the other (100 – 0.3 – 1.4 – 5) % probably hadn’t just been to confession the day before filling in the survey?
And given that this was published in Nature is it reliable? Maybe the article had been peer reviewed by an anonymous fraudster?
KevinUK
#45 I can’t find the Nature link any more (it’s on a different computer), but here is a Washington Post report on it.
I personnally know of one researcher in my field who published fraudulent results, and was never caught, even though we all knew about it. One of his students left him to come complete his Ph.D. with me because he couldn’t stand it any more. But a reviewer has to presume innocence, and trust that the author hasn’t tweaked his/her results. You can’t just say: I don’t believe it. You have to have a strong argument, and in most cases you don’t. I reviewed one of his papers, and I knew the results had been “bonified”, but how could I prove it?
Eventually, when the tech bubble came, that guy realized there was more money to be made by defrauding investors, rather than scientific journals or grant agencies. He founded a company based on a technology that didn’t work. They never made a single product that worked, let alone sell one, but went public, and he sold his stock near the peak, making many millions. He was eventually ousted, so he now lives happily with his millions somewhere on this planet, at a good distance from all those people whom he screwed…
re 30:
” … but coauthors were insisting that some colleague or other had made a significant contribution to what went into a diagram, etc. There is also some quid pro quo stuff going on as well. People like to have their names on papers since they get “credit” for this at their institution.”
Its interesting that universities would use an easily gamed system to determine who is removed and retained, who is praised and punished and who is tenured and terminated. Its almost like letting Sports Illustrated decide when to fire the football coach.
re: #47
Maybe we need some sort of power factor to determine how much to count for a person’s publication, like PF = Cp/n where C is a constant depending on the Journal published in, p is the number of pages and n is the number of co-authors. This would encourage the more important authors to hold down the number of less important authors.
Re #47: in some subfields, the publication inflation factor has become ridiculous. in evaluating my faculty members, i focus on the papers that orginate from their research group (i.e. the faculty member or a student or postdoc in their research group) is first author. Being 15th author on 15 publications ends up counting pretty much as zero in my evaluations.
I think the problem with requiring a statistician to review or co-author climate papers is that it is too easy in most fields to find someone who is sympathetic enough to overlook the problems and to concentrate on other areas of the paper, like discussing the number of PCs rather than testing for the undue influence of bristlecone pines. Steve has illustrated this with the Nychtka example.
I think that what is already happening is the most effective way to keep the science honest. That is, people like MM have to be able to publish an analysis that de-constructs, then the original authors have to have a shot at defending themselves, etc.
Any kind of government-imposed “solution” will be very easy to get around. They always are (hey, where is the foundation for that over-arching conclusion? Well, it’s an opinion)
re 50
A statistician will likely value his reputation among his peers in statistics. If he/she does slipshod analysis to support some pet climate cause, he/she may gain kudo’s in the climate community but denigration among the statisitcal peers who can hand out grants, academic postings etc.
#50 — “I think that what is already happening is the most effective way to keep the science honest. That is, people like MM have to be able to publish an analysis that de-constructs, then the original authors have to have a shot at defending themselves, etc.”
Having spent most of my adult life in academic science, where there are large egos galore, I have never, ever seen the aggressive obscurantism that climate science tolerates. This improper accommodation is especially egregious given the public exposure of climate science, in which researchers are regularly interviewed by reporters and regularly testify before Congress. Under those circumstances, where a climate researcher is almost ipso facto a public figure (does one ever modestly decline an interview?), there is no excuse for not publicly calling for transparency in data and methods.
That no one in the field has publicly and assertively stepped up to this plate is astonishing and reflects very badly on the workers in the field, and is, frankly, slightly nauseating. After all, who more than a climate researcher has the standing to call for open climate science, and who more than a climate researcher has the ethical duty to call for an end to the active and public obstructionism by some leading climate scientists?
And regarding #43 — “Once a paper has been published in the litterature, a real discussion can begin. That’s why I think that the very valuable work done here by Steve M. …,” I’ll add this: In my own field, I have never observed that important published papers are so starkly wrong as MBH9x, without someone inside the field, perphaps even the original author, discovering the error and publishing a correction/retraction. This is the proper sort of self-correction that is typical of healthy science.
But in climate science someone outside the field had to critique it. The fact that external correction was necessary at all, all the while MBH9x was being widely accepted at face value by other climate scientists, and all the while MBH9x was even being elaborated upon by other scientists using analogously flawed methods, and all the while MBH9x was being uncritically and aggressively touted by IPCC-associated climate scientists, together shows that climate science is become largely corrupt. This conclusion is only reinforced by the continuing loyalty shown by climate scientists to the MBH9x results and methods despite their clear analytical disproof by M&M and despite the brazenly political defense of the hockey team.
I don’t think any other parts of science share this systematic failing; not even biomedical. I can also say that to my knowledge in no other branch of physical science are the researchers so kind to their theory as to accept as worthy, predictions that are no better than about 0.2 sigma unique with respect to (still uncalculated after 20 years!) confidence limits. This laxness, too — facilitating a tendentious conclusion-mongering as it does — signals a corrupted intellectuality.
The broom that finally arrives will probably be harsh, and will probably sweep away more than just the deserving.
43. Certainly, Steve can do whatever he wants to. While 90+% of scientists have an implicit (or explicit) requirement to publish to justify the funds spent on their work, or even if they are self-funded have professional obligation to the field to publish, Steve has none of these. He can do as he likes. Of course, it is VERY reasonable not to take seriously the selective snippets of unfinished criticism posted on this “controlled territory”. Or not as seriously as if they were finished, were quantified, were fair, etc. Note, I’m not “telling” Steve to publish. I’m “telling” readers to take the comments on this blog with a liter of salt, given the lack of publication.
Pat: Amen, man.
TCO: all in good time, man. I’ll bet there are some imminent publications. BTW, who needs your negative comments, anyway? The “snippets” are valuable in their own right. Many quotes from Einstein are from his “snippets,” not his publications.
TCO, this blog has seen many discussions and defenses of the claims put forward here. Errors have been found here, and rectified. There are a host of very smart people here who take very little for granted.
Your post assumes that there is no critical examination of the claims made here. However, I’ve been busted (and quite rightly) by people (including yourself) when I have made errors. I found an error in Steve M’s calculation of Deken’s G. ruber calculations, which he acknowledged.
Given that constant, on-going, unrelenting examination of everything posted here, I find quite telling the number of unanswered scientific questions that remain in the various threads. As an example, I posted asking for an explanation of the remarkable divergence of temperature records from adjacent boreholes. You’d think that if a borehole scientist had a good answer, or even a reasonable conjecture, they would post it … but my question remains unanswered.
Your faith in refereed journals, while touching, is quite naive. See, for example, the excellent discussion at Refereed Journals: Do They Insure Quality
or Enforce Orthodoxy?, by Frank J. Tipler, Professor of Mathematical Physics, Tulane University. Here’s a quote from the paper:
So yes, take the postings here with a reasonable dose of salt … but no more salt than you would use for results published in refereed journals. Me, I got tired of being rejected by journals for not toeing the party line, or for not having a PhD or an institutional affiliation, so I post my results here.
w.
#56 Willis,
Well, maybe it’s time to found a new journal! (I’m only half joking…)
The fact is, there have been comments that the issues raised here never quite get closure. Writing them up in a paper would do that. Again, it all depends what Steve M. wants to achieve. But if you want to be considered “legitimate” by other scientists, it’s best to publish. The GRL paper is proof of that. Of course it ain’t easy. My comment on the “Gray” thread about my first scientific paper didn’t mention that, submitted to the Physical Review, it was rejected. Of course, I had no name in the field and my supervisor wasn’t very well known either. I had to publish it in “Canadian Journal of Physics”, which is where every physicist in Canada publishes papers rejected elsewhere… My most cited paper (246 on Google Scholar) only got mild reviews. It is still regularly cited 20 years after.
Any manuscript written by one of the posters here (including Steve M.) could be criticized, and improved, to the point where it would almost certainly meet any reviewer’s criterion, except dishonesty… From then on, it’s just a matter of persevering with the Editor.
Yes, we all know this, but I think we need periodic reminding of this so that we don’t end up getting all hung up on the minutiae and over react to a single flaw in a paper and infer that the larger hypothesis has been refuted. Skepticism about whether an argument has been made convincingly and the identification of flaws does not imply that hypothesis has been refuted and the converse must therefore be true.
Indeed, excellent points.
However, the situation thus far is this:
We have all been told that technology is killing us. We have been told that our technology is going to cause a great cataclysm, a horrible event during which countless human lives may be lost due to our technological arrogance. We have been told these things because some people have a very real fear of such catastrophes( and for good reasons mind you), and these same people have been MIS-using portions of good and bad scientific inquiry to justify and further sell other people on their own fears. They have been selectively showing what supports their fears and ignoring what does not support their fears.
We have all been told that Global Warming will doom us. We have all been told that Global Warming is a result of hubris. We have all been told that in order to avert this we must subscribe to a belief in reducing our fossil fuel use for a better tomorrow.
An attack upon the distortions of scientific inquiry used to sell fear is always fully justified regardless of field provided scientific inquiry continues in a manner which progresses human knowledge. Doing such a thing actually cleans science of those who would misuse it for lies and re-deprecates experimentation to the thankless, fameless grunt work that it truly is and must remain if we are ever to mature as a species. Let the painters do their job in revealing truth, do not look at their last stroke and claim the sky is falling just because it may point downward.