Here is a sociological study of how GCM modelers relate to their models. [Lahsen, 2005. Seductive Simulations?
Uncertainty Distribution Around Climate Models, Social Studies of Science, 35, 895-922.] I kid you not. Lahsen:
“At least at the time of my fieldwork, close users and potential close users at NCAR (mostly synoptically trained meteorologists who would like to have a chance to validate the models) complained that modelers had a “fortress mentality”. In the words of one such user I interviewed, the model developers had “built themselves into a shell into which external ideas do not enter’. His criticism suggests that users who were more removed from the sites of GCM development sometimes have knowledge of model limitations that modelers themselves are unwilling, and perhaps unable, to countenance.”
Sound anything like Wegman’s size-up of the Team? Another comment:
Recognition of this tendency may be reflected in modelers’ jokes among themselves. For example, one group joked about a “dream button” allowing them “Star Wars style” to blow up a satellite when its data did not support their model output. They then jokingly discussed a second best option of inserting their model’s output straight into the satellite data output.
Afterwards re-read the executive summary of the U.S. CCSP in which they discuss the inconsistency between models predicting that the trend in troposphere temperatures would exceed surface trends and observations, which show the opposite. Faced with this inconsistency, Wigley and other senior climate scientists blamed their lying eyes:
A potentially serious inconsistency [between model results and observations] has been identified in the tropics. The favored explanation for this is residual error in the observations, but the issue is still open.
It is quite a remarkable article. Enjoy.
Climate Audit 
75 Comments
I’ve seen similar trends growing in chemistry. Some people tend to treat the output of computer QM calculations of molecules as *data.* This trend is, if anything, growing. One of the reasons, in my opinion, is because computer output graphics are so visually compelling. The images have 3D lighting, the colors are attractive. It all is made to look convincing, and people go for the image. After a while the mental state takes over and becomes habitual. The researchers lose the big picture. Perspective becomes lost. Calculations become data. People actually write papers with titles like ‘Doing Chemistry with Density Functional Theory,’ such as, as though theory-based calculations can produce physically real results. I think the tendency is dangerous, and going to lead to some really bad mistakes. I asked a physicist friend about this, and he said the tendency is there, too.
Seen it before, somewhere else.
Looks like a must read. But jokes about destroying equipment or hacking data are a tribute to uncertainty not a denial of it. In my own corner of the DM world, I like to joke about time machines. Not because the people who have caught themselves in a circular reasoning trap really believe in time machines. And not because I want to win friends and influence people either.
It is only to be expected. Some of these guys have got 25 years invested in this work, they are not going to let outsiders come along and point out errors in the models just for the sake of it…..
Type in “GCM experiments” AND “climate” into Google and you’ll get a 27,900 hits! This terminology, which I recall being used as far back as the late 1980s, suggests that GCM are not models, but reality.
This reminds me of a comment from a professor of mine in graduate school. A student asked the professor how usefull a particular computer modelling software package was. His response “Well, it makes pretty pictures, but the numbers are meaningless.” Oh how many times we fudged the inputs until we got the results we needed, then went back and justified our fudges or just hid them. Of course, we were just graduate students…..
There is no experiment that can make predictions, only models have the power to do that.
Re #1
When I was a post doc (after a PhD in Chem, I worked in a physics group) I saw the tendency froth over violently at times. Nano-bio technology is an area where the models and pretty pictures so outweigh any results that it was easy to get lost in seminars and forget that you were not, in fact, seeing something that someone had actually done.
The pictures of nanotubes, coated with kinesin or some molecular motor, followed by dynamics simulations…the head spins, and at the end, you are shocked that no physical demonstration of any portion of the proposed device had been made. I think the idea is/was really neat. I was just surprised at the amount of ‘virtual’ work being done, when the biomaterials and nanotubes were readily available. If the modeling had been done to support or explain observations. But talk after talk was the pretty pictures about what we can/will be able to make. Nothing about what we have made or are trying to make.
Re #8
And only experiments can falsify models. Insert your favorite pithy quote by Feynman on the relative merits of theory and experiment. Models that make predictions that are not observed are incomplete or wrong.
re: #9
Surely you wouldn’t complain to an architect if he showed you his sketches of a proposed building, or an engineer who showed a blueprint of a product that they should be going out and actually building their house or product, would you? It’s possible that some of these nano-tech computer jockeys were forgetting that what they were building was simply a sketch or blueprint, but I wouldn’t see it as much of a systematic problem. AFAIK, the proper sort of chemical bond strengths, etc. are being used in the models.
I don’t know that this should come as a surprise to anyone. The term “bunker mentality” has been around for a long time. The natural tendency is to close ranks when people start questioning your results or requesting information on how you put things together. Many people tend to get defensive in these situations. Not that it is right, of course. Just understandable.
The problem here though, is that while in many cases, the effects of the situation are minimal in scope, in the case of climate change and whether there is a A in GW and whether it is related to the emissions of A, will have huge ramifications in terms of policies implemented based on the case. This is not an area where we can afford to be wrong, in either outcome. As such, it would be of utmost importance, for all involved to be as open and honest as possible.
This here is another very interesting paper on the sociology of climate modelers, and has a lot of good stuff on flux adjustment (which was recently discussed on another thread). A bit old (1999) but worth the reading.
#10
-No, Dave, you are completely correct that the design phase is important. It is a big mistake to just start synthesizing stuff, without some guidance from theory. I was talking more about really elaborate modeling of how the device should act. While I would want a little simulation up front, to get a feel for what to expect, my contention is that this has been substituted a lot for just finally doing the experiment. This is especially true in nanotech of all stripes, especially ‘Drexlerian’ nanotech, which really doesn’t exist yet. You have to have the nanoscience to get nanotechnology. If I ran the world, we’d do the occasional experiment to see what we could get, then base new, wildly optimistic simulations on that 🙂
None of that is to detract from the value of modeling to experimenters, nor the value of modeling to act as a test of understanding, when the results are compared to experiment. Prediction is just harder.
I would liken what I saw to designing a house, then running a detailed, decade long simulation of temperature distribution in the house, and then declaring it being shown to be comfortable. The line between useful and excessive is blurry, and even with my critiques, I thought that their work was stunning. I couldn’t characterize excessive faith in models as systemic, although most of the modelers I know have a visceral love for their models that they don’t have for the nitty gritty of bench work. My bias is clear, and not prescriptive, and my observations anecdotal. I love computers too, and I like being able to run QM calculations on new materials on my desktop. I don’t trust the results, especially coming from my calcs, but they are a useful guide, and I am better off having them available.
In the case of the nano-bio stuff that I saw, there were some fairly unrealistic assumptions about the nanotubes. About what we could actually make. Some of the stuff I saw was support for a NSF grant proposal, so I guess it being wildly optimistic is fine. There is often a friendly friction between theorists and experimentalists. I just realized who was going to be asked to actually produce perfect unbundled straight nanotubes of uniform length…
RE: #1 Eli Rabbat made that precise argument, namely, that a model was better than a good chem lab, over on Pielke Sr’s blog recently. I was both astounded and saddened.
The natural climate system is composed of a tremendous number of variables that are constantly changing and interacting with one another. This maze of variables and interrelationships among variables results in a system that is difficult, at best, to simulate with even the most sophisticated computer models. The challenge in modeling the climate system arise in the areas of mathematical complexity of the governing equations, the temporal and spatial scales at which climatic variables are acting (and interacting), and in the perceived importance of particular variables and climatic processes.
Jenn
RE: #15 – we draw and quarter spammers. 😦
That text she posted was copied/pasted from here:
http://sedac.ciesin.columbia.edu/mva/iamcc.tg/GCM_thematic_guide.html
Spambot?
#14 — amazing. Theory without results is just
religpost-modernism.It’s a stunner how so many scientists can lose sight of what science actually is: Theory and results. Either one without the other leads to nonsense. Typically, tyranny.
#9 Dave Eaton
Models can also be falsified by other means: false assumptions and inconsistency for example. Above all else, scientific models must make falsifiable predictions. (Karl Popper, empirical falsifiability)
Steve,
I enter this discussion with some trepidation, but I would like to try to clarify some of the methods and goals of climate modelers, as long as the discussion stays reasonably civil. I thought I would start by defending the CCSP executive summary quote concerning the vertical structure of the temperature trends. But before getting into that, let me just state that from my perspective the climate modeling community is a diverse group of individuals motivated primarily by a desire to understand the complex and beautiful climate system that we are dependent on, who find simulation a useful tool for that purpose, and who find themselves in the middle of an extraordinary vortex of activity and attention because their work has become central to profound policy choices facing the world. Some relish the spotlight; others are repelled by it. Some oversell their research, others are exceptionally modest and even overly critical. Some are theoretically oriented and are interested in studying the resulting models as fascinating chaotic dynamical systems, but most spend the bulk of their time confronting the models with observations and studying the differences. Why does our El Nino have too short a period? Why doesn’t our summer rainfall in the Midwest US have a nocturnal rather than mid-afternoon maximum? If you want a constructive dialogue with some people active in this field, you will need to start by avoiding stereotypes and assuming instead that we share the same goals.
With regard to the quote on the vertical stucture of the temperature trends, you make it sound like it is the models vs observations, but that is not the whole story. It is useful to start with El Nino rather than the trends. In El Nino the tropical atmosphere warms, and the vertical structure of this warming is indeed close to that obtained by assuming that the atmosphere stays near a moist adiabat, that is, the warming is substantially larger in the upper troposphere than near the surface. Models simulate this structure of the warming very well — they are not tuned to do this; it is very hard to get them to do anything else. Indeed, this kind of response is pretty basic to our understanding of the dynamics of the tropical atmosphere irrespective of any computer simulations. At least on El Nino time scales, the atmosphere, models and simple theories agree. It is this happy picture of agreement between models and observations (not a blind faith in the models) that is challenged by the estimates of the vertical structure of the temperature trends. The models behave the same way whether the warming is due to El Nino or a more uniform warming of the surface. There are no obvious physical reasons to expect anything else, unless there are other forcing agents in the upper troposphere (ozone?) overprinting the moist adiabatic behavior expected from a warmer surface. So there is a real tension here, which is as it should be — if the models were so flexible that we easily could get them to do anything they would be useless.
If the upper tropospheric warming trend is smaller than, or even of the same amplitude as, the low level warming in the tropics, this will be fascinating and disconcerting, as this means that the tropical atmosphere is becoming more gravitationally unstable with time, which likely means much more dramatic changes in tropical meteorology than the case of a more “neutral” warming following a moist adiabat. We should be praying that the models are right!
Ah, Dr. Held, good to have you here. I naturally went and looked you up on the internet finding
Dr. Held’s bio
It sounds like you would have a lot to contribute / teach us here. I hope those who might find things to question you about are able to do so in a respectful and intelligent manner.
I note that the first recent paper listed on the page I linked to seems concerned with the “maximum entropy” concept that we were discussing here not too long ago. This concerned a paper by “Ou” or something like that. The concept seemed quite intreguing to me. Is that paper of yours available somewhere as a .pdf? I don’t have easy access to a technical library and hate paying several dollars just for one paper. [Probably not good form to ask for a freeby to start with, but we skeptics tend to be on tight budgets.]
Anyway, concerning your message above, I’m not sure quite what you’re getting vis a vis the El Nino at since the real question concern AGW is what the water vapor feedback is. I note you also have a — review article it looks like — on precisely that subject so I hope we can get into that subject and exactly how the feedback is instantiated into existing climate models and what the relationship is between water vapor and cloud formation and what experiments have been done to verify the assumptions made on this relationship.
But again, welcome!
The paper’s available, here’s how to search the Internet for it:
http://scholar.google.com/scholar?sourceid=Mozilla-search&q=Journal+of+the+Atmospheric+Sciences%2C+59%282%29%2C+125-139
Dave,
Dr. Held’s bio page is both exceptionally humble, and out-of-date. He is widely, dare I say, massively, published in the atmospheric sciences, with a more recent example being this article in BAMS (Bulletin of the American Meteorological Society), here.
In fact, I am currently trudging my way through an AMS monograph titled “The Life Cycles of Extratropical Cyclones,” to which Dr. Held was a contributor.
For a better perspective on his pubs record, one can type his name into google scholar (or one’s favorite academic search tool).
Sorry…I never like the feeling of shining somebody up, but in this case the linked page may give some the impression he is just a casual scientist who occasionally publishes, which is not the case at all.
#20 Isaac: Welcome! I, as a student of this material, am delighted to see your comment both because your presense here offers the opportunity for me to learn (much easier when people who actually know something offer insight) and because I am impressed by how carefully and elegantly your comment itself is worded — wise, substantive and useful. I respond “with some trepidation” myself.
In response to your comment, I wonder if you could elaborate a bit on the possibility that the
What sort of changes in “tropical meteorology” would you expect to see? (More, or more powerful, hurricanes, perhaps?).
Isaac’s GFDL home page is better; scroll down for links to pre-pub papers or click on the biblography link for a page with links to his published papers. GFDL has a very nice page that makes available the papers published by all of its scientists.
Not to put words in his mouth, but I suspect part of Isaac’s trepidation may be about being asked to explain and justify climate modeling from first principles.
Not to put words in Bloom’s mouth, but I suspect part of his trepidation may be about serious people like Dr. Held engaging here.
Dr Held, welcome indeed.
Dr. Held, welcome to the discussion. I am impressed with your bio, and with the one paper of yours that I have read, “Simulation of the recent multi-decadal increase of Atlantic hurricane activity using an 18-km grid regional model”. I found it to be an interesting new approach, where you build a box, and run the model inside the box with the sides, top, and bottom constrained, or “nudged”, in your terms, by real-world data. This seems like a potentially more productive method than historical attempts to model hurricanes.
I was curious about your model, in particular, how much of it is based on physical laws, and how much of it is parameterized? As examples of the ends of the spectrum, the Jacobson GATOR-GCMOM model appears to be based entirely on physical laws, with no parameters. At the other end, GCMs seem to be a mix of parameters and physical laws, with many tunable parameters. Where does your model fit in this spectrum?
Finally, I would like to commend you on the paper referred to above. You spend as much time in the paper discussing the shortcomings and problems of the model as its undeniable successes, and you are very open about the places where the model needs improvement. This is a very refreshing and unusual point of view, given that the majority of modeling papers I read are like Hansen’s “smoking gun” paper, Earth’s Energy Imbalance: Confirmation and Implications, wherein they make the claim that their model can detect a 0.85 ± 0.15 w/m2 energy imbalance in the Earth’s energy budget, and that this is the “smoking gun” that “proves” that CO2 is responsible … right.
Given the general climate of these inflated model claims and climate science hyperbole, your paper is a very welcome example of real science in the field.
Finally, I notice that you use the HURDAT dataset for your comparison to your model results. I would be interested in any comments you might make about that dataset.
My thanks for your contribution,
w.
DD: My take on the El Nino comment was that he is saying that the models show El Nino behaviour, when they are not tuned to do so. It would be like having an argument about bcps being cherrypicked for a 20th century trend and then looking at year to year variation and seeing that the hills and valleys of temp and ring size corresponded (doesn’t happen with tree rings, does with corals). I would like to see some quantification and display of this phenomenon (El Nino prediction) if I have got that right.
Isaac:
The article on modeling was interesting. I think that modeling is fraught with danger for fooling ones self given all the time and effort of Ph.D.’s going into it, given the huge complexity, given the seperation from the real world, etc. There is no reason that this HAS to happen. It is just a psychological danger that scientists will as Feynman would say, “fool themselves”. I have seen the same thing happen on very tough experimental problems with a lot of variables, where people after doing a lot of experiments, but not having cracked the problem convinced themselves that they had something, had to…given all the time spent.
On your article, I think the primary benefit of pushing for elegance and for physical intuition is that it will help people stop from fooling themselves with “black boxes”. It is also interesting that Steve has been asking about lower dimension models and what they could tell us, and that you have done some useful work with simplified approaches. What benefit, do you think one can get from 1, 2 D models, things with radial symmetry, etc.?
One thing that you did not push so much in the article (but which I think is important) is the need for true “out of sample” checking of model predictions. If people are always running to the next model and never holding the old ones up to scrutiny, that may be an incentive to not have one’s work checked by reality. There is an interesting article by an economist on the need to wait for 20-30 years to get out of sample checks on models derived from old data. Perhaps, GCMs also need to wait to have info assessed. That doesn’t stop you from publishing or making some intutions, but you really can’t make a strong claim until it is checked by reality.
Dr. Held, perhaps you could point me towards an explanation of some things that stump me:
Here is plot that shows the trend in specific humidity of the middle troposphere (500mb) in the global tropics (25N to 25S). There has been a definite drop. Is such a humidity drop understood and expected from global warming?
And, here is a similar plot for the upper troposphere (300mb). The humidity jumps in 1976-77 and then resumes the downward trend. What triggered the 1976-77 rise?
I expected to see upward drifts in humidity, and no major change like what happened at the 300mb level.
This is NCEP radiosonde data, which ought to be a good source, especially after the early years of the plots.
Thanks,
David
Re #24: In some vague sense I guess I would expect a more active tropics if the stability decreases, but the bottom line is that we have little confidence in predicting how tropical convection will reorganize if something as fundamental as the gravitational stability changes significantly.
Re #29: I was not referring to predicting ENSO, but just to the vertical structure of the tropical warming in the atmosphere that occurs during El Nino episodes. One can study an atmospheric models ability to simulate this by imposing the observed SSTs as a lower boundary condition. Predicting those SSTs is a separate issue.
I am afraid that I don’t have the time to maintain a running dialogue on various aspects of climate modeling here, but feel free to e-mail me with specific questions. Just this once:
Re #21: While I have written about the entropy budget of the atmosphere, this has little connection to the work that you refer to on maximizing entropy production (MEP). We are always on the lookout for some general principle that we could use to shortcut all of the complexity of our models. But none of the papers on MEP come close to being convincing as providing a basis for such a principle.
Re #28: I am going to refrain from discussing the hurricane modeling paper that we just posted on the web. We are just starting the process of asking our colleagues for comments, and I am not the senior author in any case. But I am encouraged by the result, which makes me more optimistic that models like this will help clarify climate change/hurricane links in the next few years.
As for your comment about the ability of models to simulate ocean heat uptake, I am afraid that I do not agree with you. It may seem counterintuitive to you that we do not need to simulate the energy balance of an unperturbed (equilibrated) climate to less that a few tenths of a W/m2 in order to study the evolution of perturbations of this magnitude away from the equilibrated state, but just think of the ougoing radiation as a linear function of temperature, A + B(T-T0), so that A might be 240 W/m2 and B about 2W/(m2-K), with T0 = the equilibrated temperature, and that (this is the key point) the fractional uncertainty in A is comparable to the fractional uncertainty in B. A 5% error in A creates a drift in the climate of more than 5K, which would not be acceptable, in which case one might want to tune the value of A to create a useable model in lieu of going back to the drawing board. But a 5% change in B (from 2.0 to 2.1), which controls the evolution of a perturbation away from equilibrium, results in a rather trivial change in that evolution. This is basically the situation in all climate models.
Re #31: On reanalysis water vapor trends, see, for example, Soden and Schroeder, J. Climate, 13, October 2000. More generally, you can google “radiosonde water vapor trends” or just water vapor trends.
#32
However, in Hoyos et al. no trend is Tropical tropospheric humidity is found in the last 30 years, and this argument is used in this paper to infer that tropical specific humidity cannot be the cause for the increase in hurricanes cat 4 5
#20, 32. Isaac, it’s Canadian Thanksgiving weekend and I am borrowing time here and there; otherwise I’d have replied more promptly. Thank you for posting the above comments.
Posters who have expressed scientific criticisms of viewpoints expressed here have been treated civilly and comments from authors such as yourself have been welcomed. In particular, I might add that, in the one thread where I had occasion to comment on you personally, you were described as "eminent". There’s no reason to fear incivility here.
I take your point that climate modelers in private worry over fine points of model success. HOwever, as an external reader of the CCSP, I was struck by the firmness of its conclusion that several sets of observations were incorrect as opposed to the possiblity that there was a systemic flaw in the models, which was very reminiscent of the mindset described by Lahsen. Hence the comment in the thread.
CCSP does allude to the possiblity of model flaw. For example, they state:
or
In the accompanying press release, as so often in climate science, the caveats were not carried forward. For example:
In your post, you refer to thermodynamic requirements of any set of observations. Are you saying that present observations – tropical surface trends exceeding troposphere trends – are impossible thermodynmaically? If so, that is a claim that the CCSP itself did not make. They at least allowed for the possibility of the observations being correct and that the models were mis-specified – although they thought it unlikely. If you disagree with this aspect of the CCSP report, it would be worth communicating this to them as you seem to hold a stronger view than they expressed. As a matter of interest, are there any parameterizations or forcings which could be changed in directions that would go towards yielding this observed pattern?
One other possibility was not alluded to in the CCSP report, but seems to be at least logically possible. If surface and tropospheric trends are as tightly coupled as you say,
while CCSP concluded that the troposphere trends were likely to be at fault, surely it’s at least possible that the surface trends are not as high as reported. Accordingly, this data should be examined in detail. The CCSP said that data should be made available, but Jones of CRU has refused to archive his data and methodology. Analysis of details of this dataset is pretty low-tech; there are dozens of people interested in this sort of analysis but CRU’s obstruction is total. Obviously your support for getting this data and methodology in the public domain would be welcomed.
Again, thanks for dropping in.
And from the link we have:
Doesn’t this really say it clearly and tell us what the IPCC is all about. It is not as though they are hiding anything. They have a message to deliver and deliver it they will.
Re: #35
Should have been headed with Re: #12
Francois Ouellette says:
Re #34: Steve, thanks for the hospitality. No, I certainly did not mean to imply that I thought that it was thermodynamically impossible for the lower troposphere to warm more than the upper troposphere in the tropics. One can never prove anything in this business. But the models are all pretty “stiff” in this sense, and the El Nino evidence (referred to as “short time scales” in your excerpts above) supports this picture. I was just defending the idea that it might be reasonable in some cases to use models as a method of quality control on observations. In short-range numerical weather prediction this is commonplace. Forcing a model to assimilate observations that are inaccurate is a commom reason for the failure of weather forecasts. As models improve, predictions improve not only because they evolve the state of the atmosphere more accurately, but equally importantly because the predictions provide better quality control. One should not assume, as a given, that climate models are inherently incapable of beginning to play an analogous role.
Isaac: broad question: how sensative are the models to the input from instrumental results and from proxy records (in terms of the tunes)? Don’t worry, I’m not a nutter. I just want to know if Mann records for proxies and such (placid MWP) are disproven, would the models change substantially in tune and then resultant output? Or are they insensative to that? Same question wrt Jones instrumental. Although I think doubt of proxy recons is higher than that of instrumental, UHI silliness.
Dr. Held, thank you for your reply in #32. I understand your reluctance to discuss the paper at this time. However, I would appreciate an answer to my question about the model, viz:
Many thanks,
w.
#37.
That’s obviously reasonable enough as a principle – especially when the “observations” themselves are, to some extent, constructed, as is the case with satellite temperature records. But what’s sauce for the goose is sauce for the gander and the discrepancies between surface and troposphere trends surely requires that the surface data also be meticulously examined. I do not see how CRU can continue to maintain their obstructionist policty towards examination of their source data and methods or why climate scientists acquiesce in this. I’m not nearly as convinced as some readers that there will ultimately be problems with this data, but I don’t get the obstructionism.
I take your point that the models themselves tend to be stiff. Again as a query: if you were thinking of a parameterization or forcing that could be relatively plausibly modified and yield results more comparable to present observations – on the premise, however implausible, that the the observations are accurate – which parameterizations/forcings would be most likely to move things in that direction?
#40. Steve, how about NCDC?
Re#39. Willis
Parameterisations and physics are not separate issues. A parameterisation is a (simplified) model of a physical process, not just a constant that can be tuned. Perhaps this explanation given by Gavin (realclimate) is helpful:
‘In any model there are dozens of parameterisations – for the bulk surface fluxes, for the entrainment of convective plumes, for mixing etc. The parameters for these processes are generally estimated from direct observations of these small scale processes (though there is some flexibiilty due to uncertainties in the measurements, sampling issues, confounding effects etc.). The tuning for these processes is generally done at the level of that process. When the model is put together and includes all of this different physics, we evaluate the emergent properties of the model (large scale circulation, variability etc.) and see how well we do. When there are systematic problems we try and see why and learn something (hopefully). The number of parameters that are tuned to improve large scale effects are very small (in the GISS Model, maybe 3 or 4) and those are chosen to adjust for global radiative balance etc. Once the parameters are fixed, we do experiments to evaluate the variability, forced (impact of volcanoes, 20th C forcings, LGM etc. ) and internal (NAO, ENSO, MJO etc.). Again, we learn from the comparisons. The amount of data available to evaluate the models is vast – and more than we can handle, though directly assessing individual parameterisations from large scale evalulations is difficult (since it’s a complex system). Thus, most parameterisations are tested against small scale data (field experiments etc.). The problem is not so much under-constrained, as under-specified. There is generally more going on than is included within the (usually simple) parameterisation, and so parameterisation development is not really about changing some constant multiplier, but changing the underlying conception to better match the real physics.’
Re #42, gb, thank you for your comments. However, I don’t understand the distinction. You say
You are right, with an addendum. A parameterisation is a (simplified) tunable model of a physical process.
Gavin says what I just said:
There is a distinction between parameterizing and modeling physical processes. A parameterization is a heuristic approximation of a complex physical process. A physical model, on the other hand, applies known physical laws. A parameterization is “tunable”, while a physical model is not.
Now, you likely know all this, but let me go over it. The problem is that climate mostly deals with with turbulent or partially turbulent systems, from breaking waves to hurricanes. These systems are by definition non-linear, and are very poorly understood. Even the name, “non-linear dynamics”, is misleading. Stanislaw Ulam once famously remarked that this was “like defining the bulk of zoology by calling it the study of “non-elephant animals’.” Most of the subject matter in the study of fluids (both liquids and gases) is non-linear. To cope with this, we use “heuristic” or “bulk” formulas, which have been derived from trial and error to give the best fit to the data. One definition from Google defines a heuristic formula as “…a rule of thumb, generally based on expert experience or common sense rather than an underlying theory or mathematical model, that can be incorporated in a knowledge base and used to guide a problem-solving process. Most procedures used by human weather forecasters are heuristic …”
Here’s a typical bulk formula for evaporation:
Note that the formula doesn’t attempt to model the complex turbulent process going on at the air-water interface at varying wind speeds. Instead, it approximates them by using the factor (0.097 + 0.038 v ), where v is wind speed. Note also that, unlike a complete physical description of the situation, this formula is tunable by adjusting the two different parameters.
Bulk formulas like the one above are great, when we are attempting to calculate the evaporation off of a lake. In a climate model, however, they are both dangerous and deceptive.
They are deceptive because they are hidden deep within the code, and we don’t think about the large numbers of tunable approximations that are being used. And they are dangerous because once they are embedded in the code, they are easy to adjust and hard to verify.
Let’s suppose we are running our model, and we find that overall the humidity is too low. So we change the evaporation parameter a little bit, and by gosh, it balances back out, now the humidity is about right. But the problem may not have been with the evaporation at all, it may have been that the rain is overestimated. So now, we have two problems, evaporation is too high and rain is too low, but everything looks just fine. The problem is, we are adjusting the tunable parameters, not by comparing them with reality as was done to derive the evaporation formula above, but by comparing them with model results.
It is a bit specious, therefore, for Gavin to say:
Now, remember that this is the same GISS model that Gavin and Jim Hansen claimed could reproduce the global radiative balance so well that they could detect an imbalance of 0.85 ± 0.15 watts/m2 … but radiation balance is directly parameterized in their model, not calculated. They have tuned it to reproduce the temperature trend, which leads to the most outrageous claim of all.
This is the claim that if they remove one of the forcings and the model no longer fits the temperature trend, that forcing must be present in the real world. I’m sure you can see the circular logic in this one, yet this argument is made again and again.
All of this is why I asked Dr. Held about the degree of parameterization of the model. Gavin says his model has “dozens” of parameters. The GATOR GCMOM model has very few. Where does Dr. Held’s model fit in that scale?
w.
Re #43.
I do not have experience in climate modelling, but I study turbulence and its modelling. Some brief remarks (I can’t reply on everything you said).
You said
‘A physical model, on the other hand, applies known physical laws. A parameterization is “tunable”, while a physical model is not’
If you talk about a physical model for turbulence this is not true. A model for turbulence is a set of equations derived from the basic governing equations, but this set of equations contains terms that are unclosed and have to be parameterised. Such a parameterisation is based on the physical understanding we have, not just a wild guess. But in general it contains parameters. Quite often we have an idea about what the order of magnitude of these parameters is but to constrain them more precisely one compares the model results with measurements (for some basic case). In turbulence modelling it is just not possible to derive everything from first principles. Approximations have to be made and validation with measurements are necessary.
The same I think can be said about cloud modelling. It is impossible to resolve all processes in detail (for example the formation of droplets) and parameterisations of the physical processes are unavoidable. Parameterisations are thus always necessary but the development of such parameterisations does not exclude the use of physics.
#43, Willis
Thanks for you very succinct summary of the main issue with climate models, namely their use of parameterised linear models of non-linear physical processes e.g. turbulence. I particularly like your use of Stanislaw Ulam’s quote. I think the tuning of these models to reproduce what is at present a questionable (as more than adequately demonstrated by Steve, bender, yourself etc in your/their auditing of the proxy temperature reconstructions etc) past temperature variation and the subsequent claim of validity because of agreement between the two is perverse logic. To then as you have recounted remove one of the forcings so that the model no longer fits the temperature trend and so then claim that that forcing must be present in the real world is doubly perverse.
Now I’ll admit that I’m not 100% conversant with the details of the GISS model but I am extremely sceptical (justifiably so IMO given Hansen’s alarmist claims since 1988) of Gavin’s claim that “The number of parameters that are tuned to improve large scale effects are very small (in the GISS Model, maybe 3 or 4) and those are chosen to adjust for global radiative balance etc.”. Is this claim being made because only 3 to 4 tuned parameters sounds more credible better than say 50 tuned parameters just like the claim of no more use of flux adjustments is intended to give the models more credibility?
KevinUK
I’ve been wondering where to post this opinion and this entry seems to be a good enough place.
I often think about why I am skeptical of the so-called “consensus” on AGW. There is no doubt that there are some people in the game who are just in it for the money/publicity/self-flagellation. However, in my experience, that’s true of virtually every profession or calling.
In the end, I’ve come to conclude that my skepticism derives from the sheer certainty and fervour with which the AGW-argument is propounded. “The consensus is…”; “The argument is over…”; “There is no denying…” This entire refrain is religious – not scientific.
This is not to say that I think that (by and large) dendrochronologists and others of their ilk are not doing their best to reconstruct past climate and to make future projections. The majority are probably truly concerned about what their studies show.
The fundamental problem (and there is no delicate way to put this) is that those who recontruct past and predict future scientists see themselves as “hard” scientists, whereas they are no more scientists than an historian, or economist, or meteorologist.
They are “social scientists”, in the same vein as economists – it’s just that many have convinced themselves (and the public at large) that they are “pure” scientists, in the way that mathematicians, biologists and physicists are.
Nobody has managed to predict the winner of next year’s Grand National; nobody has managed to predict what the level of the stockmarket in 12 months time; nobody has managed to predict a country’s weather 7 days in advance.
In fact, the physics community has had to suffer the ignominy of admitting probability distributions into their science the moment they tried to predict and detect where a particle might be a fraction of a millisecond into the future.
Indeed, I believe that the physics community’s inability to avoid the dreaded probability distribution has been responsible for all sorts of things being admitted as “science”, when they are nothing of the sort (think epidemiology, for example).
No, my problem is not with the effort to reconstruct past temperatures – my problem is it’s use (and with such certainty) as a crystal ball for the future.
#37, Isaac
Welcome to climateaudit. Like the other visitors I am also glad to read your postings on this blog. At the risk of perhaps upsetting you can I please ask you to elaborate on your quote
“One can never prove anything in this business.”
There are many of us who visit this blog who agree with you but sadly this is the problem with climate modelling. The predictions of climate models are being presented by some (Hansen et al) as PROOF of anthropogenic (CAUSED by man) global warming. My physics training has taught me to doubt predictions when they differ from measurement yet some climatologists would have me do the opposite namely believe the prediction of the model and instead work on the assumption that the measurement is faulty and needs therefore to be adjusted so that it comes into agreement with the model prediction. I think this is perverse. What do you as an eminent climatologist and climate modeller think? What are your opinions on the politicisation of climate science? Do you think it is justifiable (as people like Schneider do) for someone to distort the ‘summary for policy makers’ conclusions of an international report like the IPCC TAR?
KevinUK
#46, BradH
I’ve enjoyed you amusing post particularly the quote
“In fact, the physics community has had to suffer the ignominy of admitting probability distributions into their science the moment they tried to predict and detect where a particle might be a fraction of a millisecond into the future.”
I could recount stories of my earliest impressions of sub-atomic particle research as an under-graduate at Liverpool University in the late 70s but I won’t. Suffice to say that as a consequence, as with climate change I have my doubts about the quality of data that underpins particle physics research. I must confess that as a UK taxpayer I also have my doubts as to just what my return on investment will be for an investment of US $8 billion in the Large Hadron Collider (LHC)?
KevinUK
[snip – if you want to post a comment, please do, but no time for picking fights today]
Willis: with regard to your question about the model we use in the hurricane study, as compared to the one that you link to and claim has very few free parameters: the main difference is that the latter model uses “bin-resolved microphysics” — that is, it tries to predict the size distribution of water drops and aerosols, etc. We use bulk microphysics, which predicts only the total cloud water etc. The former is far more expensive computationally and would be impossible for us to include in our work, even if we were convinced that it was relevant (which I am not). Choices like this are the essence of modeling in all fields. Your different posts strike me as somewhat inconsistent: in #43 you worry about the use of an empirical drag law for evaporation and the “parameters” that this contains, but the odds are that the mesoscale model that you seem to like uses the same kind of parameterization of surface evaporation. The unresolved turbulence that this kind of expression tries to account for takes place on the scales of cm to m. It will never be resolved in climate models. The challenge is to give these parameterizations a firm physical grounding and to understand the sensitivity of our results to these formulations, focusing on those that really matter.
Steve: Is it possible to create a GCM that simulates a top-heavy tropical response to El Nino SSTs and a bottom-heavy response to an increase in greenhouse gases? I don’t know how to do it in a physically plausible way. If someone could construct such a model, I would be very interested.
KevinUK: As for the politicization of climate science — I have always thought of myself as drawing a sharp boundary between my work and my “political life”, but (I ask myself) why then would I try to correct what I see as unbalanced discussion of climate modeling on this blog? in any case, to the extent that I decide to contribute here it will be on climate science and not on the politics or key personalities in the public debate.
TCO: The question of model “tuning” is a complex one. But with regard to the specific question of whether the proxy reconstructions of the last millennium have any bearing on model development, I can say that the answer is emphatically no. As for our own laboratory (GFDL), we have never even run our IPCC-AR4 models with estimated forcings over this time frame. It’s a pretty computationally intensive thing to do, since you would presumably want several realizations. Once you decide on what forcings to use in the model (esp volcanic and solar) you can get a pretty good answer for the deterministic response with a simple energy balance model tuned to your model’s sensitivity and heat uptake characteristics, if all you are interested in is global mean temperature. You don’t need to run the GCM for that purpose. The long runs of the model that we do perform are more focused on analyzing the characteristics of the low frequency (decadal-centennial) intrinsic variability of the model, but this is not part of any tuning process. Interest in performing runs with estimated forcings for the last millennium focuses primarily on variability in hydrology and the possibility that proxies might provide some hoops for the model to jump through in that regard.
#50, Isaac
Firstly I don’t know you and you don’t know me but thank for at least answering one of my questions which sadly Steve Bloom hasn’t done despite repeated reminders.
For others reading this blog may I please recommend reading the link in the headline at the top of this thread labelled ‘sociological study’. After reading it all (as I have done today but be warned that unfortunately it contains a lot of repetition within it – but then again so does the message of AGW’) then read Isaac’s reply to TCO and then try to work out which ‘Modeler’ Isaac could be (clue: what is a realization?)?
Isaac, you’ve piqued my interest by mentioning your ‘political life’. I think that we’ll all admit that sadly we all have politics in our lives – we are after all, all human beings and where there are human beings there will be politics. Fortunately the job that I do (I’ve mentioned what I do elsewhere) doesn’t involve politicians any longer (not for the moment anyway) and it is, as a result, better for it. There is a price to pay of course for lack of politicians i.e. reduced funding but you can’t have your cake and eat it so the saying goes. At the risk of getting sacked, I am however content to point out that the politicians involved were one and the same politicians involved in the Kyoto Treaty (that’s a big enough clue). The problem with being involved with a politicised industry (like climatology) is that the politicians have their own agenda. In the case of climate change in the UK we have Sir Crispin Tickell and Maggie Thatcher to thank for starting it all off. In your case (US), I’m not sure which politicians kicke dit off to suit their own ends but the sad reality is that as sure as eggs are eggs if politicians get involved then real science will take a back seat and junk science will come to the fore. From personal experience once they are no longer involved then junk science will fade away and real science will once again re-emerge. Dare I say it, the trains might even start to run on time and people will no longer be killed in avoidable train accidents (I may live to regret that last statement). I don’t know whether this will ever happen in climate science but for sure I certainly hope it does sooner rather than later.
KevinUK
Re #32 isaac held writes
This may be true if we are simply trying to gain an insight into the physics but these uncertainties are much greater than the “signal”, i.e. radiative forcing by CO2 itself. Surely it must mean that the model has no predictive power at all. It is these non-physical fudges that concern me rather than the necessary parameterization of sub-grid scale processes. If a model has to be tuned like this in order to be “usable” then surely it is worthless as a predictor.
#52, John R
The sad reality is that these models don’t give any ‘insight into the physics’. Please read the link in the headline and you’ll know what I mean by this statement. Computer models are exactly that models. They are not reality. They ARE NOT ‘realizations’. Sadly it seems that rather than carry out expensive real experiments we have bred a generation of ‘scientist’ who think that computer model ‘experiments’ represent reality. Nothing will ever be further from the truth.
KevinUK
I certainly hope that we would find it appropriate that someone summarize this discussion and, for that matter, in context with the lead-in article. It would be of interest to include the contribution of Dr Held to the discussion in the summary and perhaps opinions of what was learned from it.
I would like to throw my amateur view out for validation and correction. I see two possible approaches to modeling the climate starting with first principles or an entirely empirical approach. It appears to me that the chosen approach is that using first principles in combination with parameters and at least in earlier version fluxes to keep the models from drifting out of control.
The physics behind the parameters can evidently be a well understood and tested empirical relationship which appears a next best substitute for an entirely first principle approach. The inherent ability to tune these parameters seems a point of contention in this discussion. Most observers in this discussion, I am guessing, would find much less objection to their use if they were used with an a priori reasoning and not tuned to better match final results for validation after the fact. I am assuming that when the tuning is done it must meet some independently established reasons for making the changes and that it cannot happen arbitrarily. The use of fluxes would appear to me to be more problematic but I understand that there use has been significantly reduced or eliminated in the latest models.
What I have not gotten out of this discussion is any better understanding of the modeler awareness of the potential to over-fit the models by tuning the parameters and the precautions taken to avoid this practice. What I think I got from Dr. Held was that the models are “stiff” in nature and thus less amenable to tuning. And of course the big issue here is how or whether practical quantitative confidence limits or uncertainties can be placed on the outputs and predictions of the climate models.
After reading the lead-in article I get the idea that the only reasonably confident way of validating the models’ predictions will be with out-of-sample results (which has it own set of problems). Of course then we are in conflict with the claims that the next generations of climate models are better and therefore we need to restart our out-of-sample time in order to test the newer versions.
Dr. Held, thank you very kindly for your answer. You say:
This is a very good statement of the challenge, particularly because far too often the parameters seem to be adjusted on an ad-hoc basis, with an eye on the model results rather than on the physic of the situation.
w.
Re #49, I’m not picking a fight anymore than Brad was, unless trying to post another pov be that? But, since I’ve been snipped, people wont be able to judge. will they 😉
I see on this thread and others references to “insight into the physics”. Why should a climate model be based on physics? We all ( I think) believe that physics dictates the properties of single atoms, but when we study the interactions of many atoms we have a new discipline called “chemistry”. When we study a certain class of very complicated molecules, we have a new discipline called “biology”. When the number and complexity of physical interactions is too great, we cannot model the physics of the system and we must look for other consistent behavioral structure. Obviously, each theory must be consistent with physics, but it is obvious that theories of complex systems cannot be derived from first physical principles. As an example, in chemistry, thermodynamics (which explicity acknowledges complexity) can predict some vague generalizations about a vat full of chemicals based on its temperature and pressure, but cannot predict the reactions that take place. Some complex systems can be studied by using controlled experiments, but global climate, seemingly, cannot. Can anyone step back from the highly developed models and discuss the complexity of climate, how a model deals with this complexity and what we can expect from a model?
Re #43
I agree with this assessment. It may help to distinguish between free and fixed parameters. Parameters that are “fixed” are heavily constrained by results of physical experimentation. There’s not a lot of room to adjust these parameters. Parameters that are “free” are relatively unconstrained due to a lack of supporting experimental data. These are primary targets of tuning efforts. All parameters are “tunable” to some degree, but the free parameters are more justifiably tunable than the fixed parameters.
I argue – in contrast to Gavin Schmidt in #42 – that the problem of tuning is under-constrained because there are too many degrees of freedom from too many free parameters being available for arbitrary parameterizations. For example, can it be proven than a particular parameterization is optimal? Given the number of parameters and interacting processes I doubt it. Schmidt suggests the problem is one of under-specification. Hmm, well if the climate system models were any more detailed (increased specification) then this would worsen the problem of under-constraint; you would be including ever more processes for which you have no data to fix the free parameters.
I take this comment away from Dr. Held’s participation here as most important to those with skeptical views of the capability of climate models and the potential over fitting the model. While Dr. Held has replied to many specific questions and a number related to the underlying physics of the climate models, should not someone be putting the general questions to him about the precautions that are taken to avoid over fitting with parameters, the use of fluxes in the later versions of climate models and how is the determination of uncertainty and confidence limits of the results (predictions) handled.
I strongly recommend the readers here to read Shackley et al.’s paper. These authors find a distinction between “purist” and “pragmatist” modelers:
About parameter tuning, they cite one modeler:
Also:
Interesting paradox!
The paper has a very interesting discussion on the pros and cons of flux adjustments. Among other things, flux adjustments reduces the computer time, so it is more readily used by the “pragmatists”. But flux adjustment relies on assumptions that are not necessarily based on solid grounds, e.g. it assumes linearity, whereas the correction is often as large as the flux itself.
Isaac: I just took a look at your very interesting and compelling manuscript “Robust Responses of the Hydrological Cycle to Global Warming,” [Held and Soden, 2006]. This is exactly the sort of approach that many of us think would be really useful: A low-order model, based on direct application of established physics without free parameters. Nice.
I do have a quibble, or perhaps just a question, about semantics: Do you really believe that [o. 15]:
My sense is that even for relatively large basins, floods are usually short-duration hydrological events typically resulting from a confluence of “high-frequency” meteorological events. The right-hand tail corresponding to low-frequency variability in the flow field would more likely be described as “wet conditions” — not the extreme hazard connoted by “flood”. The point, I guess, is that while droughts can result from persistent lower-than-average flows — say a 20 percent reduction — floods are typically orders of magnitude larger than mean flows, and therefore it seems unlikely that a small shift in the mean would even be noticeable with respect to floods.
Does it matter? Well, because words convey meaning, and “wet conditions” (if that is an accurate description) is distinctly less alarming than “flood,” perhaps it does.
Re: #60
Francois O that link in #60 is the same as the one you listed in #12. The paper as you noted before is dated (1999) but the comments were fascinating to me. I would like to hear Dr. Held’s views on the basic issues covered in this article/survey and get an updated version of the state of the art (science) with the current climate modeling.
I did copy some comments from the article that I would like to see discussed here, but first I would say lets hear what Dr. Held has to say.
#50. Isaac, a couple of points.
First, your reply re-characterized my original question to link it to CO2 in a way that differs from the questio. I had said:
In reply, you stated:
With respect, my question had no connection to a “bottom-heavy response to an increase in GHG”. Indeed, I have no personal doubt that the impact of increased GHGs would work from the top down. I was really thinking more in terms of clouds or something like that.
Given, your answer, I bring the following paragraph from Emanuel (2005) much discussed in hurricane topics:
Despite your assertions that models preclude the possiblity of atmospheric trends being less than SST trends, Emanuel uses this supposedly impossible situation in his explanation of PDI increases in the above paragraph. I presume that you disagree with Emanuel here.
Re TAC (#61): thanks — I like that paper myself. Now that we have a really nice database of global model results, thanks to PCMDI and the IPCC, a lot of people are starting to do this kind of thing i.e, isolate robust aspects of the model responses and offer simple plausible physical explanations for these responses. It is interesting to look at the list of diagnostic studies underway with this database at http://www-pcmdi.llnl.gov/ipcc/diagnostic_subprojects.php .
Re Francois (#60) I have just read the paper by Shakley et al on flux adjustments. I was not aware of it, and found it quite reasonable and well written. The distinction between purists and pragmatists is a useful one, although, of course, this is a spectrum and not a black and white separation. But I am not sure that there is anything distinctive about climate modeling in this respect.
I would argue, as do some of the modelers interviewed, that there is no sharp distinction between flux adjustments and “tuning” of other sorts. Both are attempts by “pragmatists” to get a model that has potential relevance, one hopes, to the problem at hand. My personal preference for tuning is that the latter results in a model that is a testable hypothesis for how the climate system behaves, while a flux adjusted model is not. Anyway, I don’t think there is any point in focusing on the flux adjustment issue in isolation.
I do want to comment on the tuning question. I haven’t the time right now — I will get back to this in a day or two.
Re Steve (#63) Perhaps the moral is that we should be careful what we wish for. My point, to repeat, is just that we need to discuss the trend and the response to ENSO together. To me, the fact that the models get the latter right is significant. I don’t think I have anything else of substance to say about this issue.
#64. I don’t see what "wishing" has to do with understanding things. I take it then that you reject what Emanuel said as being inconsistent with models.
#64 (#61) I am still curious about why the word “flood” (here defined as inland fresh-water; not coastal or closed-lake) shows up so often in discussions about AGW. Given the enormous variability in hydrology among basins (e.g. annual rainfall varies by a couple of orders of magnitude, with single-day events that exceed at-site annual means) it seems hard to believe that the distribution of future floods in fully developed or undeveloped basins will be much different from what we see now. Other factors — for example, land-use change (i.e. unwise development), which can double the magnitude of the 1-percent-exceedance event (see Konrad) and greatly increase losses by placing more property at risk (see Pielke Jr.) — clearly matter, but is the prospect of AGW really that important with respect to inland riverine flooding?
I once worked in a hydrodynamic modelling group. We were “consultants to consultants” to the offshore oil and gas industry. My job was to tailor numerical models to the bathymetry, tide and wind environment at specified locations. The client would run each model with a variety of wind-field and tidal inputs in order to provide a worst case scenario of sub-surface currents which would be used in turn by engineers to design an oil/natural gas production platform at that location. All the work concerned the North West Shelf of Western Australia which is similar to the Gulf of Mexico but with massive tides as well as hurricanes.
The models we provided were always stringently tested against real currents measured by current meters in known wind fields and temperature profiles. If a model did not predict the measured currents accurately it was rejected by the client. We had to be sure. People’s lives and hundreds of millions of dollars depended on getting it right.
I have just read Chapter 8, Model Evaluation of IPCC TAR. The point I want to make is that they do not “get it right”. They do not even come close to getting it right.
17 of the 34 OAGCM models evaluated used flux corrections. Let us be clear about this. Flux corrections are not about how the model is parameterised. Certainly quantities like diffusion and surface roughness need to be parameterised for coarse grid models but flux corrections do not fall into this category. They are fudges. They are arbitrary quantities for which there is no physical justification. They are added after the event to force the model output to resemble real world observations. If a child did this in a math test we would say he was cheating. Flux corrections are a form of cheating. Any model which needs to use flux corrections can immediately be dismissed as inadequate.
The authors of Chapter 8 appear to exhibit a complete ignorance of statistical reasoning and scientific method. Nowhere are confidence limits used. There is no discussion of a null hypothesis. They purport to use statistics by comparing the variance of different model ensembles with one another and with the real world. This does not imply that the models represent the real world. It only show that the models behave similarly to the real world. Of course they do. Had they not done so they would have been tweaked until they did or they would not have been published at all. It does not mean that the models are homologous to the real world or that they have any predictive power.
If the authors used accepted statistical methods to evaluate each model they would proceed as follows: Firstly specify a null hypothesis such as: “the model accurately predicts the real world and any differences between model predictions and real world observations are due to chance”. Secondly use ensemble methods to calculate the probability that these differences are indeed due to chance. Thirdly compare this probability with some specified threshold. If it is less than the threshold then the model is dismissed as being wrong.
I believe that the authors are in fact well aware of this methodology. They give themselves away in the introduction to Chapter 8 where they state
In other words we are not going to do the statistics properly because it doesn’t give us the answers we want. Eyeballing the diagrams in Chapter 8 which summarise differences between model output and real world temperatures shows that in every case there are significant hot spots or cold spots which, I believe, would preclude the model from passing any proper statistical evaluation. The “task of evaluation” is in fact quite easy: all the models fail. The conclusion must be that there are aspects of the world’s climate system that have not been properly accounted for in the models. This is hardly surprising; it is a very complex system and, as the failure of the models shows, it is not yet completely understood.
In many ways OAGCM climate modelling resembles astrology. They both make predictions using a complex and arcane methodology. They both deliver outcomes which resemble the real world in their character. They are both eager to exaggerate their successes and ignore their failures. Neither of them is the outcome of the scientific method.
#67 John, thank you for making the point that:
Moreover, modeling does not need to be done this way. For example, take a look at Isaac’s manuscript “Robust Responses of the Hydrological Cycle to Global Warming,” [Held and Soden, 2006] (here). No free parameters! Personally, I found it both refreshing and reassuring.
Which raises a question: If all the “fudge” were eliminated, what would the OAGCM models (and all the others) report? Would the differences just be quantitative (i.e. greater, but unstructured, discrepancy between model results and observations)? Or would new, interesting, “artifacts” of the models emerge?
re: #68 TAC,
I decided to read the Held & Soden preprint you link but I’m not sure why you say there are not free parameters. Consider:
This from early in the paper means that we already have an important parameter fixed higher than most of us skeptics are willing to accept. This means the results may be of interest if you’re already a warmer, but otherwise they’re not that useful. Further, though I’ve just started through the paper, they say that the starting material are runs of the various models used by the IPCC. We already know there are things all these models have in common, like the positive cloud feedbacks, which we don’t accept, so why should we accept anything which they all have in common?
More, perhaps, later.
RE: #67 – “17 of the 34 OAGCM models evaluated used flux corrections.”
This is shocking. Even I am taken aback by this. I would say, IPCC and all feeder processes into it, needs to be audited much more thoroughly than the rather modest actions taken until now.
#69 Dave, I see your point and agree with you. It was the hydrologic response model (“fixed flow-fixed relative humidity response”) that impressed me, both for its simplicity and its lack of fitted parameters. But you’re absolutely right about the GCM underneath.
#64, Isaac,
I’m interested in your quote
“Now that we have a really nice database of global model results”
Does this imply that you consider the output from GCMs simulations to be data? If so, why? can you explain how you consider the output of a computer model be data? Because it’s used as input into other computer models e.g. economical forcasts of the effects of global warming etc? To my mind, in the context of computer model simulations there can only be two types of data:
Input data: e.g. initial conditions, physical constants, coefficients for physical property functions e.g. for density of CO2 as a function of temperature/pressure etc
Transient (forcing) data: data which varies as a user defined function of time that ‘forces’ a change to the steady state e.g. the speed of rundown in the gas circulators in a gas-cooled reactor etc.
Output from a computer simulation is NOT data (even if it is input into another model), it is a result (largely determined by the initial steady state and the transient forcing conditions) from the simulation. Similarly any perturbation to the input and/or transient data only provides an indication (not proof) as to the sensitivity which the computer model (a combination of equations) has to this input/forcing parameter. The variation in the output (e.g. maximum post trip fuel temperature reached) as a function of a perturbation in an input/forcing parameter is NOT a measure of the uncertainty of the model.
The combining of the difference in predictions between several ‘similar’ computer models (ala IPCC TAR style) for a given defined scenario (initial state and defined forcings) is NOT a measure of the uncertainty of the output result e.g. predicted global mean surface temperature over a 100 years. IMO it is statistically invalid to combine the outputs of computer simulations in this way. IMO the IPCC only does this so that it can claim a ‘consensus’ and avoid possible disputes between different national climate modelling communities throughout the world which would otherwise undermine this claimed ‘consensus’.
What do you think?
KevinUK
#84, Peter H
Thank you for your reasonable reply to #84 and I apologise if my reply was worded in a belligerent way.
I agree with you. I’ve posted about this previously, namely that the UK nuclear industry was founded on a lie in order to justify the UK tax payers funding of it. That all changed with the ‘dash for gas’ the best example of which I can give is that BNFL built a combined heat and power (CHP) at Sellafield (Fellside CHP plant) to provide steam for processing heating and electricity for THORP as well as surplus electricity to the national grid. If they actually believed in the econimics of nuclear power generation then they would have built a NPP. The fossil-fuel levies as they were called back then have now been re-directed into the renewables industry to enrich the likes of John Selwyn Gummer who a cabinet minister (with MAFF portfolio) under Maggie T was involved in the whole ‘dash for gas’ decision making process. One of the reasons why I post on ‘nuclear stuff’ on this blog to to hopefully ensure that any resurgence of nuclear power generation in theUK isn’t based on another lie, namely AGW.
Other than the fact that its definitely off thread, I also agree with you that its not worth discussing the CAP.
On the subject of T(im)L(ambert), I’ve post this before but I have him to thank for getting me onto the subject of global warming in the first place. As I suspect you already know I am a big fan of John Brignell’s NumberWatch web site. Up until seeing a criticism of John B (he called him a crank and wouldn’t justify why) I hadn’t taken the time to get seriously into the whole AGW debate. After reading the thread on his site, I got far more interested as IMO (perhaps not yours) I think a lot of what JB posts on Numberwatch is bang on the mark. For example read this months (October) Numberwatch particularly the bit about the BSE fiasco. If TL hadn’t called him a ‘crank’ there’s a fair chance I wouldn’t have put the effort in (nor have persuaded a lot of my friends to do likewise), hence my thank you to TL.
KevinUK
Re 72
KevinUK I agree with just about everything you say on this except maybe the following:
I don’t believe that there is any sort of conspiracy or that there is anything particularly sinister about the IPCC. I checked it out on the web and it looks very similar to the dozens of other international science bodies that non-scientist seldom even hear about. Such bodies usualy pride themselves on their even-handed internationalism.
I think the problem lies with the peer review system. Scientists tend to form themselves into little clubs around their own specialty. Each club has its own culture which defines what is acceptable research methodology and what is not. They even have their own vocabularies, e.g. principal components are called “empirical orthogonal functions” by oceanographers and surface wave specialists talk about the “celerity” of a wave rather than its velocity.
The trouble is these cultures tend drift apart and their adherents lose contact with other disciplines. This is particularly true of statistical methods. Each discipline has its own ideas about what constitutes statistical evidence and these can be at odds with the methods used by real statisticians. Many climate scientists have a background in applied mathematics which is often an alternative to formal statistics in undergraduate courses. Consequently they tend to “wing it” when it comes to stats and throw around terms like “variance” without really understanding the underlying concept of hypothesis testing. Their isolationism leads to dysfunction. IPCC TAR Chapter 8 is a classic example of this dysfunction.
Isaac’s GFDL home page תולעי×? is great! thanks!