David Letterman sometimes has a segment entitled “Stupid Pet Tricks”, which is an apt title for today’s post – more parsing of Mannian CPS, recently discussed here. With helpful contributions from Jean S, UC and Roman M, I can now pretty much replicate Mannian CPS, but only through a variety of devices that fall into the category of stupid pet tricks. I’m not sure which is stupider – the original pet trick or spending the time trying to figure out what’s going on. Probably the latter.
I realize that this isn’t the most interesting material in the world, especially compared to talking about U.S. politics, but this takes a lot of time to do and I find it useful to document these results while they’re fresh.
Smoothing
The first stupid pet trick is Mannian smoothing. With particular thanks here to Roman M, we now have an exact replication of Mann’s smoothing method in R. Mannian smoothing has been described in two different articles in GRL (see references below); the algorithm as used in Mann et al 2008 differs somewhat from either.
I suspect that we are the first people – including the reviewers of Mann’s two GRL articles – who have actually figured out what Mann’s smoothing method actually does. The real issue is whether the world needs Mannian smoothing – there are standard smoothing algorithms developed by statisticians (e.g. lowess).
And why GRL is publishing articles on smoothing methodology? In my opinion, if Mann (or me or anyone else) has something to contribute to smoothing literature, it should be submitted to a statistical journal where it can be properly reviewed. Review by a couple of climate scientists at GRL is worse than pointless as it gives faux authenticity to the method.
The tricky thing about Mannian smoothing is, as noted before, the use of butterworth filters. These may be meritorious in sound engineering where you are, say, trying to recover music which is governed by frequency. But these methods seem less apt when you are dealing with red noisy time series, not least because they have strong mean-reverting properties at their endpoints.
Because Mann doesn’t even bother centering the series, this leads to very odd endpoint behavior, which he tries to patch with endpoint padding of ever increasing length. The most recent aspect of this method winkled out by Roman M. is that Mann’s mean padding is the mean of the entire series, rather than the mean of the last M (say M=10) values, the method used in IPCC padding.
Here’s a plot comparing our emulation of Mannian smoothing for Dongge to a result extracted by UC using Mann’s source code. The validity is further confirmed by the ability to replicate other results, as shown below.
Counting more than once
The next stupid pet trick relates to double-counting and even quadruple-counting of certain proxies, a point already identified (the issue was first noted by Jean S, who observed the identity of a couple of Mannian gridcells.)
If the Mannian lat-longs for a proxy (which may well be in error) are on the border of a cell e.g. exactly 50E as with the (incorrect) Socotra location, Mann places the proxy in both adjacent cells. A proxy located precisely at a 4-corner is allocated to all 4 gridcells.
It took a while to figure out this stupid pet trick, even with the code. This got most gridcells right but there were further traps for the unwary.
Wrong locations
The next stupid pet trick wasn’t easy to find. I eventually tracked the difference between the allocations that I was calculating and Mann’s results to gridcells where the latitude was at the north of a gridcell.
UC had recovered a Mann file showing the latitudes for gridcell centers that Mann had calculated – these ran from -88.5 to +86.5 (S to N), rather than the correct -87.5 to 87.5 (S to N). So a proxy located in the top 1 degree of a gridcell got assigned to the wrong gridcell.
This goof can be 100% confirmed in one of the Matlab runs that UC sent to me. Re-doing with (Mannian incorrect) gridcell centers, I was able to match the AD1000 selections exactly.
Smoothing over and over
The next stupid pet trick is that the proxy data are smoothed over and over again in all 19 steps. On each occasion (e.g. AD1000 which we’ve been studying), the proxy data are truncated to AD1000 and then padded at their beginning even if there are actual values.
Mannian smoothing is not especially efficient in the first place, so, aside from using padded values when known values are available, the procedure adds to the overheads. It would be perfectly feasible to smooth the master proxy data set once and select from this.
In any event, for a given step (e.g. 1000 here), Mann truncates the data to start at AD1000 and selects the subset of series with “passing” correlations. The smoothed series are all given short-segment standardization on 1850-1995 – note that this period is a little different than the 1855-1995 used in correlations – and then averaged. The average is then re-scaled to the mean and standard deviation of the instrumental gridcell over 1850-1995.
Here is the comparison of the emulation to the series (that UC extracted from Mann’s Matlab code) for the gridcell containing the Dongge O18 series. A perfect emulation.
You’ll notice that, unlike the Socotra O18 speleothem series, the Dongge O18 speleothem series has been flipped over, a point that I’ll return to in a separate post. (Gavin Schmidt excoriated Loehle for using Mangini’s O18 speleothem series where the O18 series had been flipped over, but, for some reason, failed to criticize Mann for flipping over the Dongge O18 series.)
This example only had one contributing proxy, but the emulation in a gridcell with multiple proxies is also accurate, as shown below for the gridcell containing the 4 Tiljander proxies (which Mann inverted from the usage of the original author, who warned against non-climate anthropogenic effects in the latter part of the series.)
After making these gridded series, Mann then re-grids all series N of 30N into 15×15 gridcells, weighting the two gridcells by their cos latitude. This yields 15 regrid-regridcells with contributions.
I’ve compared all 15 to versions extracted by UC and the emulations are mostly exact, with the “worst” emulation being the one below combining the Tiljander and Tornetrask gridcells, affected only by very slight rounding.
Fat files
While I’m thinking about it, another stupid pet trick are the huge Matlab files that Mann created in the CPS program. The gridded series for the 9 steps from AD200 to AD1000 were over 250 MB in size. In the AD1000 step, 2567 out of 2592 columns were empty. In addition, all values prior to AD1000 were empty (as used).
Adding to the absurdity, Mann created a huge 3-dimensional matrix of these series. I guess they simply buy more and more powerful work stations to perform stupid pet tricks.
If you simply kept the portion of each portion that’s used, naming each column to keep track of it, and keep the information from each step in a list rather than a 3-D matrix, the same information took about 1MB in R. So the handling of the information became effortless rather than a strain. This absurd data handling was repeated time after time.
I’ll finish off this post through to the NH reconstruction in a little while.
Smoothing References:
Mann, M. E. 2004. On smoothing potentially non-stationary climate time series. Geophys. Res. Lett 31: 710713. —. 2008. Smoothing of climate time series revisited. Geophys. Res. Lett 35. http://holocene.meteo.psu.edu/shared/articles/MannGRL08.pdf
Soon, W. H. S., D. R. Legates, and S. L. Baliunas. 2004. Estimation and representation of long-term (>>> 40 year) trends of Northern-Hemisphere-gridded surface temperature: A note of caution. Geophys. Res. Lett 31: L03209. http://www.cfa.harvard.edu/~wsoon/myownPapers-d/SLB-GRL04-NHtempTrend.pdf
![Reblog this post [with Zemanta]](https://i0.wp.com/img.zemanta.com/reblog_e.png)
Climate Audit 
101 Comments
I’ll post up exact code after checking for turnkey-ness.
As a trained physicist, Mann is far better trained in math than virtually any of his colleagues in proxy climatology. The latter, at best, are adepts of the methodology described by Fritts in “Tree rings and climate.” Mann has taken the full course in numerical methods, preceded by the full range of mathematics likely through tensor algebra.
Mann hasn’t done the same analysis twice. Every new paper has some weird new method, incompletely and obscurely described. As you deconvolve his various methods, Steve, I get the distinct impression that Mann invents various complicated approaches to give the impression of deep scholarship, on one hand, and make the method so bewildering (intimidating) as to discourage any attempt at replication. The combination lubricates passage through peer review, and produces an almost unquestioned acceptance of verity. We see this in the very widespread acquiescence to his reconstructions by the scientific community at large.
I suspect he never expected someone like you to show up, and successfully comb through his methods. Do you get the impression of a studied and deliberate obscurity, not just in description, but also in choice of methods?
Re: Pat Frank (#2),
Mannian use of a wide variety of tools in his analyses with some being obscure for the purpose, in my mind, is a form, not of data snooping, but of something similar that could be called methods snooping. If this is done ex post, as the obscurant methods imply, then it is definitely snooping and a snooping that needs at least to be tested by some sensitivity analyses even if it cannot be statistically adjusted for the snooping.
Using, ex ante, tried and true methods that have wide acceptance would certainly avoid this impression. Maybe even not doing a different analysis of the same proxy data for each study, but using completely new data would at least put the methods snooping to an out-of-sample test. Of course, as long as the answers keep coming out “right” who is going care about method –even though it is my distinct impression that the latest Mann 2008 version is far less “right’ than its predecessors.
Re: Pat Frank (#2),
I was going to ask Steve this exact question but I realized it goes to motive so asking Steve to comment is asking him to look into Mann’s motivations. Then again Steve is perhaps somewhat qualified to do so having made Mann’s work, and hence Mann’s mind, the nearly fulltime subject of his efforts for years.
Having read this blog daily for over three years I often pity Mann. It must be intimidating to know that a whole blog community is waiting to scrutinize your work. Then again a good scientist would welcome such scrutiny. (Craig Loehle comes to mind.)
Here’s a short script showing the erroneous latitude grid-centers:
This shows that Mann’s latitudes go from 88.5S to 86.5N instead of being centered on 87.5S to 87.5N. Is this an “error” or does it “matter”? No more than any other stupid pet trick.
Matlab has excellent support for sparse matrices that allow conventional (FORTRAN-style) indexing and operations without wasting space storing zeros. So add trained programmers (who RFM) to your list of things to get Mann for Christmas.
I’ve never really played with MATLAB’s sparse matrix functions (no need). Does it make a speed difference to perform operations using the sparse representation over the full representation?
Mark
Re: Mark T. (#6), IT can make a very big difference in performance if you can use the sparse matrices in Matlab. However, not all of the Matlab functions support sparse matrices as arguments.
This is what is known to programmers as a “fence post error”. It’s where you accidentally count the ends when you shouldn’t or don’t when you should etc.
Normally a programmer would be pretty careful about this sort of thing – I would be anyway. For example you might write “if( a >= b && a < c)” in order to avoid including a in both the range “b to c” and “c to d”.
It’s not a surprising trap for an amateur programmer to fall into but you’d think with the amount of attention these studies get they’d use professionals or at least experienced programmers.
Re: Nicholas (#7),
I am not sure I agree with this assessment. Admittedly, I could not find the code snippet from JeanS but using a boundary station in 2 or 4 grid cells averages seems perfectly reasonable as described here. Can you revisit your original analysis and describe why you think this is an error?
Assuming that I am wrong and it is reasonable assert that the measurement describes only one of the bordering cells– is the error really OBO?
Is it possible this double counting of proxies is the means by which Mann’s program implements his proxy teleconnection theories?
Steve: No, I don’t think that it “matters” that much in the sense that you intend. That’s why I call it a “stupid pet trick”, as opposed to a real serious problem. Using the Tiljander data upside down is more serious.
Re: Scott Brim (#8),
Possible yes. Mann’s explanation of the “teleconnection” between certain trees in Colorado and the Global Mean Temperature however remains firmly in the realm of “magic”
Re: John A (#12), wrote, “Mann’s explanation of the “teleconnection” between certain trees in Colorado and the Global Mean Temperature however remains firmly in the realm of “magic””
It’s a kind of ‘proxydruidology,’ isn’t it, in which tree ring series teleconnect through the trees to their familiar gods, who have clearly mucked with the weather elsewhere.
Steve Mc:
Bingo! Add to that Mann’s rant against the R2 statistical test when his previous outings failed that test so he writes an article in a climate science journal decrying it and claiming that no true climate scientist should have anything to do with it.
Its like the Hockey Team are creating their own separate branches of mathematics, physics and logic, together with their own unique take on the scientific method, and some extremely gullible journals and journalists are letting them.
For some reason I find myself thinking of the novel economic tests produced by promoters of dotcoms during the boom in the late 1990s. Things like “eyeballs”, “stickiness” and “rankings” to replace old economy boondoggles of “profitability” and “return on investment”.
Mann is in the vanguard of creating a self-sustaining bubble in science, but it will only last as long as the general market in climate science is still invested into by gullible politicians.
I believe Letterman has a spin off from Stupid Pet Tricks called Stupid Human Tricks, but then that might have been a bit too direct. Maybe just call them Mannian Tricks and leave it at that?
Steve,
When you speak of passing proxy correlations, have the series already been truncated and padded or does that occur after the selection phase? If done before it is yet one more oddness to add to the list.
Steve: Yes, the reported correlations are between infilled proxies and infilled temperature. It;s quite bizarre. It’s really quite a chore to document every bizarre aspect of this corpus.
A general question from a layman:
Am I right in considering the work of ( at least in this one instance,- and a few others ) Michael Mann as a total fabrication without any merit scientifically? If that is the case I must say I am going to become very disappointed with the scientific community for not interfering with this. How can Michael Mann´s colleques allow him to continuously harm the standing of climate science by presenting such flawed work?
Best
Hans Kelp
Steve: Little purpose is served by going a bridge too far in commentary and your question here does that. Mann’s work is what it is. What purpose is served by trying to label it a “total fabrication”. This is not language that I use – so why use it? There’s a big difference between work being “flawed” and being a “total fabrication”.
Matlab’s ‘profile’ command is sometimes useful if you want to check efficiency of the code,
gridboxcps.m spends most of the time in line
ibox=find(glat >= ila & glat = ilon & glon < ilon+15);
(regridding to 15×15)
For AD1000-present, function ‘butter’ is called 23400 times. ‘butter’ designs the same digital Butterworth filter every time. But this kind of inefficiency doesn’t matter much, the problem is that normal desktop doesn’t have memory to execute gridboxcps. ( And the real problem is double variance matching combined with Mannian smoothing )
Another question as to why “New and Improved” methods keep showing up…
Do these new methods really improve things or are they change for the sake of change? You mention that the smoothing is using Butterworth filters. Have all his smoothing algorithms used Butterworth? If so, how is this different (except in the ever increasing number of lines of code needed to do the filtering)?
I think your efforts were important and you all should, at the least, be given a nod of appreciation. The SM team has provided a means to make the HS team work more transparent and provide a firm basis for detecting any HS edits going forward and to avoid the side issues of not correctly applying a method – no matter the correctness of the method.
UC, the other aspect to the inefficiency is that it exemplifies a lack of understanding of what their algorithm is doing. As Mann wrote it, his algorithm cannot be executed on a desktop. My emulation runs in nanoseconds and hardly uses any resources. The other advantage of clean code is that you can see what you’re doing. Mann’s code, even in Matlab, looks like Fortran.
The algorithmic inefficiency is pattern is familiar from MBH98 which can ultimately be reduced to a few lines of code, once the weights are established. And there, as here, the issue is not so much the algorithmic inefficiency but the lack of insight into what’s going on by the endless moving of the pea under the thimble.
Unless a well thought-out design is used in writing any complex software system using older 3rd generation languages such as Fortran, it becomes very easy to lose the forest for all the trees.
Off topic, I know.
But my curiosity on how Steve McIntyre might think or feel about Michael Crichton’s role, position or contribtion in the climate debate has simply gotten the best of me.
Perhaps you could provide a comment or two on a man who has also taken much fire from opponents.
Thanks in advance for considering.
I disagree, Steve. The pushback you’ve gotten all along tells us exactly what’s going on.
Weak correlation between proxies (even cherry picked) and local temperature implies that there is lots of uncertainty, low SNR. This problem is then swept under the carpet by inventing new calibration methods such as CPS. Conventional classical calibration would show right away that we have a problem, unprecedented warming cannot be verified with these proxies.
Steve wrote, “Mann’s work is what it is. What purpose is served by trying to label it a “total fabrication”.”
I totally disagree having watched the hijacking of climate science through most of my career. While I applaud and admire your work, it has had little or no influence on the general belief of most governments, many scientists, academics or the public. The “hockey stick” continues to be at the centre of belief that humans are causing warming and climate change.
I was given a subscription to History, a monthly magazine published by the British Broadcasting Company (BBC). The most recent issue (Vol. 9 no.10) (www.bbchistorymagazine.com) has an article by Dr Paul Parsons titled “Harbingers of Doom” (cover title “The scientists who foretold climate change.”) and there at the end of the list is the hockey stick graph and Michael Mann’s picture with the caption,”Mann led a team who put forward the “hockey stick” graph.”
snip
As you have also told us, he continues his methods in the latest attempt to resurrect the hockey stick and this after exposure of the invalidity of his methods by you and as confirmed by the Wegman report among others. The pattern of manipulations is not just the continued use of a singular method or technique, but a shifting of strategies to achieve an end result at any cost and by any means.
It may be driven by arrogance and/or unwillingness to confess either incompetence or some other motive, such as one Tolstoi so aptly explained; “I know that most men, including those at ease with problems of the greatest complexity, can seldom accept even the simplest and most obvious truth if it be such as would oblige them to admit the falsity of conclusions which they delighted in explaining to colleagues, which they have proudly taught to others, and which they have woven, thread by thread, into the fabric of their lives.” However, the responses to the criticism by the individual and his acolytes clearly show they understand the nature of the faults, but instead of acknowledging and politely thanking those who try to support by correcting problems (as discussed on other threads), they are defiant (methink he doth protest too much) and continue the behavior in modified but equally incorrect or inappropriate forms.
snip
Steve: Eeveryone, take a valium please. In particular, please discontinue editorials about the impact on policy.
#24. Folks, pay attention to UC here. I agree 1000% with this diagnosis – which is pretty much the same as our observations about MBH98-99.
I register here a protest against the snipping of Tim Ball’s comments. Dr. Ball has the standing, and has earned the climate science credentials, to make the kinds of observations he has made here, and they are completely relevant to the topic, in my humble opinion.
Steve, I think you do a significant disservice to the readership of this blog, and to your own goals for Climate Audit, by snipping the comments of someone whose authoritative observations are so obviously germane to the discussion and to the general topical interests of the CA readership.
The blog has policies that make it very clear that posters are asked not to ascribe motives to other people or to editorialize about policy (other than the occasional thread where a specific exception is made). These policies apply to Dr Ball as well. Sorry about that.
I doubt that Tim has any real problem with this. If he does, I’m sure it can be sorted out.
I do have a problem with the snip. The part removed speaks to why I disagree and its removal emasculates my contention that your work is having no real impact thus it becomes an esoteric academic and statistical exercise. The issue is sorted. It’s your blog and your rules.
Steve, you take a lot of jabs at “the Team” (that in itself is a jab), yet you don’t want us to…
Ken Fritsch, do you have aplot that shows what the correct temperature reconstruction should be, compared to the reconstruction that Mann came up with? This is not to say anything about the work that Steve is doing, but I have a cousin who is a financial auditor. He goes into companies and sees the most crazy things… Reports on napkins, people just shrugging when asked “What happened to this $100000 expense?”. But, lots of times, the final answer is correct, even with all the mistakes that go into coming up with the final answer. I’m curious if we are not conflating two different problems here. One, is Mann doing ‘good’ science? Two, is Mann’s temperature reconstruction correct? I have different motivations for wanting to know the answers to those two different questions. But, they are different questions and they both deserve an answer.
Re: oconnellc (#31),
You are no doubt unfamiliar with this blog as I am a layperson observer here who perhaps writes too much.
A temperature reconstruction, in and of itself, does not require the reconstructor to be a scientist in the area of the proxy domain. It does help, I would think, if that reconstructor did take the advice of the proxy scientists and apply as much of that knowledge a priori.
You need to listen and understand what the statistical experts say here and I can give an example from:
Re: UC (#24),
Re: oconnellc (#31),
Let me try to give you an answer to your questions on a more philosophical level. When statistical methodology is used on data with random components, it is a rare situation where an answer might ever actually be known to be “correct”. Results are stated with levels of certainty which we usual term “confidence”. That certainty is based on assumptions made about the data and the way that it was collected. The analysis of that data is based on the use of methods whose results depend very substantially on the earlier assumptions made about the data. The results are then interpreted and understood in that context.
If you change the methodology in any way, you must be prepared to justify how those changes affect the results. The calculations that went into determining the confidence levels no longer apply. Taking some arbitrary pseudo-data and running the “new” methods to show that it is “business as usual” – a “proof” often given by people who have an incomplete understanding of the analytic tools they use just doesn’t cut it in the statistical world. Without that justification, there is no genuine meaning that can be applied to the results obtained and at that point it stops being science of any kind.
So to answer your questions: Is Prof. Mann doing good science? Not even close (particularly if you add in the mathematical and statistical miscues apart from the “new “ methods). Is his temperature reconstruction “correct”? We may never know. However, since there is no meaningful interpretation of the confidence we may place in the methodology that he used, it can only be viewed as occurring by accident – by the same sort of accident that I could make by going to a computer and generating a sequence of random values that have no relationship to the real world. There is no single “correct” way of making temperature reconstructions but there are definitely many possible wrong ways of doing it.
Re: RomanM (#34),
They take this pseudo-data from simulations that support hockey stick. If someone tries to add some variance to the pseudo-data, they’ll argue that pseudo-data simulation was wrong,
(MRWA07)
Re: UC (#44),
Yes, their choice of where they get their simulation of the “real” data definitely plays a major role in getting the results for which they wish to provide a “proof by example”. Despite the fact that this sort of rationalization for the correctness of a methodology is unfounded, there is virtually never any justification that the pseudo-data has anything even resembling the properties of the real thing. If the models call this variable “temperature”, then it must be exactly like real temperature data…
Re: RomanM (#45),
Actually, this type of “proof” reminds of the (really)old joke about how different professions prove that all odd numbers greater than 1 are prime. Now, we can add a new group to the list:
Climate scientists: The models show that …
oconnellc,
Given MM’s alternative reconstruction presented using the MBH98 data, my belief is that we just don’t know. Certainly not to the degree of accuracy that nearly anyone wants to portray.
Given the documented problems with surface stations and the questions about sattelite accuracy I’m not even sure we know the current temperature as well as folks claim. At least it doesn’t make much sense to me to claim we can derive a meaningful measurement for global temps to hundredths or thousandths of a degree by averaging out a huge number of less precise readings.
The role of auditing is to ensure the verifiability and integrity of the processes and products being audited. Dr. Ball’s comments concerning the impacts of poor quality science on the choices and decisions policy makers face were completely appropriate, and in my opinion did not violate either the letter or the spirit of your blog rules. Dr. Ball’s objections to your snip are completely justified. If his last paragraph is gone, then his whole post might as well be gone too.
I speak here from the perspective of a someone who has designed and coded large-scale software systems to very demanding quality assurance standards, and who has been employed as a software QA auditor in the nuclear industry examining in detail how scientific software systems have been designed, constructed, and managed.
After seeing the kinds of issues which Steve and his team are discovering, I am becoming increasingly suspicious of two possibilities: (1) the software modules and data which were archived by Dr. Mann and which are now being analyzed by Steve and his team are not the actual software modules and the actual data which were used to perform the reconstructions; and/or (2) the output from this software was not employed directly in its raw form in reaching the final conclusions of the reconstruction study. The question then becomes, if my suspicions are correct, is the software and the data which has been archived merely serving the function of audit bait?
The fact remains that in direct contrast to how we do things in the nuclear industry, there exists no adequately documented end-to-end audit trail for Dr. Mann’s reconstruction which allows one to directly follow and to directly verify which foundational assumptions have been made, which data sets have been analyzed and/or produced, which software modules were employed to do the work, and how the information actually flowed from beginning to end through the analysis process. In other words, parts and pieces of the puzzle are there, but it is not possible to use what has been provided to verify the overall integrity of the reconstruction with any real confidence.
If we apply the same standards the nuclear industry applies to V&V of its processes and products, the answer is, “We have not yet verified with reasonable confidence that Mann is doing good science.” Therefore, for all practical purposes, the answer can only be “no” until such time as the processes and the products Mann produces have received end-to-end validation and verification using a disciplined V&V approach. Experience in this case clearly demonstrates that peer review has in no way been an adequate substitute for direct V&V.
In the world of nuclear, both the content of the analysis and the processes used to perform the analysis must be validated and verified before the products of that analysis can be accepted for further application and use. Analysis products which feed other subsequent analyses processes must be V&V-ed in sequence, otherwise the process as a whole is judged to have failed, and any products that it produces are declared “not acceptable.”
From a content perspective, let us take as an example one of the most important issues facing those who do climate reconstructions, and let us apply nuclear-grade discipline to that issue. If he is to rationally apply the reconstruction methods he uses, Dr. Mann must demonstrate up front — through proper citation of generally accepted prior research — that a strictly linear relationship exists between temperature and tree ring growth. If this assumption cannot be adequately verified by substantial documented evidence, his own or someone else’s, then all subsequent analyses which rely upon the linearity assumption fail. (How could it be otherwise?)
To my knowledge, the linearity of temperature response for the tree ring proxies being employed in Dr. Mann’s reconstructions has not yet been adequately verified by any currently available scientific research. His reconstruction fails on a very substantial content issue; i.e., a very crucial foundational assumption has not been verified.
From a process control perspective, assuming we are applying nuclear-grade process control, if we cannot verify the quality and discipline of the overall analysis process itself — separately and apart from the content of the analysis product — and if we cannot directly follow and verify the pathways through which the product was delivered, then the analysis product still remains unacceptable for further use. The adequacy of Dr. Mann’s efforts at documenting his reconstruction processes and techniques is open to serious challenge, and therefore his reconstruction fails from an adequacy-of-process-control type of perspective.
What is the practical effect here relative to Dr. Ball’s snipped comments concerning the impacts of unprofessional climate science work on public policy and the policy makers?
Steve McIntyre has said on this blog that if he were a policy maker, he would still take the advice of scientific panels such as the IPCC, even if he were aware of serious issues as to the validity of the science being used to justify the advice.
Is this a reasonable position to take once one becomes fully aware of the true nature and true extent of the serious issues now surrounding the quality and professionalism of today’s climate science?
I say that it is not reasonable to expect thinking people to blindly accept the recommendations of panels such as the IPCC while such serious questions exist as to the validity of both the climate science itself and as to the means now being employed to pursue that climate science.
Re: Scott Brim (#35), A very astute analysis, Scott. In particular the linearity assumption has not only not been verified, it is false:
Loehle, C. 2008. A Mathematical Analysis of the Divergence Problem in Dendroclimatology. Climatic Change DOI 10.1007/s10584-008-9488-8.
To carry on as if it is not important is like ignoring that people lie on surveys about their drug habits or whether they eat healthy food when doing research on those topics. When tree ring analysis was sticking to recent time periods and investigating the factors influencing tree growth (season, current vs prior years, etc), the problem was not large, but when the analysis extended back 1000 years, yikes! Audit Fail.
I ask posters not to ascribe motives to people and I attempt to adhere to this policy and I think that I do so pretty consistently. I have often snipped comments that breach this policy and I did so on this occasion. If I’ve failed to adhere to this policy on occasion – and I think that you’ll find it hard to find examples of such lapses – I’d prefer that someone point that out to me and ask me to apply blog policy to myself than to change the policy.
Likewise with exhortations that policy-makers do something or other, other than improve standards of disclosure and due diligence.
Taking “jabs” is more of an editorial thing. “Jabs” are not always an attractive aspect of my writing style and when I’m writing well, I can usually achieve the same point without jabs. However, sometimes I’m a little self-indulgent. When I do so, in my opinion, I have a little lighter touch than most commenters. Further, editorially, I don’t like it when threads descend into “piling on”. I also ask posters not to be angry and not to vent – both of which are unattractive editorially. While I’m not entirely immune, I’m pretty good about avoiding these attitudes and again would prefer that people call me out on my own posts rather than using them as excuses for poor manners themselves.
Roman, Scott, Soronel, I think we are getting somewhere, but are not yet in a common place. Personally, although I am interested in finding out of Mann is doing good science or not, it isn’t really something that I would spend a lot of time on. His employer probably cares, and in the sense that since I pay taxes, I am his employer and I do care a little. But really, I don’t care much. As a person living in the current time, I am very interested in finding out if the temperature reconstruction that he presents is accurate. In many ways I don’t care if that plot was randomly generated or accurately represents changes in a physical system. I do care if the average temperature of the planet is increasing. You could say I care about the answer to question 1, but only to the extent that it affects the answer to question 2.
Does anyone know just how much the answer to question 1 affects the answer to question 2? Steve has done a wonderful job in hunting down the inconsistencies and errors in Mann’s work, so he is answering (or trying to answer) question 1. But question 1 is only important in how it affects question 2. Are Mann’s errors significant? Are they the reports written on the back of a napkin that still accurately reflect profit and loss? Or are they the books from Enron? Steve, if I have missed something in your work, I apologize, but do you feel you can answer question 2? Are you trying to answer question 2? Do you think you will ever (if you haven’t already) answer question 2?
#37. As to errors and inconsistencies, I really wasn’t trying to “hunt” them down. I only identified them because I got stuck at various points in trying to replicate his results. I was a bit snarky in my language because I think that scientists in this field should make a greater effort to avoid this sort of stupid mistake. If they make these sorts of errors in simple programs like Mannian CPS, it is a bit disheartening to extrapolate to more complicated climate models. And it’s not as though they were unaware that this paper would be closely examined.
After I’ve figured out what he did, of course, I’ll analyze some permutations and combinations e.g. the effect of bristlecones and the Tiljander inversion but I can only do so much at one time.
As to whether the reconstruction is “correct”, no one knows or can know. I refer you to comments by Roman M and UC. In my opinion, all that one will be able to say about this stuff is that the information is too inconsistent to permit an opinion. In terms of statistical methodology, I’ll probably try to apply some of the Brown and Sundberg techniques that I was experimenting with last spring.
oconnellc,
I will echo Steve here in saying I believe the answer to your #2 to be unknowable with the information that is currently available. I believe the best that can be said is that right now is certainly warmer than say 2-4 centuries back. But trying to compare now to the 900-1100CE ore Roman warm period (not sure what span that occupies) is a losing effort, our indicators just don’t have the resolution needed for that task.
@Steve
You write “I also ask posters not to be angry and not to vent…”
If your work is exact science and the results are correct, people should speak more politely about your work. The only thing I can find is this: http://www.sourcewatch.org/index.php?title=Stephen_McIntyre.
I am writing on Stefan Rahmstorfs german blog http://www.wissenslogs.de/wblogs/blog/klimalounge/klimadaten/2008-10-24/erderwaermung-beschleunigt-sich, and I can tell you that it is sometimes not easy not to get angry. Those people seem to be rather nice, and the only reason why they behave like they do, is imho that they want to save the world.
Most climate scientists in Europe or at least in Germany don’t think that your work does mean anything for climate reconstructions or paleoclimatology.
If this is not true, the readers might have the right to be angry?!
Best regards
Eddy
Re: Eddy (Europe) (#40),
If Steve’s work is not true, then you, or they, should be able to point to papers, blog entries or other “documents” which show this, and more importantly be able to defend such refutation of Steve’s work. Fact is this has not been done. In the only cases where external entities have presented opinions, they have sided with Steve over Mann. While the NAS report put a fig leaf over Mann’s work, claiming that later research had upheld his results, if not his methods, in every case these more recent papers have the same or similar flaws.
But feel free to post here whatever you think supports your point. (You might want to find a more suitable thread to do it on, however.)
Scott Brim:
“I say that it is not reasonable to expect thinking people to blindly accept the recommendations of panels such as the IPCC while such serious questions exist as to the validity of both the climate science itself and as to the means now being employed to pursue that climate science.”
I agree. Is blind acceptance good enough for the approval of a new drug, the design of a new bridge, aircraft or power plant? Indeed the standards required for approval or acceptance for such projects are much higher and demanding than what has been demonstrated so far in climate science.
Could it be that the objectives of climate science are so virtuous that they be specially allowed to circumvent otherwise normal and rigorous approval processes?
Finally, whether the IPCC is a scientific or a political panel is yet another debate – one, I might add, becomes completely and immediately redundant once the question of whether or not climate mitigation ought to be subject to the same feasibility studies and approval processes as other projects (I myself think it must).
So far I’ve seen little data indicating governments ought to rashly accept the recommendations of the IPCC panel. The data show climate is behaving very much within normal boundaries of the last 5 million years. Any other view is politics and religion.
Eddy (Europe)
Angry I am not. But when I see governments blindly lining up behind such views. I get a little worried.
Concerning Steve’s work, I wouldn’t say it does not mean anything in Germany, where I am now living.
To the contrary, his work has been confirmed by Hans von Storch and has put many climate scientists here on the defensive. And concerning governments taking notice of his work, you are correct. True to their nature, they aren’t going to let a little science get in their way.
.
So, oconnellc, you want definitive answers to your two questions:
.
One, is Mann doing ‘good’ science?
Two, is Mann’s temperature reconstruction correct?
Here are two definitive answers, qualified by a Personally Guaranteed Confidence Level Estimate (PGCLE) of between 50 and 95%. In otherwords, my opinions are worth what you are paying for them. For what it’s worth to you, my two answers are both “no.”
As background, my personal opinion is that the earth’s climate has been warming at a more-or-less steady rate—with intermediate peaks and valleys in the long-term trend as would be expected of a natural system—since the end of the Little Ice Age. The warming trend is most likely the result of natural climatic processes, but with some minor impacts from human activity.
The climate will continue to warm until it reaches the peak of the Medievel Warm Period, at which point it will either continue to warm at more-or-less the same steady rate as it has in the past—taken as a long-term average trend—or else the warming trend will cease and temperatures will begin to fall at some more-or-less steady rate, also taken as a long-term average.
The climate has always been changing, and always will be changing, up or down. The climate will not warm indefinitely, as there is no reason to suppose another ice age isn’t in the cards, given how many have occurred in the last two million years. My sense of the situation is that I will not live to see any kind of definitive evidence as to what specific long-term trend will actually occur over the next 100 years, nor will anyone who is currently reading this blog be alive to see that definitive evidence.
If a definitive “yes/no” answer is requested, the answer clearly must be “no.”
In the absence of empirical research which demonstrates that tree ring growth patterns are directly and primarily responsive to temperature, no firm conclusions about past, current, or future climatic temperatures can reasonably be drawn from tree ring data. Yet, in Dr. Mann’s reconstruction, such firm conclusions are still being drawn.
The hockey stick is the product of imposing a temperature interpretation on data that has not been demonstrated to be primarily responsive to temperature.
Variations in tree growth cycles which are the result of a combination of environmental factors have been successfully labeled as “climate temperature proxies” through a process of blatant unsupported assertion combined with an array of convoluted analytical techniques which primarily serve the purpose of placing a veneer of scientific credibility over what is essentially an automated process for cooking the climate history data books.
If a definitive “yes/no” answer is being requested, the answer clearly must be “no”, and for several reasons, some founded in the content of the reconstruction’s evidence, some founded in the process used to arrive at the reconstruction’s conclusions.
Moreover, one cannot separate the quality and reliability of the evidence from the quality and reliability of the analytical processes used to examine that evidence. For purposes of making a yes/no answer, the two are deeply intertwined.
This could not be otherwise if the objective is to pursue “good science” in arriving at a set of conclusions. If the process used to arrive at the conclusions isn’t applicable and appropriate, then it isn’t “science” per se that is being done.
Now, a wealth of independent evidence exists which indicates a Medievel Warm Period occurred; however, we cannot characterize its exact temperature regimes or its exact nature and extent to the precision that we would like, or that some would absolutely demand.
Because we cannot characterize the MWP with absolute precision, is there reason to reject the substantial evidence for a Medievel Warm Period altogether and say that it should not play any role in our assessment of the validity of the hockey stick? How can one rationally do this when the evidence for an MWP is so voluminous and has been so extensively recognized and accepted for a period of more than fifty years?
Steve’s and his team’s examination of the quality and reliability of the analytical processes used to perform the reconstruction confirms that the reconstruction has not been pursued with anything approaching the level of rigor and discipline we would expect of people who call themselves scientists and who serve in the public’s trust.
So on two important counts which are fundamental in evaluating the “correctness” of Mann’s temperature reconstruction — the validity and reliability of the reconstruction’s evidential content, and the validity and reliability of the reconstruction’s analysis process and approach, the reconstruction has not been demonstrated by any rational scientific standard to be correct, regardless of what peer reviewers have said about it.
I myself am not concerned about climate change and global warming. The odds are that it is a natural process which cannot be stopped or even significantly modified by human intervention, intentional or otherwise. The warming process will reverse itself once Mother Nature, in her infinite wisdom, gets around to it. In the meantime, don’t worry, be happy. And enjoy the warm weather while you can.
The questions remain then; What does this mean for the vaunted “peer review” process, and does it mean this study and any related studies can be safely dismissed as nonsense?
Hmmm… This is my fault (and this isn’t one of those “I say it is my fault, but really mean it as an insult to you by implying that you aren’t intelligent enough to understand what I mean”), but I have misstated what I meant by question 2. What I think I should say is, do the errors discovered by answering question 1 significantly affect the results of the work? From what I understand Steve to be saying, the answer is “we don’t know”. That is fair. Steve also says that he only has so much time. That is also fair. To be honest, I am not all that interested in anything that Steve comes up with, until he has a chance to do some of those permutations and combinations that he mentioned.
What we might then find out is if Mann’s reconstruction is the right reconstruction or not. I use that terminology because people here have made the correct point that we don’t really know if it is ‘correct’ or not. When I say ‘right’, I mean given the proxies that we have, what do they indicate about past temperatures? If Mann’s reconstruction is ‘right’, then I don’t really care about all the mistakes. If it is ‘wrong’, then I don’t care about the reconstruction. If it is right, then we can get to the fun stuff, like trying to figure out if there really is a relationship between tree growth and temp and isotopes of oxygen and temp etc.
It seems like Mann and the quality of his work is being given far too much importance. The results of his work and the assumptions that go into his work seem like they are far more important.
Steve: IT’s not that I don’t intend to assess the impact of these things. I do. But I try to do things in order – the first thing is to see what they do and document what they’ve done. I write posts in part as a diary of my work. Not everything that I diarize is equally relevant. It’s fair enough on your part to defer judgment until impacts are analysed. However, we’ve been through enough of these things to be a bit suspicious of cherry picking and of “advanced” methods that end up papering over some subtle cherry picking.
Re: oconnellc (#50),
I have a difficulty following your conjecture about poor quality work providing the correct results here, while admitting that your apparent view is not that far from what the climate science consensus seems to claim in these situations. That claim, in my view, would go something like the following:
We know from other parts of climate science in a sort of unproven consensus (sometimes by way of a show of hands by specialists climate scientists who may be voting on inclinations about other parts of climate science) that we have AGW and work such as that of Mann et al. is merely to provide more and more easily digestible evidence for making the case for something we all agree exists (at least to some degree) and is furthermore of great importance to the future survival of man on earth. Following on this premise, Mann et al. are merely showing the way to that evidence. If they make errors in attempting to do this, it is more a case of the evidence being less digestible, but the consensus conclusion remains.
This is a very large and difficult question and one that we’ve been discussing here for 3 years. At present, my view is that, given the proxies that we have, we don’t know whether the Modern Warm Period is warmer than the Medieval Warm Period or the opposite.
Mann et al 2008 purports to be able to draw definitive conclusions. So the question for me in respect to this study is: are they able to?
oconnellc,
Let me try answering your question this way:
In terms of actually representing what the climate has been in the past and present I believe there is no way to know whether the answer Mann has generated just happens to be right, regardless of methodological problems.
In terms of whether the reconstruction Mann offers is right in terms of the available proxies, even if not to actual past climate, “right” is a slippery concept. Rightness depends a great deal on the choices made when creating the reconstruction. An example of this from a different field would be the choices accountants make when presenting the books for a company. Various (non-fraudulent even) choices can be made in that arena that will give different answers about the company’s condition.
I don’t feel that this second form of rightness has much relevance, the first is much more important. However the first form, as I already said, I believe just can’t be divined at the current time and possibly not ever. I do believe we can make relative comparisons between warm and cool periods, but less between twomoderatly warm periods which is the current question of interest. We can of course make comparisons between a moderately warm period and a hot house period, or a moderatly cold period and a true ice age, but both of those are somewhat remote at the moment.
Kenneth, I think you have misunderstood my position. I am not positing anything about AGW. In the current context, my interest is in temperature reconstructions. My interest is not in algebra. Nor is it in the quality of Mann’s work. I am approaching this from a much more pragmatic angle. If all of the errors that Steve is finding are corrected, what impact will that have on the reconstruction? If the impact of the errors is negligible, then the errors are of no interest. If the impact is significant, then the reconstruction is of no interest. In neither case am I interested in Mannian errors. Now, I am going to state the caveat that I am not interested in Mannian errors as long as Steve is around to be interested in them. He is going to find all the errors he can find.
Aside, I am not judging Steve in any way here. I am here in the first place because I think this is an interesting subject and there is sometimes interesting conversation between intelligent people that occurs here. I think Steve is doing a great job and this is not meant to be a criticism of what he is doing or how he presents his results to us. To paraphrase, you built it and I showed up…
Your last sentence in post #3 is a microcosm of why I posted: “Of course, as long as the answers keep coming out “right” who is going care about method”. Hear, hear! I couldn’t agree more.
“even though it is my distinct impression that the latest Mann 2008 version is far less “right’ than its predecessors.” Why? Don’t get me wrong, I respect your right to have an opinion. I hope others respect my right to ask “why” after you express your opinion in such a public way. Given this, I find your first paragraph in post 54 to be an odd one and hardly in line with what you wrote in post #3. The results are what count! Anything else is a philosophical argument. This could perhaps become a good case study for a class on ethics in science, but would you really have the governments of the world react differently to Mann’s reconstruction because the method was faulty, even though the results were correct? This, I think, is the most interesting question.
Re: oconnellc (#55),
If the method is faulty, how can we know if the results are correct? In any case, we already know that the errors in MBH (1999) affect the medieval/modern temp relationship. In today’s post, “The Rain in Spain”, we see that the mislocation error of one proxy in Mann et al (2008) alters the reconstruction estimate for the SH by about 0.5 deg C for an entire century. The sensitivity of climate reconstructions to proxy selection has already been well documented by Steve. It seems unlikely that Mann’s results are “correct”, but as I said, how could one possibly know if they were?
It would be better, as Steve has suggested, if the IPCC dropped the paleo stuff altogether.
The word ‘right’ here is referring to rightness in keeping Mann funded, bearing absolutely no relationship to actual correctness WRT past climate.
So yes I would hope that governments would take method into account when that method is designed (or discovered if it’s post hoc) in a self-serving manner.
So long as the reconstruction shows that the modern period is the warmest from the dawn of humanity it will be considered ‘right’ in Mann terms. This claim, however, rests on very shaky foundations, such as trees having a linear growth response to temperature. To try and claim a temp in 10000CE to within .2C seems ludicrous to me. And without such narrow claimed range it cannot be said that the current period is the warmest.
Soronel, couple points… First, I would love to hear Kenneth tell me what he meant in post 3. Second, your post exemplifies the issue I am trying to get at. You kind of make it seem like you are answering my question about caring about the method. But you aren’t. You are asking a question that no one answered. You are welcome to do that, but don’t address your answer to me as though you really are answering my question. Also, you are confounding two issues. The first issue is about if there is a physical relationship between tree rings and temperature (linear or quadratic or whatever). The second is if trees are linearly related to temperature, does Mann’s reconstruction accurately represent that relationship. If Mann makes two large mistakes and they have the effect of canceling each other out, so that the reconstruction reflects that relationship, why should I care about the mistakes? I care about the reconstruction. If he chose a self-serving method, why should I care? If it came up with the reconstruction that reflects that physical relationship WHY WOULD YOU PUT MORE IMPORTANCE ON THE METHOD THAN YOU WOULD ON THE RECONSTRUCTION?
And, no offense, but the fact that something seems ludicrous should have no bearing on things. My father was born in a time when a bathroom was outside, in another building, and you frequently had to dig yourself a new one. He told me that at different times in his life it was ludicrous to think about sending a rocket to the moon and it was ludicrous to think that you would need a computer your house for anything. He was wise enough to know that how ludicrous something seemed to him was irrelevant. So, why waste the effort trying to decide how you feel about someone saying that they know the temperature to within .2C. Lets let the people who understand the math just do it. Until then, what is the point of an editorial?
pardon me, I meant to say “you are answering a question that no one asked”.
I was pointing out the differing uses of ‘righ’ in #3 and #54. In #54 right was referring to actual correctness, whereas in #3 it was referring to ensuring continued funding.
You were the one who brought up the dichotomy between those two posts, whether I wrote them or not.
And, Soronel, thanks for answering, but I’m more interested in hearing from Kenneth what he meant by those words… If “right” means “showing lots of AGW”, then why would the second part of the sentence have said:
even though it is my distinct impression that the latest Mann 2008 version is far less “right’ than its predecessors.
James, what I don’t understand is why it seems that people keep trying to confound two different problems. One of them is: what is the relationship between paleo proxies and temperature. The other is: given an assumed relationship between paleo proxies and temperature, what is the temperature. Those are two different problems. If there is no relationship, or if the assumed relationship is wrong, then who cares anything at all about the method? Why spend time on it? If the assumed relationship is correct, then the only thing that matters is: does the method determine the correct reconstruction?
This seems simple to me, so maybe I am missing something obvious, but if you assume a certain relationship between the proxies and temperature, then there is a ‘correct’ reconstruction. It is correct based on those assumptions. If the assumptions are wrong, then the reconstruction is not interesting. But, it is correct relative to those assumptions. So, it seems that the question of concern should be, is Mann’s reconstruction ‘correct’ relative to those assumptions? I don’t know the answer to that. I think Steve has said that he doesn’t know the answer to that question. It seemed that Kenneth said that he knows the answer and that the answer is that it is incorrect. I asked him what his reason/proof was for saying that.
So, what question are you interested in getting an answer to? Is it the question about are the assumptions about the relationship between proxies and temp correct? If that is the question, what could Mann’s method possibly have to do with the answer?
Re: oconnellc (#62),
I think we are on the same page. There are two problems, but I don’t think I’m trying to conflate them.
Good question, Mann et al would say that there is a relationship, and assume a linear one at that. It seems a pretty dubious assumption, especially with regard to tree-rings.
Don’t get you here. One can grant the assumption about proxies representing temperature, and still critique the methods that derive temp from the proxies. It seems to me that you are conflating the problems.
Well, yes. If the assumed relationship is correct (doubt it, but concede it) then is the method correct?
Are you saying that if there is a clear relationship between the proxies and temperature, the method of extracting the assumed signal doesn’t matter? Or that the proxies selected (or the method of selection, if “method” is the right word) don’t matter?
Actually, I don’t get your point at all. Perhaps you could rephrase it?
Re: oconnellc (#62),
Oconellc, when I say that Mann et al. (2008) is less “right” then its predecessors, I mean that a careful analysis shows that this reconstruction going back into historical times exhibits significantly more variation than the original and many of its predecessors. The paper also explicitly comments that one cannot state with reasonable certainty that the global or SH past decades of temperature are warmer than past temperatures in the reconstruction.
You also might want to look at the reconstructions without the instrumental temperature record floating at the end of the reconstruction. Mann refutes that he splices instrumental temperatures to the reconstruction and therefore there is no way to relate the proxy and instrumental record and as a result we must confine our observations to the reconstructed temperatures.
I think Mann et al. (2008) and its predecessors received passes from critiques from a climate science community that has implied that they know the final answer through some vague consensus and a consensus that I cannot comprehend.
Here, Oconnellc, I think you are being a bit naïve. The HS was a tremendous visual aid for those marketing mitigation for the perceived detrimental effects of AGW for its shape was easily understood by layperson observers. It and its reconstruction predecessors have been used as a counterpart to the climate models as something more readily digestible by the consumer, if you will, than the models. If the uncertainty limits claimed by the authors for the reconstructions are accepted, they would tend to be smaller than those of the climate models. In my view, the consensus in the climate science community clings to the HS, not for its good quality, but for its imagery and I think out of concern that a severe rebuke of reconstructions and what they are capable of reproducing puts more pressure and focus on the climate models.
Now for myself and some others here (Bender comes to mind) I think that we have reached the conclusion that it cannot be shown with any reasonable certainty what these historical temperatures were compared to the current ones. In my view this conclusion would be primarily based on the analyses of currently used methodologies for these reconstructions that provide much evidence for data snooping, cherry picking, improper attachment of uncertainties to the results and lack of an a priori understanding of the proxy science involved. Having said that, the validity of future reconstructions, that pass muster with the proxy science and statistical methodology, cannot be ruled out. We also have the more opaque climate model methodologies to analyze and understand.
A further consideration of which you perhaps are unaware, is that many of us here derive pleasure and satisfaction from analyzing, and observing Steve M analyzing, these papers and methods and others from climate science. I think there are number of us here less interested in “saving the world” and more in solving puzzles (and indirectly determining what makes the climate science community click). If you do not enjoy the analyses and puzzle solving or find it a waste of time you are probably at the wrong blog.
quote poor quality work providing the correct results unquote
Apart from the ‘method wrong plus answer correct = bad science’ objection, if the science in this case is wrong then one cannot know the ‘correct’ results.
-snip: policy
Time to make an end.
Was the hope drunk
Wherein you dress’d yourself? hath it slept since?
And wakes it now, to look so green and pale
At what it did so freely? From this time
Such I account thy love. Art thou afeard
To be the same in thine own act and valour
As thou art in desire? Wouldst thou have that
Which thou esteem’st the ornament of life,
And live a coward in thine own esteem,
Letting ‘I dare not’ wait upon ‘I would,’
Like the poor cat i’ the adage?
Stab, man! To the heft!
JF
Julian, I assume you are responding to me. I’m not sure where we got so far apart… I’m talking about Mann’s reconstruction. Thats what I thought Steve was talking about. I’ll be honest, I don’t know who you are quoting and I have no idea what it means. I was thinking that Steve might be able to detect some problems with Mann’s method, figure out what what needs to be changed to fix those problems and then re-execute Mann’s method with the changes. Then we would compare the ‘after’ result with the ‘before’ result. Then we would know if Mann 08 is more or less ‘right’ then the predecessors.
Then, if Mann is ‘correct’ we could start talking about the assumptions he made (two different problems, right). Then, we could come to some conclusions. Wait, maybe that is what you are saying. I promise not to ruin the economy until Steve says it is ok.
“I’ll be honest, I don’t know who you are quoting”
http://www.shakespeare-online.com/plays/macbethscenes.html
Our host — to whom the advice was directed — would get the reference: he’s an Oxford man….
JF
The problem with the reconstructions is that it is all based on assumptions. We can’t do an experiment to test our hypothesis (a particular recon) and the answer is not in the back of the book. There are lots of assumptions and they differ for different proxies (not all tree rings). In a case like this, since you can’t do an experiment you CAN test the assumptions. For example, the assumption that the dendros know how to select cherries to make cherry pie was tested and Steve showed that you can easily select them to get no hockey stick or a warm MWP, meaning that subjective selection of proxies is fraught with subjectivity. The assumption that the PCA and other methods are objectively selecting proxies from a pool of proxies was shown to be false and that bristlecones and a few others were being selected and weighted heavily without justification. It has been shown that the errors such as using the Tiljander series which include instrumental data in the recent period “matter” for the result. It has been shown that the data are padded at the end and that some proxies are used upside down etc etc. I mentioned above that the linearity assumption is false, but since the authors of these papers carry on as if it is valid, these other types of audits are necessary. So while it is not possible to check the answer, it sure looks like “wrong methods + subjective methods + sloppiness = bad science”.
In response to oconnellc’s latest comments, I’ll be even more blunt with my opinions concerning the correctness of Mann’s reconstructions.
Mann’s temperature reconstructions are manufactured products being constructed to a pre-determined product specification.
The product specification is for a temperature reconstruction which can be further employed to discount the role of natural variability in recent climate warming. Hence no Medievel Warm Period in the reconstructions.
Mann’s reconstruction products certainly fulfill their product specifications and have been highly successful in the AGW marketplace as a result.
But as we would expect of any manufactured product, the various parts and pieces of the fabricated components, when closely examined, bear the imprint of tool marks.
Having been a professional auditor and a professional software designer constructing large-scale software systems, it is my opinion that the technical issues Steve and his team are discovering with the reconstruction software are the residual tool marks of the component fabrication process.
In light of the substantial evidence that a Medievel Warm Period did occur, even if we cannot yet characterize its temperature regimes with the precision we would like, I say that in my opinion, the Mann temperature reconstructions are not correct.
Oconnellc, is this rather blunt opinion of some further benefit to you in your quest for discovering the truth concerning the “correctness” of Mann’s temperature reconstructions?
Scott, sorry, but it really isn’t much help. I also design software for a living. One thing I have learned is that my opinions rarely matter. Software behaves the way you tell it to behave. It doesn’t behave the way you want it to. I feel that reconstructions/math are the same way. If they are not, then this website is a waste of time. A frequently interesting waste of time, but a waste nonetheless. When Steve demonstrates that substituting one equation for another makes a significant impact on the result, then I will know something about the result. I will not have an opinion about the result. My opinion is not terribly interesting on the subject, especially when there is a completely competent mathematician working on the issue. I’m not trying to be insulting, but to be blunt, as you say, your opinion on the subject doesn’t matter.
Re: oconnell (#69),
I don’t know that this will help you, but it may provide a bit of context. 10-15 years ago when I started discussing this issue, the position of the AGW crowd was 1) that it was clear that increased CO2 would warm the earth’s surface because it would absorb emitted IR. But they also claimed that 2) there would be additional heating from a positive feedback from H2O. But 3)they also said that the AGW wasn’t directly observable from temperature measurements and would only gradually become large enough to rise above the noise level.
When Mann et al came along, they claimed they could see the temperature rise and separate it using proxies (primarily tree rings). A bit later, Steve McIntyre and Ross McKitrick came along and claimed that Mann’s work was junk. At the time I noted on the board I frequented that M&M had better be right in their criticism or they’d possibly destroy the entire anti-AGW position. IMO M&M are not only right, but have, on an intellectual plane, destroyed the entire AGW edifice. I know Steve disagrees, but that’s my opinion.
OTOH, the original 1) of the AGWers is most probably correct; additional CO2 will cause some warming. IMO it will be smaller than the pure CO2 increase because it’s likely H2O is in a negative feedback situation. That is, an external temperature forcing of whatever source is likely to cause changes in convection and cloudiness which will decrease the net temperature increase below what would happen if the forcing agent were considered without feedbacks.
If you want any further discussion find a thread here on feedbacks as I suspect this thread is due for a trimming due to being off-topic.
Re: oconnell (#69),
See for example http://www.climateaudit.org/?p=3720#comment-305039 , we are talking about quite significant changes.
craig, thanks for your reply. However, I’m not sure you read what I wrote. I was trying to be explicit that at a certain point, you base the audit of the reconstruction on a certain set of assumptions. To try to avoid conflating two issues, I was stating that we accept the assumptions as valid. Then, check to see if for that set of assumptions will the method generate the correct reconstruction for a set of proxies. First, we assume that the absolute temperature of the planet in Kelvin is 10 times the width of a tree ring in millimeters. If the width of the ring is 14 millimeters, then the temperature is 140K. If I say that the temperature is 120K, then that reconstruction is wrong. But it is only wrong in the context of those assumptions. If I say that the temperature is 140K, then that reconstruction is correct, based on the accepted assumptions. It certainly looks like Steve has spent the last few years basically trying to check the math. First, he makes sure that the ring width really is 14 millimeters and then he checks to make sure that 10 x 14 = 140. I would think that if that all checks, you could say “Yep, the reconstruction is correct”. Then, if you wanted to have a separate conversation about whether or not the temperature really is 10 times the width of a tree ring, you certainly could do that. You actually probably should. But those seem to be two separate conversations to me. I don’t understand why everyone who keeps responding to me seems to think that that is one conversation.
oconnell,
I’m not sure about everyone else, but I certainly believe Mann has applied his method(s) correctly, regardless of that method not being documented very well. Method here is simply the code, bug ridden or perfect. As ou said before computers do what you tell them to do.Errors suchs as the misplaced precip record are just garbage going into the process and do effect the result as has been noted.
However just like a financial audit of a company you have to audit the assumptions made rather than take those assumptions on face value. IE, is it actually reasonable to say all of the profit from this contract accrues at the time of signing rather than over the life of the contract?
When dealing with novel methods auditing the method may very well be more important than auditing the result. A method can be devised to always come back with the same result.
f(x) = 1
Does looking at the result really make sense if it behaves this way?
Unfortunately, because the documentation and code are incomplete the work of reproducing the method is not a simple task. If it were fully documented then perhaps skipping by would make sense.
There are typically multiple layers of issues:
The results of these studies are typically very sensitive to presence/absence of a few series and the authors invariably select versions that are more favorable to a certain style of result e,g, Graybll rather than Abaneh, Briffa rather than Grudd,… Until this sort of issue is confronted squarely by the poroponents, it’s hard for me to see how people can take these things seriously.
One area where statistical methods can be demonstrated to be “wrong” is in confidence interval calculations. This was the sort of argument that Santer et al leveled against Douglass.
The difficulty with Mann et al is first figuring out what they did, then how they did their confidence intervals. As far as I can tell, there is zero documentation on how the confidence intervals were calculated even though that has been a longstanding issue with MBH99, an issue known to the referees. I suspect that the calculations are incorrect in some way, though it’s hard to show that without knowing how they were calculated.
It definitely appears that the benchmarks do not allow for various procedural biases e.g. things like pick-two, flipping series, etc.
There are also the “sloppy” errors – do they affect results? It’s not something that I particularly expect; I wsa surprisd at how much impact the rain in Spain had. This sort of thing provides a little bit of a trail as I can compare the corrections to the original results and that helps the decoding a little.
If the rain in Spain has an impact, the inverting of the Tiljander series probably has an impact as well. I’m close to being able to assess this for CPS, but EIV is still a mystery.
The sort of answers that I would expect to see at the end of the day are 1) the “true” confidence intervals are from “floor to ceiling” because the inconsistency of the underlying data does not enable much to be concluded; 2) slight variations in proxy selection or procedure can lead to very different results. This was the style of conclusion in MM03, MM05 and I suspect that the same thing will apply here.
Kenneth, I am trying to understand your comment that I am being naive. I make a comment about being unconcerned with the method, and you respond with a paragraph about the HS and the political implications of that image and AGW. You then later state that you are less interested in saving the world than you are in solving puzzles. I’m sorry, but I don’t think you are being forthright. In the interest of staying on topic, I will come back to what I thought was the point of this thread… an inconsistency in the Mann study. I am interested in the effects this inconsistency will have on the reconstruction that this study generated. If you look at much of the discussion on this thread, it follows this pattern:
The study by Mann (MBHXXX) has the following errors {followed by a list of errors/inconsistencies}. In my opinion, that makes this study {followed by a list of adjectives}. I keep wondering why everyone thinks that their opinion is of importance to something that is solved using math. Don’t get me wrong, I understand that this is a blog and people have opinions and that is why we are all here. But if I say that 2 + 2 = 5, you can say that it is either right or it is wrong or you don’t know. I just don’t understand that while we are all standing around waiting for the guy with the calculator to key in the numbers, it is necessary to say “In my opinion, the science is bad and because of that it won’t be equal to 5”.
Re: oconnell (#76),
Oconnell, I will attempt to be more forthright.
Without a good appreciation or experience in analyzing statistical methods (which these reconstructions actually are – given that one can accept the proxy science on which they are based) I suspect it may be difficult for the uninitiated to make a judgment on the validity of the Mann et al. (2008) reconstruction or its predecessors.
I think the time has come for you to tell us your background in and exposure to statistical methods. Do you understand the statistical implications of data snooping and cherry picking? Do you understand the classical methods of regression analysis and the usefulness of these methods in determining whether the model has any reasonable predictive value? Do you understand the implications of using non standard statistical methods, and particularly so when the user admits to not being a statistician? Do you understand the implications of using proxies, that are not based on an a priori choice that can be referenced to the science on which the proxy must necessarily be based.
The analysis and discussion here provide one with at least some of the information that will allow them to make a judgment on the validity of the reconstructions. You have appeared to shy away from any questioning of the specific statistical issues. There are many issues and layers of issues, as Steve M has noted, that have to be digested. There is no simple sentence or paragraph that will provide what I think you seek. If you have analyzed any papers before you surely must realize that that is the norm.
Re: oconnell (#76), You can only say it is simply a question of math when you know what the math is. In much of science there is mass of data and phenomena and we do not know what math to apply. So, we do studies and try things out. In paleo work, no one can check the results, so it all depends very heavily on the methods. These studies often use novel methods (math, stats) which no one knows how they work and which have never been demonstrated to work.
James, let me try to rephrase:
I said this (which you said you didn’t follow): The other is: given an assumed relationship between paleo proxies and temperature, what is the temperature. Those are two different problems. If there is no relationship, or if the assumed relationship is wrong, then who cares anything at all about the method?
You didn’t follow. My point was, if there is no relationship between trees and temperature, what is the interest in how someone derives temperature from trees? Would you be interested in my method of deriving temperature from the number of cans of beer in my fridge? I would think the last thing you would do is say “Well, we don’t know if he buys more beer in summer or winter, and since he keeps some of it in that drawer we can’t get a good count of how much beer he has so it must be wrong.” Those things are, as a professor of mine used to say, orthogonal.
Or, lets say that I have the widths of a bunch of tree rings. One person writes a paper where they say that they average all the widths, then multiply that number by 10, then divide by 2 and then multiply the result by 3. Someone else takes the same data and multiplies each width by 15 and then averages all the results. Person B comes up with an answer of 80 and person A comes up with an answer of 50. I’m not terribly interested in the method. I just want to know which is right. It turns out that person A really multiplied by 2, not 3, and person B multiplied by 16, not 15. It turns out that each method was correct. Its just that the calculators have small buttons. I don’t care, I’m just happy to know that the temperature was really 75. Looking at the big picture, we are going to (as a country/planet/take your pick) spend a lot of money based on whether or not the temperatures was really 75, not whether or not we should average first or average last or multiply by 3 or by 16. Does anyone really think that if we find out that the temperature is really 75 that we should wait to check the method to see if it is self-serving or not?
snip – no need to editorialize on policy
Another thread hi-jacked, beaker-style.
Re: oconnell (#76),
It appears that the problem here is trying to explain statistics to someone who has little understanding of and/or experience with statistical inference. Everything is black or white, correct or incorrect, just straightforward arithmetic, a recipe to be followed. Who cares how you get as long as the answer is correct. Steve Mc is just checking the arithmetic. I give up…
Kenneth:
I couldn’t agree with you more. In fact, I would go so far as to say it will be difficult for all but a very elite few in a few specific disciplines.
Really? Why is that? What affect would my answer have on how you respond to me? What if I said I have a phd in statistics and have spent the 17 years since leaving school doing experimental design? What if I said I had no formal training in any advanced math, and I am just a guy who likes to read a lot?
Good questions. Do you? What are the implications? Can you quantify them? Can you even qualify them? How does Mann’s choice of proxy affect his results? Does it change the reconstruction? The confidence intervals? Please demonstrate for me how the removal of 10 random proxies from the population affects the results. I’m very eager to see. I get the feeling that if I posted my opinion that removing 10 proxies would increase the MWP, I would be met with great camaraderie. Looking back on this thread, I think I have done nothing but ask those same questions. What is the impact? Is it significant? The only other thing I may have done is state that opinions on the matter aren’t that impressive.
I have to admit, I’m curious what you and the other posters here thing my point of view is. I get the feeling that you think I have to be defended from. Would it surprise you to learn that I do not believe in AGW? I do believe the planet is warming and has been for some time. I don’t know if it is warmer now than it was 1000 years ago. I am surprised that what I have posted appears to have drawn more outrage than a comment that a correct answer arrived at by a self-serving method would be given less credence than one arrived at by… what? Something more pure?
I’m sorry I appear to have hijacked this thread. I guess I thought when I posted that I might trigger some discussion around how to go about verifying the impact of some of the things that Steve has discovered? What would be the best way to present the results? Is that something that should/could be published in a professional journal? Perhaps even something might come up that I could help with, in the middle of the rest of my busy life. Steve addressed my question, but everyone else just seemed upset that I would consider something like verifying the impact of all these issues before declaring Mann 08 destroyed. You are correct, there are issues and layers of issues. That hasn’t seemed to have had much affect on the amount of conjecture that appears. For example, do you believe that Mann’s correction of the Rain in Spain really affected his work by .5C in the 18th century? What if he made a mistake implementing the correction? What if the impact was really just .01C?
Re: oconnellc (#82), I notice you dodge responding to my comments. Everything you mention has been tested here for Mann and other papers (cherry picking, truncation, filter choice, errors, wrong use of PCA, etc.) and they all “matter” — they all affect the result. Try reading around on the blog a little before asking these questions.
Re: oconnellc (#82),
Oconnellc, hang on to your hat. Maybe others are having a go at you, but you seem like an OK guy to me, I’m just trying to understand what you’re saying.
First, you seemed to be saying that the key issue is the validity of the proxies as temperature proxies, and that there is no point in arguing about methods before that’s resolved. Well it’s certainly a key issue. Mann et al argue that a temperature signal can be recovered from tree-rings. Given that the IPCC reports the paleo-reconstructions, we can presume that the IPCC takes this view as well. Plenty of people here would disagree, and even some of those in the dendro community. So the issue is unresolved, and may not be resolvable to everyone’s satisfaction.
Therefore it is reasonable to grant that something might be recovered from the proxies and examine how they are handled in the reconstructions. Of course this is a different issue from whether the proxies are any good, but it doesn’t mean that it’s irrelevant.
I realise you’re new here, but Steve has provided many examples of the sensitivity of the reconstructions to the inclusion or exclusion of various proxies. Want an elevated MWP? Let’s make apple pie:
http://www.climateaudit.org/?p=581
A point you seem to miss is if you want to examine the sensitivity of reconstructions to proxy selection or methodological choices, you have to work out what the paleo crowd have actually done. If you can’t replicate the methodology, you can’t do sensitivity analyses. That’s why Steve spends so much time trying to reverse engineer the methods, a point he makes time and time again. If he finds and reports amusing errors along the way, so what? Maybe they don’t “matter”, although it does suggest something about the quality of the work. BTW, for an excellent sensitivity analysis of methodological choices in MBH, you should read Burger & Cubasch (GRL, 2005).
At this point you completely lose me. Someone’s supposed to audit Mann’s corrections to his errors? Why don’t you do it and report back?
oconnellc,
Okay, I’ll try addressing these points. Various reconstructions can be put forward, showing very different past climates, and while some it may be possible to say are “wrong”, you can be left with multiple “right” reconstructions.
As for auditing whether Mann even performed his own method correctly, it first has to be determined what that method actually is. That step has not been completed yet. One must also determine exactly what data sets were used to calculate the published reconstruction. Only after those steps have been done (maybe others I’m not thinking of as well?) can whether the method was applied correctly be determined. Also only then can alternative data inputs be tested, or the assumptions fully tested. Partial tests can be done however before those two major steps are completed.
Beause CA is a blog Steve M uses as a diary of results we get a bit at a time, rather than a condensed paper about what has been found.
Soronel, thanks for the note. I agree with everything you said.
oconnellc, thanks for your postings. I have the sense that you are confused about what we are doing here.
You seem to be of the opinion that if Mann’s outcome is “right”, then it doesn’t matter that his methods might be wrong. The problem with that theory is that at present, the only thing that we have to determine whether Mann’s outcome is “right” is whether his method is “right” …
If I understand you correctly, you have proposed that we examine two questions, viz:
1. Is there a relationship between the proxies and temperature, and if so what is it?, and
2 If there is such a relationship, does Mann’s method extract the “right” answer about the temperature from the proxies?
Unfortunately, we are into deep water here. Lets start with question 1.
First, there are proxies and there are proxies … ice cores, tree rings, CaCO3, lake sediment varve thickness, and others. For each proxy there are theoretical reasons to think that they may be related to temperature … but very little experimental evidence that in fact they are. In addition to that problem, different examples of the same proxy (e.g. two different tree ring datasets, or even two different trees) may have very different (or no) relationships to temperature.
Next, regarding Q1, we have to ask if the postulated relationship to temperature might change over time (e.g. CaCO3 changes due to changes in ocean salinity).
Finally, with regards to Q1, if a relationship exists, what is the exact form of the relationship? This leads to a most important question … is the relationship invertible? That is to say, for a given value of the proxy, is there only one temperature which will give that value, or is there more than one temperature? For tree rings, the answer is clearly “more than one”.
From that first question, we can move to Q2. Unfortunately, we have absolutely no way to determine a priori whether Mann’s result is right or wrong. For temperatures in say the year 1000, we have no information other than that from proxies. So your claim, that we might be able to say Mann’s result is “right” despite his errors, is simply not possible.
Now of course in a perfect world, scientists would:
1. Do a number of experiments to determine the answer to Q1 from first principles for each candidate type of proxy (e.g. ice cores, tree rings).
2. Put reasonable error bars on the answers.
3. Reconstruct the temperature using each proxy separately, and compare the results.
Unfortunately, Mann has taken a different path. Rather than consider any of the questions you posed, he has taken all kinds of proxies and thrown them in the pot together, assumed a linear relationship between all different proxies and temperature, and then used convoluted, bug-ridden, theoretically doubtful methods to come up with an answer.
Does that method accurately portray past temperature? As I said above, we have no independent evidence of the temperature in the year 100, so there’s no way to know.
However, we can determine whether his methods have any substance to them. We can determine if he has applied the method correctly. We can see whether they contain any foolish errors, the “stupid pet tricks” referred to in the title. We can also discuss whether the answer to question 1 is yes, no, or sometimes, for any given proxy. We have done that in a variety of threads here.
You seem to think that we are engaged in determining whether Mann’s results are “right” in the sense of accurately representing past temperatures. We are not. We are engaged in determining whether Mann has used valid proxies, if he has used correct methods, if he has applied them correctly, and if he has put honest error bars on them. This is a very different quest to trying to reconstruct past temperatures.
I hope this might serve to clarify some of your confusion.
My best to you, and everyone,
w.
craig, I wasn’t intentionally avoiding you. I’m not sure how to respond. I just forced myself to go back and read this entire thread. Part of what I am confused by is the dichotomy between the existence of this blog and the opinions of the people who read it. Am I correct by summarizing that the current feeling here is that there is no way of knowing if Mann is ‘correct’? We can only know if he is incorrect. And we can only really know that by demonstrating that he incorrectly implements a significant number of statistical procedures. But, I don’t know what that significant number is. Nor do I know what procedures are significant. And if Mann’s method is novel and untested, how can anyone know what is significant in the method? I would have expected that the test for ‘significance’ would be the impact of that incorrect implementation on the result. Is this expectation the glaring symptom of my naivety?
I find this thread somewhat disheartening, and I cannot predict how busy any particular day at work will be, so if I do not respond to you quickly in the future, please do not assume that I am dodging you.
Willis, thanks for your note. We appear to have been typing at the same time. Despite current opinion to the contrary, I am not trying to fight with anyone here. I am actually quite sympathetic to this blog. I hope you take my note in that light. But I read your description of what this blog is trying to accomplish. My first question is, what is the point? Lets say that you find several incorrect procedures, or correct procedures misapplied. Then what? You can certainly give Mann a grade, but what else? You can say this demonstrates that his results are incorrect. But, then he says it does not. He can say that the proxies were not cherry picked, but are the correct proxies. You end up with two camps staring at each other over a large fence. Except the people on Mann’s side of the fence have legitimacy. Chair of UN committees, great graphs, mainstream press, cool titles and possibly even groupies. This side of the fence has a blog.
Now, Steve has said that when he finishes his audit, he plans to run some of those parametric studies, changing proxies to see the affect on the result. Aside from being an interesting mental exercise, there is little to get excited about without that last step.
And, I’m curious where this goes if Steve does decide to do those parametric studies and the result is unchanged. What if the result is immune to proxy selection, method selection or application?
oconnellc,
We’ve already seen that this method is sensitive to proxy selection, the misplacement of the precipitation record from Spain to Kenya points that out.
I do hope that Steve puts together a paper after all the kinks are ironed out, like he and Ross M did before.
In some ways, the work MM have pushed has already borne some fruit, Mann did a much better job archiving his work this time around than was seen for the 1998 paper. We can hope that the pressure causes others in the field to re-evaluate how they conduct their work.
I for one would welcome properly conducted science, no matter what the final result.
oconnellc, I appreciate your reply.
Surely, however, you can’t be serious when you say:
Steve, and I, and others, have shown in a host of ways that the Mann’s results are in fact sensitive to proxy selection, method selection, and application.
But on a more general note, I can’t think of any reconstruction done by anyone that is “immune to proxy selection, method selection or application. Perhaps you could give us some examples of what kind of reconstruction you might be referring to, that has those characteristics.
w.
Willis, you are right. I wasn’t serious. I had no particular study in mind, nor did I think that that was a particular possibility.
However, I want to thank you for addressing my point. I originally made a comment about isolating the method from assumptions about proxies/temp relationships and then testing the method to see if the result changes. Here we are 90 posts later, with people upset at me for proposing something like that, and it turns out you have been doing it all along.
Re: oconnellc (#90),
I am certainly not upset with you, even though I cannot understand why you apparently think that there must be an established means of knowing temperatures 1000 years ago so that we might conjecture that Mann et al. (2008) is using or could be using incorrect statistical methods to obtain the correct answer – but since it is only the correct answer that is important we are collectively wasting our time.
Others here, far more qualified than I, have time and again pointed you to the fact that we do not know what the “right” answer is and that to obtain a “right” answer one would have to use valid proxies and do the proper statistics. The debate then boils down to: Are the proxies valid and were they analyzed with the proper statistics? In analyzing papers on just about any related climate subject this is the essence of the discussion and that is why some here have been, not upset, but perplexed by your failure to get the point.
You may also not appreciate what Steve M and other statistical professionals are doing in preparation of testing Mann’s methods and that is what might appear to you as minutia, but it is critical to understanding the sources of all possible errors and the ability to carry out the calculations precisely as Mann et al. does. This avoids the wave off by the paper’s authors of any criticism by making exact duplication of the methods (and even the parts in error) a side issue. It has been done before.
I will make two more quick points and end my comments. Firstly, I am not so much interested in answering your queries as I am in putting in words my own understanding of what transpires here so that others can critique it if it gets too far off the track. Secondly, your underlying attitude comes through, load and clear, when you continue to reference the waste of time these discussions (of Mann’s paper, not the ongoing exchanges seemingly for your edification) appear to you. To that all I can say is that they push my buttons, but to each his own.
Thanks, oconnellc. You might consider taking a cruise through previous posts. Look under the dropdown that says “Select Category” near the top left of the page, grab a topic that appeals to you, and cruise on through. You’ll find plenty of meat in there that addresses your concerns.
All the best,
w.
Kenneth, I don’t think I have effectively communicated things to you.
That is not what I think. I have tried to avoid saying that at all costs. I think people here got an impression of what they thought my point of view must be, mostly because my comments seemed as though they were attacking the status quo here.
Despite what you think here, the rest of the world does think that, though. The UN, most ‘official’ government positions, mainstream media, etc., etc. certainly do think that it is possible to know the temperature 1000 years ago. And when they see a temperature reconstruction like Mann 08, it passes peer review, is published and has results similar to other papers on the subject. How often have you read that Mann’s method doesn’t matter because other people have gotten the same result? Outside this blog, the result matters. And inside this blog, what happens on the outside matters. I have never said that the discussions on this blog are a waste of time. What I said was a waste of time were the ‘my opinion is that this is wrong’. What I find interesting are the discussions that involve application of fact. An interesting discussion is one where you say “my opinion is that this is wrong because I discovered this resource and it states blah blah blah”. I’m sorry if that seems like a fine point to you, but it really isn’t. And I never really said anything about anyone stating their opinion until people started going out of their way to push their opinion on me when I asked a question that called for a fact. I never demanded that anyone answer my question, but I did mind when I asked a question and someone pretended to answer but instead took it as an opportunity to editorialize. I have gone back and re-read this thread. This all started because I just asked you if you a graph or something that demonstrated your statement in post 3.
Steve said this in post 74:
That is exactly the sort of thing I was hoping to find. Almost every other post by someone else has been someone stating that asking for that sort of thing is naive. I have been a fan of this blog and I have read in other place people complaining about treatment of anyone who seems to question the consensus here. I used to think those were sour grapes. Now I’m not so sure. If you disagree, just look at all the posts calling me naive when Steve said he actually hopes to produce exactly what I was hoping to see.
Re: oconnellc (#93),
If you have been a long standing fan of this blog why have you not been more specific in referencing what it is that you seem to be questioning. Steve M does sensitivity studies with the reconstruction data, but I have not gotten the impression that he plans to do a “correct” reconstruction at this point or that he judges that one exists.
I called your approach naive and if that becomes the focus of our conversation then I think that would be because you are more interested in blog style here and taking the opportunity to give your opinion of the posters and postings than in what you could learn from your participation here. You seem to be aware of what the climate science consensus is in these matters yet naive on how one would go about analyzing a paper on it. Of course, post comments will sound more like opinions without a knowledge basis to you if you have little knowledge of what they are referencing.
When you asked me for a graph from my post (Re: Kenneth Fritsch (#3),) , that was either naïve or completely out of context. If you had a specific issue in mind vis a vis all the analysis that have been done here and what the consensus claims – fire away. In post # 3, I was pointing to the fact that Mann et al. uses new methods with old data and that I was throwing out my view that in doing this the authors were in this instance data snooping with methods and not data. I would be willing to discuss the statistical implications of that with you or any other poster here, but here is what you replied:
That comment could be taken as one totally out of context with little or no importance to the discussion or one that is implying that one needs one’s own reconstruction to show the correct temperature before commenting/criticizing that of Mann et al. In light of the helpful comments that you have received in the meantime here would you admit that your comment was either out of context or naïve (I would accept smart-alecky if you were pointing to my propensity to post)?
You also said in your first post on this thread the following:
Now somehow you had taken a situation where the “correct” answer can be ascertained independently of the (questionable/incorrect) methods used by someone and have applied it to a case where you do not have an independently derivable answer and thus the correctness of the method becomes critical and limiting with reference to the “correct” answer.
Now that you have received a tutorial here on that matter, would you now admit that that statement was either out of context or naïve?
You went on in Post #62 (Re: oconnellc (#62),) to say the following:
Here you are generalizing away what goes into a reconstruction and attempting to separate parts of that are not separable – and in my view in order to continue your same line of argument.
You can assume a relationship between a proxy and temperature, but that is not what is done. Ideally you have some a priori science of the proxy that strongly suggests that something measurable can be related to temperatures and such that the relationship will not have changed over time. Now you must calibrate the proxy response against a reasonably accurate instrumental record that is sufficiently proximate to the proxy that both are responding to the same temperatures. Calibration usually takes a form of regression. Once that part is in hand you must validate the calibration by testing it against a withheld part of the instrumental record. Now there are really no assumptions made and more to the point there are critical selections and choices along the way that need to be guided by the “correct” proxy science and “correct” statistical methods. It is the “correctness” of that science and the statistical methods that are then critical and limiting, and that is what is being debated.
Now based on my, admittedly less than comprehensive, description of the process would you care to show me where you obtain this two part question and answer to which you keep referring?
Re: oconnellc (#93),
But that was not exactly what you asked in oconnellc (#31):
You seemed to think that these were two separable questions. They are in fact one and the same. Mann’s specific reconstruction is the result of applying a series of decisions and techniques. It is a result of the methods (plural) used in its construction. Your first misimpression was that somehow a particular calculated result can be verified as being “correct”. Since that requires that it be directly compared to what actually happened in the past, this is impossible to do (as in many situations in the real world). Thus, the only remaining avenue is to look at what decisions were made and what methods were used (correctly and/or incorrectly) in the process – in your words, is it “good science”?
Much of what you said about methods indicated to me that you did not understand that statistical methods are not mere straightforward applications of mathematical equations. Properly derived and validated methods are ways of extracting certain types of information from the data. They are not “correct” or “incorrect” per se, but they are valid for a particular purpose if the assumptions for their use are satisfied and if they are correctly applied. There is no single set of statistical procedures which gives THE “correct” reconstruction, so your request for the Mann results to be “corrected” is unrealistic.
How do you evaluate the result? Many ways. The one you seemed to think was useful to you above is one – how do the results respond to the input data. But, that is an examination of the methodology used in the reconstruction. Are the results internally consistent? Were the methods applied properly and correctly interpreted? A statistician always provides “error bars” – estimates of how far the results might be expected to fall from the original temperatures. Are those meaningful and realistic? All of these answers come from an examination of the methodology of the reconstruction. Was “good science” used in the entire process?
Steve’s work is not simple arithmetic to see if the calculations are correct. It is a step by step search to discover exactly what the Mann team did. Getting past all of the unmentioned steps and undocumented changes is a real experience and I admire his ability to do so. Only after all of the steps have been nailed down, can you finally answer the “good science” question. This is done by evaluating the steps, finding technical errors and forming “opinions” about whether proper decisions were taken. Your statement “I keep wondering why everyone thinks that their opinion is of importance to something that is solved using math” is completely off the mark since that IS what a professional statistician does all the time when reviewing statistical results. It is not cut-and-dried and requires understanding and experience.
Finally, I think you are wrong in stating
IMHO, the somewhat negative response you may have received on this thread was not due to any perception that you were not in agreement with the general viewpoint that may be shared by many of the regular contributors. Rather, it was because of a continued troll-like repetition of a number of off-the-mark ideas. If you are open to the multitude of reasonable explanations you were offered about what is actually going on at this site, you might actually find you were given answers to the questions you should be asking.
Re: RomanM (#96),
Thanks Roman. By the way Oconnellc, you may have reason to disregard my comments, as I admit to being a grumpy old man who posts too much, but Roman is a professional statistician, who has taught statistics at the university and is deservedly respected at this blog.
oconnellc,
Perhaps part of the problem here is the fine distinction between fact and opinion in this field. The reconstructions, whether Mann’s, Steve M’s, or anyone elses are all opinion. They are opinion drawn up in terms of math but still opinion. Tree ring measurements are fact, local temperature measurements are fact (though global temps are again opinion). Even those local measurements while precise may not be very accurate.
So it boils down to either making up your own model/opinion or doing enough work to decide to trust or not trust someone else. Steve has done enough work that I am comfortable not trusting Mann et al, but not so much that I am comfortable trusting his own reconstructions to have any particular relationship to the actual past.
.
If this series of discussions has sometimes taken on the flavor of “bring me a rock”, that’s OK with me as most of the rocks have been value-added rocks.
In pursuing any kind of scientific research, and in reaching any kind of firm conclusions based upon that research, it is perfectly acceptable and appropriate to step back from the lowest level of detail and to ask the question, “Do the conclusions of the research make any sense within some broader context of what we already know to be fact?”
If the conclusions don’t square with some broader knowledge of scientific fact, then it’s time to start asking deeper questions about the research and to follow those questions wherever they might lead.
In other words, “Does this dog actually hunt, however well-groomed he is?”
Auditing the climate research to some great level of detail doesn’t mean automatically rejecting the research if problems are discovered. But it does mean that hard questions will be asked and that appropriate issues will be raised as the specific audit findings indicate are necessary.
Moreover, standard procedure in Quality Assurance practice is to integrate the sets of individual audit findings into a broader picture of the overall situation, and to draw further conclusions about that larger picture based on the assembled evidence.
Concerning the pursuit of climate science itelf, as I see it, two general and competing perspectives exist concerning the broad nature and character of the earth’s climate, perspectives which greatly influence how their adherents believe climate research should be conducted.
One perspective views the earth’s climate as being predominantly a machine which operates in a predominantly deterministic fashion.
The other perspective views the earth’s climate as being something more akin to a living organism, one which exhibits greater inherent variability but also greater tolerance for changes in its driving physical factors.
The climate-as-machine adherents rely almost exclusively on mathematics and numerical simulation as their tools for examining climate data and for generating future climate predictions.
In contrast, the climate-as-organism adherents use mathematics as but one tool in their tool box, and tend to integrate their mathematics with other kinds of analytical tools and methods.
As I have observed them, the climate-as-machine adherents assume the earth’s climate to be fully capable of maintaining a stable temperature regime for thousands of years on end, in the absence of significant distubances to its key physical parameters.
In contrast, the climate-as-organism adherents assume the earth’s climate will tolerate some level of disturbance in its driving physics while at the same time exhibiting some greater level of internal variability. (In other words, Mother Nature is predictably fickle.)
The two perspectives affect not only the methods their respective adherent’s employ in pursuing their climate research, it also affects their decisions concerning which larger bodies of knowledge they will choose to include or to exclude while developing their conclusions.
OK, what point am I getting to here with this kind of analysis?
There are those who don’t believe Steve McIntyre’s work will have any impact on the course of public policy debate concerning AGW. I say it is much too early to make that kind of prediction, simply for the fact that serious anti-carbon measures have yet to be implemented, and so the policy debate over AGW hasn’t yet begun in earnest.
If carbon limiting measures result in serious economic dislocations, the public will begin demanding a much higher standard of quality assurance and quality control in climate science research, because it is then greatly in their interest to do so.
In my opinion, the warmers, by relying so heavily on mathematics and numerical simulation in pursuit of their brand of climate science, have effectively chosen to “go nuclear” from a quality-management / quality-control perspective.
Steve McIntyre is the first nuclear-grade auditor to show up on their doorstep; and if my assessment is accurate, he won’t by any means be the last.
Which of the nuances of which of these were we discussing?
1. Water contributes up to about 85% of the greenhouse effect.
2. Methane is one of the gases that absorbs outgoing energy from the Earth’s surface.
3. The human activities of burning fossil fuels and changing the uses of land results in an impact upon the Earth’s climate system.
4. The global mean temperature anomaly trend is up by .7 since the late 1800s.
5. The concentrations of long-lived well mixed GHG have gone up since the mid 1800s.
6. All else held equal, an increase in the concentrations of the non-water vapor GHG would result in an increase in reaction to outgoing energy from the Earth’s surface.
7. Some glaciers are receding or shrinking.
8. Models show a lowering of ocean pH levels.
9. Since the mid-1800s, carbon dioxide atmospheric levels have gone up from 280 to 390 parts per million by volume.
10. Since the mid-1800s, the planet’s population has risen from 1 billion to 6.5 billion.
11. Some plants and animals have become extinct or moved habitat in the last 3 million years.
12. If it were possible to keep the same atmosphere except for the removal of the long-wave infrared reactive gases, the probable outcome would be a reduction in the average temperature of the Earth.
14. The sky usually appears blue because the intensity of scattered light is the fourth power of its frequency.
15. Meteors or the results of them might have killed the dinosaurs.
16. The atmospheric pressure lessens the higher you move up into the troposphere.
17. Ozone absorbs incoming ultraviolet in the stratosphere.
18. Everything tastes better dipped in melted cheese, even cheese.
One Trackback
[…] Here is the original: Comment on Mannian CPS: Stupid Pet Tricks by Scott Brim […]