From time to time, scientists inadvertently use contaminated data and their results are affected. For example, some time after publication of Grand et al (PNAS 2004), the authors determined that their results were erroneous as a “result of contamination of genomic DNA with plasmid DNA”. They promptly issued a retraction, expressing their regret for the error and any inconvenience. The reputation of the authors does not appear to have been diminished by the retraction. Mistakes happen and the mistake was promptly dealt with by retraction of the article.
Like Grand et al, Mann et al 2008 (M08) used contaminated data, in their case, the Finnish sediment data of Tiljander et al, the modern portion of which had been contaminated by agriculture and bridgebuilding. In addition to using the modern contaminated portion of the data, M08 made a second error by using the Tiljander lightsum and XRD upside down to the interpretation of its originators. Their handling of Tiljander data has been sharply criticized on different occasions by two eminent Finnish paleolimnologists – Atte Korhola here and Matti Saarnisto here.
In contrast to Grand et al, Mann et al have not issued a retraction or corrigendum or even admitted an error. Instead, in multiple venues (without explicitly admitting an error), they’ve asserted that, in any event, the error doesn’t affect their “central conclusions” [their PNAS reply, Feb 2009 here] or, more recently, “any” of their conclusions [Mann et realclimate, June 2011 here], as though that ended the matter.
It doesn’t. Even if the error didn’t have a material impact on the results, a Corrigendum should have been issued. Kaufman et al 2009, for example, issued a corrigendum when they learned that they too had used data (including their Tiljander series) upside down. But the situation for Mann et al 2008 is quite different. The most prominent claim for Mann et al 2008 was its supposed achievement of a “skillful” reconstruction without tree rings for the past 1300 years. Unfortunately, this “achievement” can now be seen to have been a complete mirage, dependent on the use of contaminated data in the EIV reconstruction without tree rings.
Realization of this failure has taken some years since publication, because the impact of contaminated data is not the same on all reconstruction variations and the original analyses only reported the cases of least impact.
For example, if the reconstruction also uses strip-bark bristlecones, the contaminated sediments do not have a material effect. On the other hand, the contaminated data turns out to have a dramatic impact on the EIV reconstruction without tree rings. The impact on the CPS reconstruction without tree rings is intermediate.
The original SI to Mann et al 2008 (and again in the recent realclimate thread on Kemp et al 2011) only showed the case where contaminated data has the least impact – reconstructions which also include bristlecones. To date, the impact of contaminated data on the no-dendro EIV and no-dendro CPS reconstructions remains unreported at the official Supplementary Information at PNAS. The no-dendro EIV was not even reported at Mann’s website for Mann et al 2008, where he has been publishing informal corrections to the official SI. Instead, the disquieting no-dendro EIV results were quietly released within the SI to a different article (Mann et al 2009, Science) without any mention in the running text of the article itself, without any corrigendum at PNAS and without even a realclimate thread. Nor, as shown below, did the SI to Mann et al 2009 clearly show the full impact of the contaminated data. This became much clearer with AMac’s recent re-coloring – see his post here – which prompted my re-examination of Mann et al 2008.
The Return of Mann et al 2008
Mann et al 2008 came back into the news with the recent publication of Kemp et al 2011 and its associated thread at realclimate.
In response to renewed questions about the Tiljander sediments, Mann stated om June 23. 2011:
People have asked whether the use of the Tiljander proxies in the Mann et al (2008) EIV surface temperature reconstructions matters for the conclusions of this or any related studies. The answer, as provided previously in the literature (see this reply to a comment in PNAS) is no.
The reference in this assertion is to their Reply to the Comment by Ross and I where they stated:
Potential nonclimatic influences on the Tiljander and other proxies were discussed in the SI, which showed that none of our central conclusions relied on their use.
Similar claims had been made by Gavin Schmidt (seehere for example) and by Kemp coauthor Martin Vermeer (see here ) during the long simmering blog discussions of the effect of the Tiljander sediments on M08 reconstructions.
AMac responded to Mann’s June 23, 2011 posting here, prompting a suggestion from reader MikeN that AMac post a relevant graphic from the SI to Mann et al 2009. To that date, this graphic had never appeared or been specifically referred to in the lengthy discussions of M08. On June 25, AMac complied, posting up two figures shown below. Mann et al 2009 SI Figure S8 (left) shows the reconstruction with contaminated data from 700 in wide solid blue, but the reconstruction without contaminated data is hard to make out: the thin narrow dashed green from 100 to 1500 are barely visible. On the right is AMac’s recoloring clearly showing the EIV no-dendro reconstruction without contaminated data (solid green). Click on thumbnails below for larger versions.
|
|
|
Left- NH EIV NO-dendro reconstruction showing effect of contaminated data. Left – from Mann et al 2009 SI; right – as recolored by AMac.
Despite the many assertions from Schmidt, Mann, Vermeer and others that M08 use of contaminated data made “no difference”, there is an obvious and large visual impact on the EIV no-dendro reconstruction, as opposed to the lesser effect on the variations that they’ve preferred to discuss.
The original SI and the recent RC post both only consider the impact of contaminated data on the with-bristlecone EIV reconstruction (“all proxy”), where the impact is far less as shown on the left below in the graphic from the most recent realclimate post re-stating M08 SI Figure S7b. The November 2009 update to the PSU website for Mann et al 2008 only showed the effect of contaminated data on the CPS no-dendro reconstruction (not EIV), where the impact is noticeable but far less than the more prominent EIV reconstruction – see the below right graphic originally posted up at Mann’s website in November 2009, but it is obviously smaller than the corresponding. The CPS reconstruction was mentioned in a contemporary CA post, but today’s is the first CA post showing the EIV no-dendro reconstruction.
|
|
|
Left – Effect of contaminated data on the with-bristle (all-proxy) EIV reconstruction; right – on CPS no-dendro reconstruction.
The pronounced visual impact of contaminated data on the EIV no-dendro reconstruction prior to AD1500 extended to RE statistics and “skill” claims. New information on this impact became available on Aug 22, 2010, when Mann et al archived verification statistics for the EIV no-dendro reconstruction without contaminated data at Mann’s website for Mann et al 2009, a different article in which no mention is made of no-dendro reconstructions in the running text.
The new information is directly comparable to (and contradict) the statistics shown in the inset of Mann et al 2008 Figure S4a(d) (see left below.) Similar insets for no-dendro reconstructions are shown in no fewer than 8 panels of M08 and all are compromised. The statistic, shown in the inset for each of 19 steps, is one that, to my knowledge, does not occur elsewhere in statistical literature. For each step, Mann et al calculated the average of the “late-miss” and “early-miss” RE statistics. In many cases, one is very high and the other can be negative – an unappetizing statistical result that is brushed off and left unanlysed. On the right panel, I’ve shown these “average” RE statistics for two cases: with and without contaminated data. The former is drawn from the PNAS SI, the latter from the Aug 2010 addition to the Mann et al 2009(!) SI.
At right, instead of showing the running maximum, I’ve shown the “average” RE statistic for each step, with the result with contaminated data on the left and the result without contaminated data on the right. Values without contaminated data are much lower and “insignificant” prior to AD1500. Without the contaminated data, the reconstruction no longer “validates” between AD700 and AD1500, a point quietly conceded in the SI to Mann et al 2009 and in an inline comment at realclimate to Nicolas Nierenberg last year, but not yet reported to PNAS.
The actual inset shows an even quirkier statistic – the running maximum “average” RE value. If the addition of more data reduces the fit, the additional data is ignored in the supposed interest of avoiding “overfitting”. (None of the rationalizations make any sense, but that’s a story for another day.)
|
|
|
Left- NH EIV NO-dendro reconstruction as shown in M08 Figure S4a(d). Right – “Mean” RE statistics with contaminated data (left) and without contaminated data (right).
The “Central Conclusions” of Mann et al 2008
All of the above impacts have now been admitted. The next question is the degree to which these impacts affect the “central conclusions” or “any” of the conclusions of M08. And there can be no doubt that they do.
The central claim of Mann et al 2008 was its achievement of a “skilful” reconstruction without tree rings for at least 1300 years – an achievement that seemed to rebut Climate Audit criticism that Mannian reconstructions merely recycled flawed bristlecone chronologies, the strip bark chronologies which the NAS panel had recommended “avoiding” in reconstructions. (Notwithstanding this recommendation and claims in M08 to follow NAS panel suggestions, M08 used all 20 strip bark chronologies in the MBH98 CENSORED file anyway, a point that I’ll return to. “No-dendro” in this context is really almost the same thing as “no-bristlecone”, as it’s the inclusion of the bristlecones in the “all-proxy” reconstructions that make the difference.
M08 described their no-dendro reconstruction as a “significant development”:
A skillful EIV reconstruction without tree-ring data is possible even further back, over at least the past 1,300 years, for NH combined land plus ocean temperature (see SI Text). This achievement represents a significant development relative to earlier studies with sparser proxy networks (4) where it was not possible to obtain skillful long-term reconstructions without tree-ring data.
Claims about the importance of the no-dendro reconstruction – and, in particular, for the no-dendro EIV reconstruction – occur over and over in the running text of M08:
Because this conclusion extends to the past 1,300 years for EIV reconstructions withholding all tree-ring data, and because non-tree-ring proxy records are generally treated in the literature as being free of limitations in recording millennial scale variability (11), the conclusion that recent NH warmth likely exceeds that of at least the past 1,300 years thus appears reasonably robust…
We place greatest confidence in the EIV reconstructions, particularly back to A.D. 700, when a skillful reconstruction as noted earlier is possible without using tree-ring data at all.
and again here:
We find that the hemispheric-scale warmth of the past decade for the NH is likely anomalous in the context of not just the past 1,000 years, as suggested in previous work, but longer. This conclusion appears to hold for at least the past 1,300 years (consistent with the recent assessment by ref. 2 [the NAS Panel]) from reconstructions that do not use tree-ring proxies, and are therefore not subject to the associated additional caveats
The significance of the achievement of a 1300-year reconstruction was emphasized in the accompanying Penn State press release (itself the basis of numerous stories in the specialized press and blogs):
“Some have argued that tree-ring data is unacceptable for this type of study,” said Michael Mann, associate professor of meteorology and geosciences and director of Penn State’s Earth System Science Center. “Now we can eliminate tree rings and still have enough data from other so-called ‘proxies’ to derive a long-term Northern Hemisphere temperature record.”
and again:
Ten years ago, we could not simply eliminate all the tree-ring data from our network because we did not have enough other proxy climate records to piece together a reliable global record,” said Mann. “With the considerably expanded networks of data now available., we can indeed obtain a reliable long-term record without using tree rings.”
All of the figures in M08 and all M08 archived reconstructions and verifications used contaminated data. As noted above, the impact of contaminated data is greatest on the no-dendro variations which are pervasive in M08 graphics. No-dendro reconstructions are used in 20(!) different panels in 7 different figures (Figures 2, S4a, S4b, S4c, S5, S6 and S7) and in 14 verification statistics spreadsheets. The effect on the EIV variations, as noted above, is particularly marked, but all no-dendro results are affected.
The supposed “achievement” of a “skilful” reconstruction for the past 1300 years without tree rings has fallen apart.
Discussion
The achievement of a “skilful” reconstruction without tree rings has been widely hailed as a refutation of criticism from Climate Audit and elsewhere that Mannian reconstructions were little more than strip bark bristlecones dressed up in various disguises. The recommendation of the 2006 NAS panel that “strip bark” (i.e. bristlecone) chronologies be “avoided” in temperature reconstruction appeared to add teeth to that criticism, since, if applied retrospectively, it would have de-sticked MBH98-99 and other reconstructions.
In the realclimate response to controversy over the Yamal proxy in September 2009, for example, the EIV no-dendro reconstruction of M08 was put forward as a supposed repudiation of Climate Audit:
Oh. The hockey stick you get when you don’t use tree-rings at all (blue curve)?
It has subsequently become a staple in public defence of the Stick, e.g. Skeptical Science here and here and numerous others. All such assertions rely on the supposed “skill” of the M08 EIV reconstruction. Unfortunately, the “achievement” was an illusion, as Mann et al quietly admitted in the SI to Mann et al 2009.
PNAS Policies on Retraction and SI
PNAS has the following policy on corrections and retractions:
PNAS publishes corrections for errors, made by the journal or authors, of a scientific nature that do not alter the overall basic results or conclusions of a published article. PNAS publishes retractions for major errors that may call into question the source of the data or the validity of the results and conclusions of an article. Errata are published at the discretion of the editors and appear as formal printed and online notices in the journal.
The use of contaminated sediments in M08 was, at a minimum, an “error”. This has been clearly stated by Finnish paleolimnologists Korhola and Saarnisto. Kaufman et al 2008, of which Bradley was a co-author, have already acknowledged a lesser error and issued a corrigendum.
As shown above, despite claims to the contrary, the error has a major impact on the EIV reconstruction without tree-rings and its verification statistics, both of which were relied on in the M08 assertion that they had achieved a “skillful” reconstruction without tree rings for the past 1300 years.
As noted in the lead paragraph, scientists sometimes use contaminated data. If science is to be “self-correcting”, then scientists actually have to issue corrections and, if necessary, retractions. As the example of Grand et al 2004 shows, this actually happens from time to time, but life goes on.
So too here. Mann et al 2008 meets relevant PNAS criteria for retraction. Hopefully, either PNAS or the authors will see the wisdom of retracting the article before it gets used by IPCC AR5.
Climate Audit 
262 Comments
Ouch
Let ’em hang ’emselves in AR5. At some point people will simply stop listening to claims from the team and science will finally be allowed to truly move on.
Mark
No, Mark, good practice is always best. Mann should do what’s right, as his peers have, and AR5 should be unfuddled in every way possible.
Normally I woud agree, but these guys are serial offenders in doing exactly the opposite of what is right. The only way they will get the clue (IMO) is to eventually be embarrassed to the point of acquiescence, or marginalized to the point none of their peers are willing to acknowledge them.
Mark
Re: mark t (Jul 7 12:57),
Indeed. What would a repentant and reformed CRU and MM be like? The bind moggles. As a matter of practical psycho-economics, it doesn’t seem possible.
To reiterate a metaphor I used elsewhere, they’ve gone deep into a lush but dead-end canyon, and the entrance’s walls of data and replication are collapsing in avalanches, blocking egress.
SM and other sharpshooters are on the rim, firing steadily. A bloody stampede through the narrowing exit would be the most rational response, but I can’t actually see the herd abandoning their lovely forage and financial watering holes. And any that do escape are likely to find conditions rather sparse and arid for a long, long time.
Compelling analogy. (Despite Mosh’s warning below about an analogy that gets in the way, as they often do, in discussing the complexities of human behaviour in the group they frequently advance our thinking.) All the weight of my original remark was on AR5 itself, because that was where Mark began. This isn’t about M. Mann – or shouldn’t be. It’s about the world’s policy makers and public having an honest account of climate. That starts and ends with individuals. Go on, Dr. Mann, surprise us.
I do not disagree, I just think sometimes it takes a swift kick to put things on the right path. Embarrassment is the best kick for hubris (legal kicks tend to create martyrs and worsen the situation.)
Mark
Nick Stokes,
Here the issue of Tiljander was argued in an interesting way in the comments. Please let us know your issues with the result:
You may also want to peruse this post and comments before continuing your tiresome apologetics:
http://amac1.blogspot.com/2010/08/synopsis-of-some-tiljander-related.html
By their action or inaction they demonstrate either a lack of professionalism or a surplus of hubris.
If they can’t be trusted to apologise then what can they be trusted for?
The repeated use of the word skillfull speaks volumes about their insecurity about the level of skill employed. Something about protesting too much comes to mind. Nice work Steve.
Ruh Ro-bust.
========
Steve,
Would this mean that Kemp et al. 2011 should equally be corrected or even withdrawn ?
Rule one of Mann’s work , never ever admit any error even if its as ‘missable’ as pink unicycle elephant . Knowing tree ring data was problematic , even when it come to the magic trees they used, the Mann and team really needed to find something else . Tiljander give them is and so they through away good scientific practice to achieve advocacy and ego based goals , in other words they followed standard practice for the ‘Team’ included the usual back covering and starching when it came to review .
The real shame is not their behavior but the almost total failure of their fellow scientist , climate and otherwise , calling them out for it . That is what will cost science the most in the long run .
There is very little incentive for “calling someone out” and few vehicles for doing so. Journals seem loath to publish audits or autopsies (as at this point it virtually is), as if allowing people to build upon flawed work was ok. I guess gentlemen don’t do such things.
So this is science, right here. And over at Climate Etc, from some time before 8:06am local time yesterday. After the Steig-review debacle it’s not surprising Nic Lewis chose a different vehicle. What fun.
The trouble is such a view will cuts no ice in the court of public opinion , and that matters becasue that’s where the court the politicians , whose first aim in life is to get elected and second one is to stay elected, listen to. Its a similar problem to the way everything environmental has been attached to AGW , in that like a house of cards the danger is if one falls and they all fall . It will not just be climate science but all science that will be seen to have taken the public for a ride .
So even if you ignore the issue that scientist should promote the best scientific practice as matter of course to maintain the validity of science, you still have to consider if they should do it as matter of self interest .
What’s missing is competition. Players in the research-government-media (including journals) complex have found cooperation to be much more beneficial. Sure, there’s some struggle for funding, but most of that is with outsiders trying to break in. Otherwise, the system facilitates mutual back-scratching rather than auditing.
Fundamentally, to understand what Steve has done here requires that you understand that data come from someplace and that there is meaning in the numbers, that they represent some physical process (including a process of estimation/measurement). But certain people view them as simply numbers which can be used for a purpose. In the latter case, there is no “wrong” analysis or “contaminated” data. Kind of like how Enron viewed their financials.
If contamination is the issue, and a retraction is warranted, isn’t it Tiljander who should issue it?
Re: Nick Stokes (Jul 6 17:56),
Why would Tiljander issue a retraction? It was Tiljander who pointed out the problems with the proxy in the modern period.
“Why would Tiljander issue a retraction?” Because it’s her data. If it’s a data quality issue, how can Mann retract it?
because like many things it is useful when used according to the instructions, but Mann for whatever reasons ignored the cautions on the label.
This happens a lot in climatology doesn’t it Nick? Someone measures some quantities (tree rings, sediment, etc.), then someone else uses those measurements to paint misleading pictures of temperature histories. Then all sorts of defenses are made of the faulty inference, including the suggestion that the original measurements themselves are at fault, anything to defend the internationally renowned picture painter(s) and their false inferences. I guess this is all part of the process of climatology. However, what puzzles me greatly is how anyone can confuse this activity with science.
So you are saying that Tiljander should retract because Mann used her data upside down too?
Nick, are you seriously intending to say what your words suggest? Is it really your position that Tiljander intended the data to be used the way Mann used them? Or is it that Tiljander should apologize because the data were made available to someone who misused them — in other words she should have known the data would be subject to misuse by incompetent people and should have issued a warning in advance? C’mon, we know you are a smart guy and have good things to contribute to the discussion, so why oh why do you insist on digging your heels in on every conceivable occasion to whitewash the Team’s obvious junk? It hurts your own credibility.
Re: Eric Anderson (Jul 6 20:07),
I don’t think anyone should apologise. I’m simply pointing out that in this scenario, the analogue of Grand et al is Tiljander, not Mann.
Nick, Nick, Nick. Tiljander did not make an error. In her study, the lake sediments were clearly reflecting climate except that in recent years the waters had been muddied (heh!) by road construction and agriculture. After that date, she said the sediments were NOT reflecting climate. Mann ignored the warning (metadata) and used the data anyway, including the “contaminated” (in both senses) period. His algorithm then found a nice correlation with temperature using the contaminated portion of the data (which is meaningless for climate) and flipped the valid portion upside down relative to the original interpretation of Tiljander. Capiche?
Re: Nick Stokes (Jul 6 18:36), This makes no sense.
Tiljander collects data and reports the collection. She notes that some of the collection is contaminated.
mann ignores this warning. How can she retract his mistake.
Part of the problem is, of course, the silence of the lambs, who dont want to submit criticisms. I agree that Tiljander and//or Saarnisto should have submitted a comment to PNAS. They should have formally notified PNAS of the error. Similarly Peter Brown should have criticized the incorrect use of his dendro proxies, also pointed out at the time.
To the extent that Nick’s point is that Tiljander and/or Saarnisto should have notified PNAS, I agree. However, the primary onus for correcting an error has to rest with authors of Mann et al 2008. And the responsibility is not just Mann’s. Bradley knew about the problem.
Re: Steven Mosher (Jul 6 22:10),
“Mann ignores this warning”
Wearily, again, Mann did not ignore the warning. He noted the isasues in the SI and did a calc not including Tiljander. Showed it made little difference. That isn’t “ignoring”.
I don’t think there is anything wrong with Tiljander’s paper. But this post pushes the “contaminated data” notion, with the analogy of Grand et al. That comes back to the source of the data.
Nick, none of the composite series that Mann produced failed to contain suspect data. His “non-dendro” composite contained the Tiljander series with contaminated data. Any composiye without Tiljander contained suspect Bristlecone Pine series. He had belt and suspenders in his full composite. Take out one, and his pants would not fall down (Hockey Stick remained). If you take out both suspect sets, his composites are flat, which fail his hypothesis. And he never produced any composite without one or the either of his fail-safes.
Keith, what you say is true, but is not the point. The point at issue are all the no-dendro results. They are invalid due to the use of contaminated data. This is different than belt-and-braces.
If the analogy is your hang up then forget it. All analogies are bad.
I can think of no case where contaminated data was published by one author with a warning
and used by another. there is no analogy for the practices of climate science. it is alone
in its twisted logic. here is contaminated data. We tested that it MAKES NO DIFFERENCE to the
answer, that it provides no information, so we are justified in including the bad data.
Stunning.
You know however the difference that Tiljander DOES MAKE.
All analogies are bad, but some are useful…
yes, but in this case the analogy has not been useful to drive home the main point.
people are still unclear after all this time what the real problem is. Instead of a clean
discussion of the issues, we get side arguments about retractions ( plenty of nits to pick)
side tracks about upside down, BCP, Kemp. etc etc.
The only way you will be able to drive anything home with Nick Stokes that is negative for the consensus is if you manage to find him drunk somewhere and can coax him into transportation and literally drive him home!! 8>)
I would love a link to a discussion where I am shown wrong!!!!
Nick, could you engage in just one, PLEASE???
RE: Nick Stokes Posted Jul 6, 2011 at 6:36 PM | Permalink | Reply
**If it’s a data quality issue, how can Mann retract it?**
The answer is very simple. Write a retraction(for the paper) and send it in. Cannot a PhD do something that simple? Of course if he cannot, there are intelligent people here who can help. Obviously, you do not think Mann should issue one.
You just continue your trolling.
Ah, yes. The auto-makers are responsible for the accidents drunken drivers cause. That’s the ticket.
Re: Nick Stokes (Jul 6 17:56)
It is Mann’s improper use of a partly-contaminated dataset at issue. Not contamination in general.
Tiljander did not write Mann’s paper. Tiljander has no reason to apologize on behalf of Mann. Mann can mann-up and do it himself.
Just because there’s a way to do it wrong, doesn’t mean the originator needs to apologize. By that logic, no student would ever fail a test.
By your logic…
…you expect the police to apologize to the judge on your behalf, if you exceed the speed limit. “Judge, how was I to know going so fast would cause me to slide off the road?”
…you expect the manufacturer to apologize on your behalf, if you overdose on their medication. “How was I to know that taking 200 Tylenol would destroy my organs?”
I know that some people wish the world worked that way. Fortunately, that’s a fantasy.
Re: MrPete (Jul 6 19:08),
“It is Mann’s improper use of a partly-contaminated dataset at issue.”
That’s not what it says here:
“Like Grand et al, Mann et al 2008 (M08) used contaminated data”
Grand et al withdrew their paper because the data was their own, and it was contaminated. It had no place in the literature.
What I’m pointing out is this new attempt to start a meme (like the “Upside down” one that has caught on here) of contaminated data leads to Tiljander, not Mann.
The simple facts are that Mann used a published data set as a proxy for temperature. Like all proxies, it doesn’t perfectly reflect temperature, but is affected by other things. The Tiljander data was likely affected by anthropogenic effects in modern times. Consequently, when calibrated in the normal way, it behaved in the proxy period in a way that doesn’t seem right. It probably wasn’t a good proxy for temperature. That’s it.
Mann noted problems, and did a calc in the SI which did not use this data. Maybe he should have put that in the main paper. But really, as Susan says, you folks do go on.
the calculation in the SI did not show the full extent of the problem as pointed out in the post. The major claim of the paper – a “skilful” reconstruction without tree rings for 1300 years – does not hold up once the contaminated data is removed. If science is to be “self-correcting”, the error should be acknowledged by the authors to PNAS. That seems pretty fundamental to me.
By not doing so, at least three papers subsequent to the Mann et al Reply – which was a logical opportunity to admit and correct the error – have committed the same error.
While I did not review the lengthy blogosphere exchanges on the matter, the untrue statements that the error didn’t matter have been repeated by others. Surely this is not something that you endorse. Or perhaps it is.
Re: Steve McIntyre (Jul 6 20:14),
What Mann actually said in that recent RC article was that the omission of the Tiljander data made little difference to the results. And he provided graphs to show that.
You are making a different point that without the Tiljander data the data come from a more restricted range of proxies, and so diminishes the force of Mann08. That is a rather different issue (whose significance needs to be better established IMO). But in the sea-level paper they were not claiming that the data was uniquely representative. They were simply using it as the temperature measure for comparison. For that, the omission of the Tiljander data would indeed have made very little difference.
While I refer to the realclimate article, the post is not limited to the realclimate post. If you bother reading the post, you will notice that I discuss Mann et al 2008, the Mann et al 2009 Reply and the various SI, not just the recent realclimate post. Whether there is a knock-on effect to Kemp et al is something for another day.
The key issue is whether the error affects the “central conclusions” of M08 and thus requires retraction under PNAS policies. I submit that it does.
It’s not simply a matter of “diminishing the force of M08” by reducing the range of proxies. It’s that the central conclusions are untrue without the use of contaminated data. Under PNAS policies, this requires retraction. If you are concerned about the reputation of the climate community, as you seem to be, then the paper should be retracted, rather than taking the risk that other authors innocently rely on it to their detriment, as Kemp may have done.
A nicely detailed and reasoned piece that could give an honest climate scientist or two, if such could be found, the right leverage to call for Mann 08 being pulled.
It is, of course, one of the.wormy pilings holding up the whole rotten edifice of ‘consensus’ climate ‘science’, so there’d no doubt be resistance based on the sheer fragility of the mess that’d collapse should Mann be rendered mouse.
OMG do you do anything besides look for ways to rag on Mike Mann?
Yes, isn’t it just so very tiresome, to be interested in the truth?
I’ve discussed many other topics besides Mann’s articles. The present post is because new information about Mann et al 2008 has come to light and because it is in the news because of its recent use in Kemp et al 2011.
If you think that there are any factual errors in the article, please advise me so that I can correct them.
OMG Susan, are you defending shoddy science? ROTFLMAO
Hmm, isn’t this the pot calling the kettle black? Seems to me Susan you spend most of your time ragging on Steve!
It would be nice if the errors in Mann et al 2008 had been corrected when they were first pointed out so that continued discussion of this point would be unnecessary.
Susan: After four years of reading the posts on ClimateAudit, it occurs to me that Steve McIntyre knows more about Mann’s so-called contributions to climate science than Mann himself.
If it weren’t for Steve, Mann would no doubt be parading himself around as some kind of hero in the world of climate hockey.
No one is ragging Michael Mann. Steve is just doing what his blog is about?
A quotation for you “In the valley of the blind the one eyed man is king”
For those who are pc you know what to do? 🙂
I hardly think any skeptics who make mistakes are afforded periods of recess from criticism, susan. Your pleas sound as the screams of mercy in a water balloon fight from the side that used up all its ammo.
I recognize that nobody appreciates it when it’s pointed out that they’ve used contaminated data. The perils of making such observations were eloquently expressed in this video that a reader posted up in the comments to the Sep 2008 originally pointing out the problem though its specific impact on the “skillful” EIV nodendro reconstruction was then still in the future:
I didn’t provide quotes from Korhola and Saaisto in full because the post was already long. Here is what Korhola said:
Here is what Saarnisto said:
This latter quote is interesting in respect to Bradley’s role, as Bradley was co-author of Kaufman et al where an error was admitted and Mann et al 2008, 2009 where an error wasn’t. As of Feb 2010, Bradley told Saarnisto that the use was a “mistake”. However, they denied any problem in their Feb 2009 Reply. In addition, the “mistake” was clearly recognized by the Kaufman coauthors in early Sept 2009, including Bradley. At the time, Mann et al 2009 was still in submission and there was time for it to be corrected.
The correction to Kaufman et al had been submitted to Science by Oct 20, 2009 at the latest.
Reviewing more quality work from the new AGU Fellow, still wanting to understand the Mann phenomenon. We all make mistakes but Mann seems incapable of acknowledging, correcting, and advancing beyond any of his myriad mistakes. How can he be one of the AGU’s honored exemplary scientists??
PNAS provides a formal path for whistleblowers to expose misconduct:
PNAS Information for Authors
Contact Information. PNAS, 700 11th Street, NW, Suite 450, Washington, DC 20001 USA. Phone 1-202-334-2679, fax 1-202-334-2739, e-mail pnas@nas.edu.
——————————
Office of Research Integrity, Health & Human Services
Handling Misconduct: Introduction
Handling Misconduct – Complainant
Handling Misconduct – Allegations
You know, I used to give CSI pseudonym Nick Stokes the benefit of the doubt. But his undulations in this thread REALLY take the cake. Nick, you can’t honestly be saying the Tiljander made the “mistake” here. She identified the contaminated portions of the data, and gave reasons why they were contaminated (unlike the Team, who just delete data they don’t like without giving a reason). Mann chose to use the knowingly contaminated data, and then abused the un-contaminated portion. Are you REALLY saying this is Tiljander’s fault??
Mann et al 2008 was published in Sep 2008 almost concurrent with the financial crash, when mark-to-market accounting was much in the news. The following post discussed the handling of data in Mann et al 2008 in terms of accounting practices. This was written a few days before the discovery of upside-downness. It starts:
Re: Jeff Alberts (Jul 6 21:26),
No, I’m not the fault-finder here. All I’m saying is that if there were a claim that some paper should be withdrawn because of data contamination, that would come back to Tiljander et al. But that’s not my claim.
Tiljander et al do say that varves thickened in 20C probably due to human interference in the environment. That certainly advises caution. Mann did take account of this in the SI.
Nick,
You know that a publication can have value because it shows information other than the expected. For example, it can save others from going down the same path for little return.
If you read Tiljander et al carefully, you will find that there were several events pre 1710 approx., apart from ones attributed to human disturbance of the land after then. e.g. “The Lake Korttajarvi record also indicates a climatically more severe period in the 17th century. Two periods, AD 1580–1630 and AD 1650–1710, are marked by an increase in both sedimentation (varve thickness) and mineral matter accumulation (relative Xray density). Also, magnetic susceptibility values are high between AD 1650 and 1710, indicating increasing mineral matter input into the lake.”
This quote (and others) indicate that some climate events can be reconstructed in a relative and theoretical sense, before instrumental records of 1881. Because the nexus between thermometers and varve properties was disrupted after 1881, as clearly stated, the conventional quantitative calibration of the data should not be performed for projection to earlier times.
Pre 1710, there is a relative change in properties that can quantitatively interpreted to mean that named events such as the Little Ice Age coincide in time with certain varve patterns. This is a positive contribution and it does not call for withdrawl or correction for any proper reason. (An improper reason would be an attempt to show that the MWP or the LIA did not exist globally).
So, what is your reason for proposing that Tiljander et al retract? And what is your evidence that Mann did take proper account of the pre-1710 observations?
One might have thought that several hundred past papers calibrated to recent thermometry should be retracted – or better still, recalibrated – because of the very small change in global temperature since about 1998 and the growing confusion over the magnitude of GHG effects.
Nick: https://climateaudit.org/2011/07/06/dirty-laundry-ii-contaminated-sediments/?replytocom=297549#comment-297549
So, Mann “notes it” and uses the contaminated portion anyway, and then flips it all upside down. But, because he “notes it” somewhere, that means his conclusions are valid?
You also said that he removed the contaminated portion and it made little difference. Was that while he included Bristlecones? I’ll bet it was. Would you want your life to depend on such science?
Does Nick Stokes actually proof-read his own posts? does he not realise how illogical, or worse, his current statements in defence of Mann are?
One does not have to be a scientist to understand the issues exposed by Steve and being discussed here, but logical thought seems utterly beyond Stokes, IMHO.
I just love how Nick Stokes again fails to address the point and tries to change the discussion to something else that he wants to address. Muddying the waters may work over at RC. Not here.
Mia Tiljander actually said: “ Since the early 18th century, the sedimentation has clearly been affected by increased human impact and therefore not useful for paleoclimate research. “
Not possibly nor probably. No perhaps or maybes.
‘Not useful for paleoclimate research’; how big a warning is required to stop people using, or defending the use of, this portion of the data?
Re: RobWansbeck (Jul 7 07:06),
When and where did Tiljander say that? It’s not in my version of the paper.
I’ve said elsewhere (frequently) that Mann would have been better to concentrate on what he did in the SI, not using Tiljander. But I don’t think that the overall treatment requires any kind of retraction.
In her 2005 dissertation which was based on four earlier works including the one discussed here.
Click to access holocene.pdf
“The interpretation of the Little Ice Age (LIA)
from the varve properties of Lake Korttajärvi
is not straightforward, possibly because of the
accelerating catchment erosion since AD 1720
caused by human activities, and the interpretation
of the paleoclimate is no longer reliable.
Because of the increased land use in the area,
the sedimentation rate has accelerated, however,
two severe climate periods existed in AD 1580-
1630 and AD 1650-1710, just before the major
catchment erosion events. Both periods indicate
higher mineral matter accumulation (thick varves
and high average X-ray density value) and thus
probably severe climate periods.
In the 20th century the thickness of the varves
increases towards modern times and exceptionally
thick varves can be connected to human activities
(ditching, road construction etc.)”
Here is the threshhold question.
You have a data sample that is contaminated. The author says its contaminated. her disertation says it is unsuitable for paleo work.
You test it in an A/B fashion. For what purpose.
A. the results will be the same: in which case WHY DO YOU YOU NEED IT?
B. The results will be different, in which case you should HEED THE WARNING.
Yes, that’s exactly the bit Mann quoted from in the SI:
‘‘Natural variability in the sediment record was disrupted by increased human impact in the catchment area at A.D. 1720.’’ and later, ‘‘In the case of Lake Korttajarvi it is a demanding task to calibrate the physical varve data we have collected against meteorological data, because human impacts have distorted the natural signal to varying extents’’). These issues are particularly significant because there are few proxy records, particularly in the temperature-screened dataset (see Fig. S9), available back through the 9th century.”
I’ll note that you didnt answer the threshold question.
Here is data that is contaminated that makes no difference. Why include it?
he quoted this
“and the interpretation
of the paleoclimate is no longer reliable.”
Quite so. Their decision to use this data in the face of such explicit warnings should have raised these very questions on the part of reviewers and the climate community.
What did the authors know and when? Did they know that the no-dendro EIV reconstruction failed without the contaminated data? When did they know? While this is an entirely separate question from the retraction, it’s hard not to wonder.
Steve,
Yes, it would have been a reasonable decision to just proceed without the Tiljander data.
But they did do calcs with and without, and set out the issues raised by Tiljander. So what is new here? What information not already in the paper makes you demand that it should be retracted?
Steve; for the nth time, the SI to MAnn et al 2008 did NOT show all the relevant calculations with and without the contaminated data that were relevant to their article. The ones that they left out would have showed that their central conclusions did not hold. Why they failed to show the adverse calculations in the original SI is a different question, one that I intentionally did not raise in the post.
Re: Nick Stokes (Jul 7 23:46),
You ask:
Following is an excerpt from this post that I believe answers your question:
The excerpted text is about 4 paragraphs above the graphic showing Mann 08 Figure S4a(d).
In the 2nd paragraph above the same graphic, I excerpt:
Hopefully, this answers your questions.
(All bolding in original text.)
Thank you Phil,
Yes, it does help. I have been mainly, in line with the thread discussion, responding to the headline about contaminated data and ignoring Tiljander warnings etc. So yes, I agree that with regard to that specific claim about a skilful prediction in the absence of tree-rings, there is new information in the 2009 paper that the skilfulness over the whole 1300 years may be affected by the loss of the Tiljander proxies.
I still think the demand for a retraction is over the top, though.
Except for the bit about this paper being used to prove that it isn’t just all bristlecones…. so yes the fact that the result fails means it should be retracted.
Exactly
“I still think the demand for a retraction is over the top, though.”
Then at the very least an update sent to PNAS retracting THIS particular aspect ?
This whole “verification statistics depending on X” thing is so reminiscent of the initial MM response to MBH98 isn’t it ? One of the things MM pointed out was that by removing the bristlecones the reconstruction showed comparable temperatures to today around 1000 AD. The team response was that the “MM reconstruction” failed verification. It’s amazing that Mann et al are so certain in the robustness of their reconstructions when they have such verification fragility. Verification in MBH depended on bristlecones (which are recommended not to be used as proxies for temperature at all) and in M08 on the upside down Tiljander.
It’s also worth pointing to the Mann response to the initial criticisms by MM on M08:
“The claim that ‘‘upside down’’ data were used is bizarre.
Multivariate regression methods are insensitive to the sign of
predictors.”
and
“Potential nonclimatic influences on the Tiljander
and other proxies were discussed in the SI, which showed that
none of our central conclusions relied on their use.”
Oh really ?
Many comments focus on the personal, in content and tone. This is unhelpful and uncalled for, in my opinion. The objective ought to be to improve the scientific process, not to extract a pound of flesh. Each of us grown-ups has unpleasant memories of making mistakes, and at least one of them involved painting ourself into a corner. So, less schadenfreude.
I was very struck by a retrospective on breast cancer therapy given by Norman Walmark, one of the pioneers of modern clinical trial design. He points out the wrong turns taken by surgeons of the early 20th century; some persisted for decades. This doesn’t mean they were demons, or out to harm patients. It’s hard to get to 20,000 feet, see the (a) big picture, then come back to earth and apply the lessons. Many errors in experimental design are subtle — but much easier to spot in retrospect, with the benefit of hindsight. Everybody should keep that in mind.
Lastly, reviewing conversations I have had about Tiljander/Mann08 in threads at Lucia’s, Arthur Smith’s, Bart Verheggen’s, and elsewhere, I have belatedly realized that few commenters understand proxy-based reconstructions at a level that is high enough to engage in constructive conversation about technical issues. However, many people have absorbed the vocabulary, and are able to use terms in a way that suggests insightfulness. I now think this accounts for most of the frustration I’ve experienced. Many things that seem obvious, actually aren’t.
My two cents.
Upthread, I wrote —
“I have belatedly realized that few commenters understand proxy-based reconstructions at a level that is high enough to engage in constructive conversation about technical issues. However, many people have absorbed the vocabulary, and are able to use terms in a way that suggests insightfulness.”
To clarify, I think the problem is that many scientifically-literate people of good will over-estimate their own proficiency in analyzing questions relating to proxy-based reconstruction. I have to assume that’s true of most people who deride the technical details of Mann08, as well.
But people use the field’s vocabulary in a way that makes it seem that they understand the basic concepts.
So a lot of arguments get started, and quickly get hot. But there’s no point to them.
Amac,
I think it would be nice to do a layman’s version of the issue. Absent all the personal crap about mann.
1. What did Tiljander write.
2. What did mann write 2008. what are the major claims.
3. what happens when you remove the contaminated series from the record.
With all the references to kemp and Rc and all the other extraneous crap ( who should retract) I think its time for a really simple synopsis.
send me a draft
Re: Steven Mosher (Jul 7 15:20),
Good idea.
AMac, I never saw that. I think lots of people just naturally believe Mann and disbelieve Steve McIntyre. On top of which, they are misled by Mann’s reply, into assuming things about Tiljander that are not true.
Then when you go into detail, they disappear.
I like the contrariness of Nick Stokes, I am much rewarded by the rebuttal of his remarks for they show me where the truth lies. When following the threads after his utterances it becomes very clear that he is defender of the climate faith. In my opinion I think he does show the flaws in climatology reasoning perfectly.
Alexander K (Jul 7 09:51), Here is the BEAUTIFUL thing about Nick.
In the toughest of cases you can count on Nick to show up and give it his best without getting personal.
In the absolute toughest of cases he shows up. And he tries to give an argument. Not personal attacks, or huge distractions. he shows up to defend mann better than mann can.
What’s that mean? He now always has to show up and defend. He knows we expect the good defense from him. If he doesnt show up, then we know that the case is so clear that even Nick cant figure an angle.
Except his defenses are rarely any good; passionate, maybe, but almost always lacking in substance.
Mark
No, I think nick does a good service in helping perfect a case against what Mann may have done.
I think the case needs to be laid out without reference to ancillary issues.
those ancillary topics would be: retraction and kemp and upsidedownness.
Laying the issue out clearly is hard work, but it needs to be done. without allusions, analogies,
subordinate clauses, aspersions, diversions, etc.
What did tiljander say.
What are mann’s claims in 2008.
what was shown in the SI
what wasnt shown in the SI
do his claims hold?
Everytime one of these threads starts it goes in 52 different directions. As commenters I think we have to take some responsibility for that. i’l.l still urge Amac to do a LAYMANs version of this absent convoluted sentences
AMac78, as a casual bystander with only limited insight into paleoclimate, it seems blatantly obvious that you shouldn’t use data for purposes that the author explicitly warns against, let alone use it upside-down, let alone have your entire conclusion rest on this. In fact, this seems so obvious to me that it is completely incomprehensible that anyone can possibly defend this. I am however open to information which may paint a more nuanced picture.
What did you have in mind when you wrote “Many things that seem obvious, actually aren’t.”. Do you have an example of this?
pax (9:20am) —
When I wrote “Many things that seem obvious, actually aren’t,” I was referring to the obviously disingenuous arguments that Mann08’s defenders have used. (A new one, “It’s Tiljander’s fault,” has debuted in this thread.) In my revised view, most of those folks aren’t trying to deceive or confuse. It’s just that “they don’t know what they don’t know.” To coin a phrase.
Misplaced self-confidence and confused reasoning, not disingenuousness.
The most parsimonious account of Mann08’s use of Tiljander is that the paper’s authors were inattentive to detail. In the paper and the S.I., they flagged the very problems with Tiljander that render the series uncalibratable. And then used them. That’s explained better by carelessness than by Machiavellian scheming. Driven by confirmation bias? Makes sense to me.
Natures Guide to Authors states:
Maybe Nick Stokes will argue that no such standards apply to PNAS authors. let’s see.
Stokes will respond that there was no error for Mann to correct because the error was in Tiljander’s work.
In general Stokes will pick one part of your topic, (no matter how tangential), and try to narrow the discussion onto the turf he has picked. To think that you can convince him of anything is folly. The fact that people keep biting his hook testifies to his skill.
Re: Steve McIntyre (Jul 7 10:09),
I think as far as information goes, Mann dealt with it in the SI. One can argue that some of what he said should have been in the main paper. But putting it in the SI is certainly informing PNAS, and I don’t see how he could add to that.
Nick, once again, you’re just making stuff up. Mann did not deal fully with the impact of contaminated data in the SI to Mann et al 2008. It left out the impact on the EIV no-dendro reconstruction used in their central claim of achieving a skillful no-dendro reconstruction for 1300 years.
One of two things took place at the time. First, it’s possible that Mann et al did not carry out complete tests before making his claim that the use of contaminated data didn’t matter. Or that he did carry out complete tests and the results of the unreported tests are in the CENSORED directory.
On the more generous interpretation, when Mann et al (including Bradley) learned that the use of contaminated data had a material adverse impact on one of the central claims, they had an obligation to issue a Corrigendum if not retraction, if you expect findings of climate academics to be relied upon by the wider public. It is puzzling that this is not understood within the climate community, as it is readily understood by people with experience outside academia.
Re: Steve McIntyre (Jul 7 16:50),
Steve,
as far as the impact on their central claim etc, that is for the reader to judge. Many will judge differently to you. But they have (from Mann’s SI) the information to make that judgement, and I don’t see how Mann could add to that with a further statement.
Steve – once again, Nick, you are making untrue statement, something that you do far too often. The requisite information was not in the original SI. Further, it is primarily up to the author to disclose at the journal that claims in the original article did not hold up.
Re: Nick Stokes (Jul 7 17:26),
Steve,
In the SI Mann says:
“These records include the four Tijander et al. (12) series used (see Fig. S9) for which the original authors note that human effects over the past few centuries unrelated to climate might impact records (the original paper states ‘‘Natural variability in the sediment record was disrupted by increased human impact in the catchment area at A.D. 1720.’’ and later, ‘‘In the case of Lake Korttajarvi it is a demanding task to calibrate the physical varve data we have collected against meteorological data, because human impacts have distorted the natural signal to varying extents’’). These issues are particularly significant because there are few proxy records, particularly in the temperature-screened dataset (see Fig. S9), available back through the 9th century.”
That lays it all out pretty explicitly. I don’t see what more he could say.
His calc showed that the results with and without Tiljander are similar. As you say, with Tiljander gone, the results, though similar, are backed by less evidence. That’s obvious – PNAS does not need to be told that. But a very large amount of evidence remains. A retraction is not appropriate.
1) Mann notes the problems with the sediments
then
2) Calibrates the invalid portion of the sediment record against temperature (a meaningless exercise)
then
3) this calibration leads him to flip the earlier part of the sediment record upside down compared to the published interpretation (warm becomes cool, and conversely)
when trying to show it “doesn’t matter” he takes Tiljander out but puts bristlecone back in (try reading the post) and never shows the no-dendro result without Tiljander (IIRC) which Steve showed above leads to lousy results
and this is ok with you?
Re: Craig Loehle (Jul 7 18:48),
“this calibration leads him to flip the earlier part of the sediment record upside down”
Is that true? I hear it from time to time, but my understanding is that he calibrated in the normal way, and the Finns said that it then looked to be going the wrong way in the paleo period. But despite what has been said over and over here, he didn’t “use it upside down”.
So, yet again, I think that, as he explicitly recorded, it was a proxy with problems, and would have been better dropped. But he did calcs with and without, and showed there was little difference. That seems to be a reasonable way of handling it.
Steve: Nick, you have become a serial misrepresenter. Mann et al did NOT present all relevant results with and without, thereby showing that there was little difference. They showed one case where there was little difference but, either through oversight or worse, didn’t report the most important case in the SI to Mann et al 2008, one where there was a big difference – a case that was central to their results. Please stop making statements that you know or ought to know are untrue.
“going the wrong way in the paeo period” IS upside down (cool appears as warm). Really Nick.
Re: Nick Stokes (Jul 7 19:20),
Steve,
As so often, there is a variety of clashing accusations to deal with at this site. We have Mosh saying that Mann ignored the warnings (despite that clear passage in the SI). We have Craig saying that he flipped the series (which I can see no basis for). We have the usual chorus just chanting “lies, lies, lies” or whatever. There are complaints that he used bristlecones, which have nothing to do with Tiljander. And now you’re saying that he did not show an important case.
In Mann 2008, it seems clearcut to me. He simply showed the reconstruction with and without 7 problematic proxies (mostly the Finn lakes). You’re probably referring to the 2009 paper and the figure that Amac recolored.
Well, firstly it just isn’t true that he didn’t show the results – the complaint is that he used a faint color.
But this isn’t the result with just Tiljander removed. It’s removing Finn lakes and tree-ring proxies. That really is cutting into the available information. So Mann used dotted lines to show where the reconstruction had failed validation. Seems reasonable to me. Some would argue that he shouldn’t have shown them at all, for that reason.
This thread has gone off the rails.
Nic, It will not surprise you that your argument is again from an amazingly weak position, but I wonder if there is any justification for not retracting the use of sediment data upside down?
If you do/don’t want to answer that, is there any justification for not using it twice?
Jeff,
“retracting the use of sediment data upside down”
Jeff, I think you should spell out what you mean by that. Craig too talks of “this calibration leads him to flip …”. It’s said over and over here, as if Mann did something unusual with the proxy.
But as I understand it, he simply calibrated the proxy in the normal way. The general assumption with a proxy is that if you observe correlation with temperature in the calibration period, you can use this to estimate temperatures in the past. Mann did observe correlation, but the resulting temperatures did not go as people expected. The reason suggested is that the apparent correlation was due to anthropogenetic effects.
OK, that means it isn’t a good proxy for temperature. It doesn’t mean they are using it upside down. If it is correlating with something else in the calibration period, then that means you don’t have a useful correlation with temperature at all.
But my contention is that that was all covered in the SI. In effect they said, well maybe it isn’t, so lets check what happens without it. Answer – still much the same. Steve says that he, or someone, should inform PNAS. But what could they say that wasn’t already said in the SI?
Steve: that their claim to have achieved a “skilful” reconstruction for the past 1300 years without tree rings was bogus and that they regret not notifying PNAS when they first became aware that this claim was untrue.
“and that they regret not notifying PNAS when they first became aware that this claim was untrue”
But that’s the problem. Your basis for saying the claim is untrue consists of statements about the Tiljander paper that are addressed right there in the SI.
Do you expect them to go to PNAS and say “Hey, we just read our paper, and became aware our claims were untrue. We thought you should know in case you read it too.”?
There’s nothing new there. You are just making a judgement on the basis of what was originally set out that is different from what the reviewers and editors made at the time. And the scientists – the paper, with those alleged defects apparent, has been cited 184 times since 2008.
Steve; Nick, your level of reading comprehension is either zero or you are being intentionally obtuse, like so many members of the climate community. For the n+1th time, the SI to MAnn et al 2008 did NOT show all the relevant calculations with and without the contaminated data that were relevant to their article. The information that I am relying was NOT shown in the SI to the original article. It has become available much later. Read the post and try to understand it.
.
With the help of Phil’s comment below, I’ll retract that last comment. I see the basis for it in the post. The claim of skilfulness over the whole 1300 years may be in doubt without Tiljander.
I still think, though, that the demand for a retraction on this basis is unjustified.
The problem you have Nick, is along these lines: The specific physical interpretation of the varves is, say, thicker means colder for direct physical reasons. If your program inverts the correlation and claims that it “fits” better in a statistical sense with thicker meaning warmer, then your program is, to put it simply, utterly worthless. It’s worthless because thicker IS colder regardless of what your program “thinks”. Dispute the interpretation, OK, but you’d better offer some serious and cogent reasons for your interpretation, and that certainly hasn’t been done in that case.
All you’re doing is mining data for statistical correlations, you may as well use stock market movements as a proxy for temperature if you ignore specific physical interpretations. You seem remarkably resistant to the idea that proxies can have a specific physical interpretation.
Now that it has been admitted by Mann that his conclusions cannot be supported by his data, how can you possibly not support the withdrawal of the paper. As you point out, it’s been cited close to 200 times already, and every one of those cites is now in error.
Ed,
No, the program doesn’t invert the correlation. The correlation was inverted, it seems, because recently varves also thickened, say the Finns, because of disturbance. That seems to have dominated the recent temperature effect.
That can happen with anything being tried as a proxy. You reason that it should be related to temperature, but you know that is not the only factor, and other things can dominate. That’s what the divergence issue involves as well.
I don’t believe Mann’s conclusions can’t be supported by the data. If you look through the abstract of his paper, for example, I see nothing that would be affected even if it were true that the EIV RE significance level drops below 95% around 1500 after removing both treerings and the lake varve proxies. The basic purpose of Mann08 was to compute a new multi-proxy reconstruction. That is what was being used, for example, in Kemp11. And that is what people are citing.
However, what is clearly wrong in what you said is that Mann admitted that his conclusions cannot be supported. He didn’t. An admission was put into his mouth. And I doubt that this was correctly done.
Nick Stokes:
You keep saying things like “No, the program doesn’t invert the correlation. The correlation was inverted, it seems, because recently varves also thickened, say the Finns, because of disturbance. That seems to have dominated the recent temperature effect.”
You are again quite wrong. The correlation was inverted because Mann et al made an a priori decision to invert it (or, more accurately, use it upside down to the generally accepted physical meaning). A mistake. A mistake that screening should have picked up but didn’t because of the contamination issue you mention.
From Mann’s reply to the MM comment in PNAS: “Screening, when used, employed one-sided tests only when a definite sign could be a priori reasoned on physical grounds.”
From the original paper: “The screening process requires a statistically significant (P 0.10) correlation with local instrumental surface temperature
data during the calibration interval. Where the sign of the correlation could a priori be specified (positive for tree-ring data, ice-core oxygen isotopes, lake sediments, and historical documents, and negative for coral oxygen-isotope records), a one-sided significance criterion was used.
Will you salvage a shred of credibility and accept this?
No, That’s just talking about whether to use a one-sided or two-sided significance test. It is not talking about assigning a sign itself. For the varves they used a one-sided test. If they were mistaken about the sign, it wouldn’t matter for that purpose. They’d still use the same test.
I’m interested, though, in your claim that “The correlation was inverted because Mann et al made an a priori decision to invert it “. That’s what I keep hearing (as with Craig’s “flipping”). But no-one produces any evidence.
Nick,
No, you miss the point. It is about needing to make an a priori assumption about the orientation of the series. If they didn’t invert the data, of course they would use the same test. They would just get a different result (fail instead of pass). What the quotes show is that Mann et al accept that there is a known a priori physical orientation for sediments. A violation of this physical meaning should have been picked up in screening.
As regards your point “nobody presents any evidence” about the upside down treatment, the evidence has been shown repeatedly on this blog and other places. Check this thread for example: https://climateaudit.org/2009/10/14/upside-side-down-mann-and-the-peerreviewedliterature/ Note than UC and Jeans S rechecked the Matlab code and confirmed Steve’s assertion.
Don’t trust Steve M, Jean S or UC? Ok, here is a link to a “pro-AGW” computer scientist who also checked the code and found the mistake (read it carefully, while his conclusion appears to disagree with Steve, he acknowledges he found the same error) http://agwobserver.wordpress.com/2010/06/28/tiljander/
Will you now finally admit that Mann made a mistake?
close,
All I can see on that CA post you linked is umpteen ways of saying the result looks wrong. I agree. What I’m querying is your claim that “Mann et al made an a priori decision to invert it”. I see no evidence of that in what you have linked.
Nor in the discussion with Ari Jokimaki. In fact the discussion seems to converge on this:“That is negative correlation (although not very good one) and according to you the algorithm will then flip it. All this would mean that all claims about Tiljander series being upside-down in Mann et al. would be false.”
They algorithm will flip it. I wouldn’t have put it like that but it is what I’m saying. The orientation in the proxy period is determined by the sign of the correlation in the calibration period. Not by any a priori decision of Mann.
So, Nick finally realizes (see end of thread) that contaminated values were used, and were inappropriate to use. Excellent progress!
So, Nick, if this were YOUR paper (meaning Mann08) and you had used data that you found to be compromised after the paper was published, what would your response have been?
Would you blame Mia Tiljander for publishing contaminated data (something you have espoused earlier in this thread)? If so, then what would you do?
Would you claim that the results don’t matter (Mann’s claim)? Would you claim that the data aren’t “flipped” (Mann’s claim)? Or would you do something else with this information?
Will happily await your response.
Ok, now let’s move forward in time…. Having discovered that these specific data were compromised, would you then use them again in another study? If so, why would you do so?
Bruce
“Nick, once again, you’re just making stuff up.”
Steve, as I am mostly a lurker here, I will make this as short and to the point as I can. Take it for what you will.
I’m not the best poker player in the world, but I once made a living at it for a few years. You have caught Nick making things up and making the silliest of arguments many, many times. It appears to me that his main purpose (if indeed it is not his actual job) is to divert you from what you have been doing to the team for years. I know that it would not look good to censor him like so many skeptics have been at Realclimate and other blogs, and that he would love to say that you won’t answer his questions. Claim that he showed you up, etc. My “read” on him is simply to occupy as much of your time as possible, arguing with a fool. He and his friends have been severely damaged by you and Ross, and he is trying to prevent further damage by keeping you from doing more of what you do so well. He’s also trying to always get in the last word. That’s why he makes stuff up. He has nothing else to say. If he makes something up and you don’t answer, he hopes the general public that lurks here occasionally will think he was right. Deal with him as you think best.
Hopefully someday there will be a bloggers award akin to a Pulitzer, with your and Ross’s names on them.
Thanks for what you do.
Steve, couldn’t all of Nicks posts go to a special Nick thread. It would save people a lot of time. And it wouldn’t be censorship …
They don’t know what they don’t know, yet, they know what they are doing. A sure fire Rx to induce malaise. Caveat Emptor.
=============
AMac78, I freely admit that I know very little. I do however know this:
1. Tiljander explicitly warns that the modern part of the data should not be used as temperature proxy.
2. Mann did anyway.
3. A central conclusion of Mann’s paper rests entirely on the contaminated data.
What more is there to know before one can reach the obvious conclusion that this is wrong? What is it that defenders “don’t know what they don’t know.”?
AMac78, I am as interested as Pax but probably have an even more limited understanding of paleoclimate, but I am reasonably conversant with the subject of ethics and understand quite well how using another person’s work despite warnings as to it’s limitations, then doing things to it that should never have been done, is ethically and scientifically dubious to say the least.
Your last point is so mysterious it intrigues me to the point of irritation – would you care to elucidate and put me out of my misery, please.
Re: Alexander K (Jul 7 09:51),
See my response to pax at 9:54am.
In this comment at Bart Verheggen’s site from last week, I tried to sketch out the basic concept of a proxy-based reconstruction — the approach taken by Mann08’s authors. I also urged Bart’s readers to look at the data before coming to conclusions. Link to graphs of Tiljander’s series in that comment.
(I won’t be able to comment further until this evening.)
I have become very sensitive to seeing meandering temperature reconstructions in Mann (08) over the reconstructed period and with the tacked-on instrumental record as a very independent measure that merely indicates the recent run up in temperatures – but still not connected to the reconstructed period. A closer look at the calibration and verification periods in Mann (08) shows a divergence in both tree ring and non tree ring reconstructions with the instrumental record.
While SteveM points to something not clearly shown in Mann (08) or Mann (09) in this thread, I find much clearly displayed in Mann (08) that should lead to much skepticism about the claims for these reconstructions. Certainly the original hockey stick, with its very smooth handle, has undergone some major modifications in Mann (08) and we clearly can see the problem that divergence poses in the main article and the SI.
I also judge that defending the proxy use by Mann et al. here in a lawyerly fashion is symptomatic of much of climate science in matters of AGW as is the hesitancy of proxy originators to come forward when their proxies are misused. Both developments only make me more skeptical in my general view of these matters.
Bertrand Russell showed quite convincingly that if you take something that is not true, and accept that it is true, then you can use this as the basis to prove anything, true of not.
This issue is central to science. Untrue of unproven evidence taken to be true, allowing anything to be proven, untrue or otherwise.
Mann’s paper should be withdrawn and republished with the corrections applied. This would make it clear if there is any effect on the central conclusion.
The current practice, of trying to discuss the issue based on what the corrected paper would look like is hypothetical. One might as well try to nail down water.
Russell’s illustration (when asked by a student “How can you start from 2+2=5 and prove that I am the pope?”):
If 2+2=5 then 4=5; subtracting 3 from each side we get that 1=2; Let S be the set containing both you and the pope. The size of the set is 2, but really 2=1, which means that there really is only one member of the set S. This is only possible if both members of the set are identical, ergo you are the pope.
However, in relation to your point on Mann, Russell didn’t republish his work that used the principle of isomorphism, even though it was shown to be highly questionable “principal” which Russell himself later thought was dodgy.
I hate to say it but what a great answer.
Sophistry? Or to what point was your answer?
With apologies.
For avoidance of doubt, I totally support a retraction by Mann.
Bertrand Russell admitted later in life that he had a recurrent nightmare of a librarian somewhere going through unwanted volumes and pausing with ‘Principia Mathematica’ in hand (his attempted masterwork with AN Whitehead), wondering whether to throw it in the bin. Such a public acknowledgement, after Godel, was retraction enough.
Mann needs to retract, as does Nature. But I did at least like the style of Haynes, compared to increasingly desperate Stokesians.
“Mann needs to retract, as does PNAS” that probably should have been. For now 🙂
The rapid rise around 800 AD in the non-corrupted result could be a worry for some. 1.2 C in under 250 years. Blacksmiths? Though it is a proxy…
I think Nick Stokes’s position can be easily summarised in three statements: (1) it is legitimate to use the Tiljander data either way up; (2) despite (1), Mann used this data the right way up; (3) despite (1) and (2) it is Tiljander’s fault that Mann used the data the wrong way up.
Easy. Simple. Obvious. I can’t see what all the fuss is about.
Re: Oxbridge Prat (Jul 7 13:25),
No, nothing like that. I think Mann used the proxy data in the normal way. It behaved in the paleo section in a way that doesn’t look right. That isn’t anyone’s fault. It means it probably isn’t a good proxy, and should be used with care. Mann did provide caveats. Probably he should have dropped it completely.
Ok, Nick, who’s paying you to waste our time? Your insane blathering can’t convince anyone. The mere fact that you can write sentences using more or less correct grammar is proof that you’re not stupid. So what’s the point? Surely you either have better things to do with your life or you’re getting paid.
Well, he didn’t. Ever wondered why?
If a proxy is upside down, is calibrated to the part of the proxy that is known to be garbage, and the result has a high sensitivity to including that proxy or not, this suggests that the entire analysis is trash.
If someone is doing a study on the effect of sleep deprivation on response time, and their main result depends on the single subject known to be doing cocaine during the study…
Nick Stokes: you have a few times said that Mann used the proxy data “in the normal way.” I think you fundamentally misunderstand Mann’s error. For purposes of the CPS calibration, Mann had to a priori choose the orientation of the sediment series. This is stated quite clearly. Unfortunately, he chose the orientation opposite to the physical meaning believed by Tiljander. So, yes, he used the series upside down, and yes it was a mistake. To compound the error, because the series was contaminated, it actually passed screening while upside down and was therefore included in the reconstruction. If this wasn’t a mistake, Mann had an obligation to explain why his a priori assumption was different to that of the series creator. This is not “the normal way”, it is a mistake. And Steve is correct that a corrigendum is appropriate.
snip – editorializng
I think we should not overlook the plain meaning of Matti Saarnisto’s observation: “I have always thought that this was purely a case of scientific critique, but now in the last few days I have come somewhat to a conclusion that there is some purposefulness in this.”
From Matti’s full comments and the context provided by the time sequence, the plain meaning applies not only to Michael Mann but also to Ray Bradley.
A more damning violation of scientific ethics cannot be imagined, Matti’s careful wording notwithstanding.
Matti Saarnisto’s observation is in fact here. Key witness, thanks.
When I mentioned the Mann 09 chart, I was actually looking for the CPS version of the chart, since it has been described in previous posts as voiding the robustness claims. I see Lucia has described it at collide-a-scape as extending out to .7 and -1.4
I gotta hand it to Nick Stokes. His courage to defend the indefensible with anything available to say is admirable. I just hope someday he appreciates how free he was to do that here, and how other places on the web that might agree with him are so quick to delete any dissent.
I’d agree. Generally, I think Stokes tends to prevaricate and obfuscate, but give him credit, at least he’s posting here. Given time, perhaps he’ll start to engage more constructively (we can but hope).
I’d prefer to give Steve credit for allowing him to post. It takes very little effort to knowingly obfuscate and prevaricate, as you say, but heaps of effort and patience to tolerate such nonsense as Steve does.
Mark
Nick do you have a large red nose, oversized shoes and a water squirting flower to go with this routine?
They should darned well retract M08 and the climate community should be backing Steve up on this. The paper has been demonstrated defective a hundred ways, yet like climategate, there is no stopping the advocates.
While statistically correct, Steve has inadvertently given Skeptical Science far too much credit. The bloggers at SS have neither concept nor concern about the ‘skill’ of a reconstruction.
Steve asked me not to get into cancer discussions, and I shall not, so I present the following as an example of how a leading research group can alter the establishment paradigm, game set and match. Please concentrate on the evidence and the technique of unequivocal statements – and not the topic. It’s a major example of how to say “I was wrong”, for those who need to learn.
Click to access Science-Ranking.pdf
Nick Stokes —
You have dominated the discussion in this thread. That is unfortunate, because your remarks are mostly without merit.
Your writing suggests that you are uninformed or misinformed about the key issues concerning Mann08’s use of Tiljander, and their possible implications.
I do not know what causes you do put so much effort into this project. I will speculate as to possible reasons.
* You sincerely think that you are contributing to the dialog, because you overestimate your own abilities to analyze AGW-related issues. (This is akin to “80% of drivers rate themselves as above-average in driving skills.”)
* This is your version of the Socratic Method — making misleading and false statements, and then evaluating the responses that they elicit.
* As an advocate for the Consensus positions on AGW, you dislike uncluttered threads that would point cleanly to problems with one aspect of that position — namely, “the science is settled.” By making comments spiked with misleading information, you help generate many sub-threads of the “is-so / is-not / is-so / is-not” variety. The effect is to lower the signal-to-noise ratio of the thread. For the casual reader who is unfamiliar with Tiljander, this gives the impression that there are many contentious issues that the post leaves unresolved.
* You are a dyed-in-the-wool postmodernist who thinks that any notion is as valid as any other notion. There’s no such thing as “objective” facts, and people shouldn’t be “judgmental” with respect to concepts like logic and consistency.
* You’re quite aware that your posts distract and diminish. That’s your sense of humor on display.
.
As far as the third of these five possibilities, I’ll quote from an amusing reminiscence of the famed Texas criminal-defense lawyer “Racehorse” Haynes (the link will be in a follow-on comment, to keep this one out of purgatory):
Haynes had an unparalleled record of gaining acquittals for obviously-guilty clients. So in that regard, I will not “wish you luck” in your venture here.
In closing, please don’t ask that I expand on these remarks. On reflection, you should recognize that there’s no incentive for me to do so.
Well, my comment of 5:50am earned the “Your comment is awaiting moderation” citation, despite the absence of hyperlinks. At any rate, here is the link to the post with the quote of “Racehorse” Haynes’ advice to fellow defense attorneys.
Thank you for this analysis. I also sometimes wonder why you have invested so much in the Tiljander issue, which seems to me to be very minor. But the world has so many wonders.
You might like to read my latest post below. I believe you are quite wrong in equating test results from Mann09 and Mann08. The first is a spatial reconstruction – much more demanding, and naturally returning lower skill measures. There are so many more dof. The second is a reconstruction of a global average temp. It is not an apples to apples comaprison. The test results in Mann09 cannot be expected ot match those in Mann08.
Re: Nick Stokes (Jul 8 07:21),
Nick Stokes (Jul 8, 2011 at 7:21 AM)
> Thank you for this analysis.
You are welcome.
> I also sometimes wonder why you have invested so much in the Tiljander issue
I have explained my reasons, most recently in the comments at Bart Verheggen’s site. The paragraph from C.P. Snow that I quoted (Jul 8, 2011 at 6:36 AM) is relevant.
> You might like to read my latest post below.
I responded (Jul 8, 2011 at 7:25 AM).
> I believe you are quite wrong in equating test results from Mann09 and Mann08. The first is a spatial reconstruction – much more demanding, and naturally returning lower skill measures. There are so many more dof. The second is a reconstruction of a global average temp. It is not an apples to apples comaprison. The test results in Mann09 cannot be expected ot match those in Mann08.
Pursuant to the points I raised earlier (Jul 8, 2011 at 5:50 AM): If I had established your record of contributing misleading and erroneous commentary on a topic, I would be embarrassed to follow up with a paragraph such as that. It shows that you do grasp some of the key issues in Mann08’s use of Tiljander.
While refreshingly free of ad homimems, I find your commenting style to be counterproductive to informed inquiry. Whether it is intended as such, or not.
With that, I am off to work.
At a different blog, on a different topic, commenter Kylie left this gem.
Steve,
Prompted by Phil above, I looked into your Mann09-based claim that “All such assertions rely on the supposed “skill” of the M08 EIV reconstruction. Unfortunately, the “achievement” was an illusion, as Mann et al quietly admitted in the SI to Mann et al 2009.” and many similar. And I think they are wrong.
Firstly, Mann clearly didn’t admit that. You put that into his mouth. But secondly your equating of the tests in Mann09 and Mann08 is misplaced.
Mann08 is not a spatial varying reconstruction. It looks for global effects assumed to apply to all proxies, and constructs a global average accordingly.
Mann09, in the words of its intro:“Global temperatures are known to have varied over the past 1500 years, but the spatial patterns have remained poorly defined. We used a global climate proxy network to reconstruct surface temperature patterns over this interval.”
A different task. They aren’t reconstructing just a global temperature; they are constructing a spatial distribution. Of course, that is more demanding, and the predictions with the same data will have less skill.
As with Mann08, though, they did sensitivity tests, in which they averaged the spatial distribution to get back a global average, and they tested, as you observed, its performance when dendro and Finn lake proxies were removed. This is when you observed that (in Mann09) the EIV significance dropped below 95% at about 1500.
But it is a different model and a different test. This is not apples to apples. And there is no admission there that the test applied in Mann08 went awry.
Mann sets this out in the SI to Mann09 thus (p7):
“Firstly, we note that the hemispheric and global mean skill scores are modestly lower than those reported in ref. S1 [Mann08] for hemispheric and global mean reconstructions using a simpler “EIV” implementation of RegEM where a single hemispheric or global mean series, rather than the underlying spatial field, was reconstructed from the same proxy data set. This result is unsurprising, as the RegEM CFR procedure used here is designed to optimize a spatial pattern rather than a single series or index. The differences between the hemispheric and global mean series are minor, and the main differences (see Fig. 1 of main article and also Fig S5) arise after AD 1500 where the predictor data sets differ in the two studies, as the historical records of ref. S1 were not used in this study (see “Proxy Data” subsection of “Materials and Methods” section above).
“
Note the last section – they also have in Mann09 a reduced non-dendro set (apart from lakes). I think that is minor. The main thing is the new spatial requirement.
So you can’t just take tests from Mann09 and say they invalidate tests from Mann08. They don’t.
Nick Stokes (Jul 8, 2011 at 6:55 AM)
With reference to my criticism of your style (Jul 8, 2011 at 5:50 AM): this is the first comment of yours on this thread that raises points worth pursuing.
I will restate the issues that you raise in my own words. Until the weekend, I won’t have time to examine them. Perhaps Steve McI, Geoff Sherrington, MikeN, or another knowledgeable person can weigh in.
1. There are two different sets of pre-1850 reconstructions presented in Mann08, EIV and CPS. For screening, validating, and calibrating proxies against the instrumental record of 1850-1995, which compares proxies to a global average temperature-anomaly, and which uses the 5-degree-longitude by 5-degree-latitude CRUTEM3v reconstruction?
2. In Mann09, what statistical method(s) is/are used? EIV alone? Are some Mann09 reconstructions based on CPS? For each, is screening/validation/calibration to the 1850-1995 instrumental record performed against a global temp-anomaly average, or against CRUTEM3v 5×5 gridcell boxes?
3. [Nick Stokes fails to acknowledge the following point; it’s my opinion.] The key information regarding the effect of the inclusion of the mis-calibrated Tiljander data series on the reconstructions of Mann08 and Mann09 is revealed by this comparison:
* No-Tilj/No-dendro versus Yes-Tilj/No-dendro
What does this comparison reveal for Mann08 EIV?
What does this comparison reveal for Mann08 CPS?
What does this comparison reveal for Mann09?
Comparisons that include Yes-dendro reconstructions are only distractions, until the No-dendro cases are clearly sorted out.
4. Is the No-Tilj/No-dendro to Yes-Tilj/No-dendro comparison presented in Mann09 Fig. S8 (Green vs. Blue in the figure I recolored) for EIV directly applicable to Mann08’s EIV reconstructions? If not, how do they differ?
Well, to start with the answer to 4. No! As I said, Mann09 uses EIV, but to construct a spatial distribution. More demanding and you expect less skill. It’s true that some skill will be recovered in taking a global average. The plots in Mann09 use that test.
Mann08 uses EIV, but just for a global average temp. This is expected to be more skilful. The comparisons are of the test scores. Your issue is that the EIV RE drops below significance in the no-dendro no-TJ case before 1500 when the Mann09 test is applied. There is no guarantee that it would do so with the Mann08 test.
I believe that the purpose of gridding in Mann08 is to ensure that regions are weighted approximately equally, as with Gistemp type modern indices. It does not introduce a spatial variation in the temp model. But Mann09 has that, as you can see from their many maps.
My understanding is that Mann09 does not use CPS at any stage, although I have been reading it for a relatively short time.
Re: amac78 (Jul 8 07:25),
Correction there – I don’t think it’s correct to say even that Mann09 is using EIV. It’s correctly described as a RegEM CFR procedure.
Anyone want a balloon dog?
I thank Nick Stokes for consistently highlighting and reminding us how pathetic and misleading Climate Science has been and continues to be, absent the careful, tenacious and illuminating critiques of this blog and its spin-offs.
However, it is not necessary to drink the entire ocean to know how salt water tastes. Thank you Nick. Your job is done. I am persuaded! Let me recommend to you a book very pertinent to your analysis and understanding. I’ve read it twice: On Being Certain, Believing You Are Right Even When You’re Not, by Robert A. Burton, M.D.
I sometimes think that Nick Stokes is a calmer, more polite and informed version of TCO. They certainly both can attract a lot of attention from bloggers with countervailing views which in my opinion often puts the important points of the thread in the background. I do not entirely blame Nick or even TCO for that.
Nick Stokes: “If contamination is the issue, and a retraction is warranted, isn’t it Tiljander who should issue it?”
Nick, I’m reading your posts and my jaw is hanging open with incredulity.
First of all, Tiljander published her data in order to get some valuable sedimentation data into the community. She could have left off the contaminated part at the end, but she choose to include it and explicitly note it. She also interpreted the physical data correctly and she described how to interpret that physical data. So there is nothing for her to retract since she made no claims that are either untrue or even misleading.
Now when Mann used the data he didn’t delete the contaminated part (first mistake), and he used the data upside down – explicitly in the opposite direction from Tiljanders physical explanation (second mistake). Now tell me, what possible reason could he have for using contaminated data upside down. How could that ever help him get a clearer picture.
Mann’s use of the contaminated part of the data subsequently contaminated the rest of his data. Mann’s use of the uncontaminated part of the Tiljander data upside down also subsequently contaminated the rest of his data. None of this can be blamed on Tiljander.
Now, regarding your “leaving it out doesn’t make any difference” argument. Let’s see if I can get through to you on this. Let’s say that I’m holding a ball behind my back and I deny to you that I’m holding a ball in my hands. So I show you my right hand without the ball while I hold the ball behind my back with the left. You say, “oh, you’ve got the ball in the other hand.” At that point, I put my right hand behind my back, switch the ball to it and show you my left hand. Then I say, “see, no ball in my left hand – I’m not holding the ball”. And that is exactly Mann’s trick.
He shows you a chart where the Tiljander data is taken out and says, “see, no difference”. But the reason that it makes no difference is because he has bad Bristlecone data in there that gives him the same result. If he only uses good data and then switches the Tiljander data in and out – then the Tiljander data makes a huge difference. So in a real test, with good data, the Tiljander makes a difference. It only doesn’t make a difference when bad bristlecone is used. I’m sure that you understand this but that you are pretending not to.
As far as the whole point of Mann 08 being to show that he can reproduce his hockey stick without tree ring data goes, I hope that you are not also confused about that. Steve exlained it perfectly. Do you still need more?
Tilo,
I’ve said over and over that the contaminated data isn’t my claim. It’s in the headline of this post. I think it’s just another way of trying to get a few miles out of this Mann-basher.
Of course Tiljander was right to publish the data. I’m sure it was accurately measured and reported. It wasn’t contaminated. All I’m saying there is that if it was contaminated, as this post claims, a withdrawal by Tiljander would be the remedy.
What is meant, of course, is that the data can’t be interpreted as a temperature proxy. That is a different thing.
Steve may have explained perfectly about Mann not being able to get his hockey stick without tree-ring data. But he misinterpreted the tests in Mann09. They applied to a spatial temperature reconstruction using RegEM CFR, not a global mean using EIV. They can’t be used to refute tests in Mann08.
Only a small portion of the data was contaminated.
It was contaminated by non-climatic influences such as agriculture.
It was not contaminated by Mia Tiljander sneezing over it or some similar mishap.
Feel free to play semantic games over the meaning of contaminated.
Tiljander stated that the portion contaminated by non-climatic influences was not suitable for paleo research.
Tiljander et al were able to use the remaining data to give an opinion on the evolution of Lake Korttajärvi from 7590 BC until around 1700 AD, a period of more than 9000 years.
Unfortunately the small portion of the data that was contaminated was the portion during the instrumental period that is used to ‘calibrate’ data using wiggle matching.
Michael Mann in his ‘caveat’ noted that calibrating the data during the instrumental period was a demanding task.
Mann performed this demanding task by simply wiggle matching; a process that does not work if the noise, i.e. contamination, is known to be larger than the signal.
There may well be ways to quantitatively interpret the remaining majority of the data as a temperature signal but wiggle matching with instrumental data is not one of them.
There is a joke:
Q: “How many Psychologists does it take to screw in a lightbulb?”
A: “Only 1, but the lightbulb has to want to change.”
It isn’t hard to understand, but you have to want to understand it.
>I think Mann used the proxy data in the normal way.
No he didn’t. He used it upside down. Values that represent cooler temperatures are now representing warmer temperatures. The correlations and human impact are a separate issue. For the CPS case, the data was used upside-down, in a way that warmer represented cooler. The same thing happened in Kaufman 09, and they corrected.
>It behaved in the paleo section in a way that doesn’t look right.
What do you mean by ‘it’, the proxy, or the proxy as used by Mann?
It also behaved in the modern section in a way that doesn’t look right.
The proxy as calibrated.
But meanwhile, Amac has left you on stats duty.
Well, the proxy as calibrated, is also upside down in the modern period. In the EIV version because this is the way that calibrates to temperature, and in the CPS version, because that is how Mann oriented it, with colder values pointing up. Tiljander’s paper also had colder values going up, only Mann’s algorithm interprets higher values as warm, thus upside-down usage. You keep ignoring this detail for the CPS version. You have upside-down axes, and a one-sided test, that passes because data is used upside-down.
To paraphrase Johnny Cockrane: If the dirt aint legit……….you’ve got to re-submit.
Nick,
No, for a CPS reconstruction, you have to a priori choose the orientation for the calibration. The algorithm will not (NOT) flip it. That’s not how CPS works. This has been explained to you ad nauseum.
For EIV, screening should weed out any proxy which ends up with a sign different to the physical meaning.
Nick Stockes: “Of course Tiljander was right to publish the data. I’m sure it was accurately measured and reported. It wasn’t contaminated.”
Yes, the later part of the data was contaminated. Mia Tiljander said that the later part was contaminated in her original publication. This means that she did not base any unfounded conclusions on the later part of the data. And it means that there were no corrections needed because there were never any conclusions draw that depended on the contaminated part of her data. However, Mann’s conclusions were depended on contaminated data. And if you removed the bad bristlecone data from Mann’s proxy set, then the inclusion or removal of Tiljander data makes a big difference. In other words, in Mann’s case the contamination directly effects the conclusions that he wishes to draw. And this is why Mann must withdraw his paper – because his conclusions are not supportable based upon only the good data that exists in his proxy set.
Nick Stockes: “I think it’s just another way of trying to get a few miles out of this Mann-basher.”
Good. Until Mann corrects his errors he should be bashed every day. But instead, he compounds his errors by publishing the same bad data again and again and again. How do you conclude that he deserves anything less than bashing?
“And this is why Mann must withdraw his paper – because his conclusions are not supportable based upon only the good data that exists in his proxy set.”
I think that claim is based on Steve’s inappropriate application of Mann09 RegEM CFR test results (for a spatial reconstruction) to Mann08 EIV analysis. Do you have anything else?
Nick,
nice try on distracting from Mann and companies many problems, but, you just have to try don’t you.
Nick,
This is what you are trying to bail out:
http://rogerpielkejr.blogspot.com/2011/07/why-ipcc-has-lost-trust.html
amac78 Posted Jul 8, 2011 at 7:25 AM Perhaps ##, Geoff Sherrington, ##, or another knowledgeable person can weigh in (on Mann08 & Mann09).
Sorry. To paraphrase and to agree with Prof Richard Muller of Berkeley, there are some authors whose papers I no longer read – and have not for a couple of years.
Such authors can turn the lights out on me, if they wish. I have only a handful of academic, old, peer-reviewed publications. Our global cutting edge work was kept company-confidential or filed away with Governments. However, my small team of a few colleagues who did most of the heavy lifting, and me, have created many billion $ of new golbal wealth, which is more positive than the negativity of those whom I choose not to read.
What is more, our contribution is proven to be real, unequivocal and correct.
Over to you, Nick. I’m sure you can argue that you have done more.
“I think that claim is based on Steve’s inappropriate application of Mann09 RegEM CFR test results (for a spatial reconstruction) to Mann08 EIV analysis. Do you have anything else?”
Gosh, I hope so.
As Mann himself states in your quote:
“The differences between the hemispheric and global mean series are minor,”
But what I find hilarious is exactly what you are contesting. We know that in Mann09 there is a strong difference between leaving out and including Tiljander data when the Tiljander data is added or deleted from only the good data series. We know this from Mann’s own charts. But you want to conclude that even though the results of the differences in method are minor, as Mann claims, that the results of leaving out or including Tiljander data from only the good data would be completely different for Mann08 than they are for Mann09. And since Mann has never been willing to show us that data set for Mann08, instead showing us only the data sets where either Tiljander or Bristlecones are present, you feel that you are entitled to declare that everything is fine with Mann08.
Common Nick!
Tilo,
Well, the claim in this post was:
“All such assertions rely on the supposed “skill” of the M08 EIV reconstruction. Unfortunately, the “achievement” was an illusion, as Mann et al quietly admitted in the SI to Mann et al 2009.”
If we don’t have that “admission”, then what “conclusions are not supportable” that would require retraction? Can you point to them?
Nick, my belief is that the information on the EIV no-dendro reconstruction was published deep within the SI of a different article to avoid the correct course of action – a retraction in PNAS. By burying the information in the SI to a different article, it gave Mann et al a sort of plausible deniability – if pressed, they could claim that they had disclosed the failure. However, by not retracting the article or even issuing a Corrigendum, they were able to avoid losing face. As a result, people like you end up being tricked.
The artifice was remarkably successful in delaying detection of the collapse of the EIV nodendro reconstruction. Although the SI to Mann et al 2009 was published in Nov 2009, no one noticed the connection to the EIV no-dendro reconstruction until Gavin Schmidt mentioned it in an inline comment at realclimate on July 30-31, 2010, nearly 9 months after publication. And even the full impact of this diagram was not fully appreciated until reposted at blogs in June 2011, starting with AMac’s blog. In my opinion, if you doubt my interpretation of Fig S8 as pertaining to the M08 case, take Gavin Schmidt’s word for it. Schmidt conceded – to the astonishment of Nicolas Nierenberg and others – that Mann’s EIV nodendro reconstruction failed before 1500AD without contaminated data.
And yes, there’s more. In Aug 2010, about 2 years after the original SI, Mann published verification statistics for the figure in the Mann et al 2009, which I plotted up in my post. It’s too bad that you didn’t bother reading my post prior to arguing old issues. The first case in the new spreadsheet precisely matched the corresponding case in M08. So whatever variations were claimed to have been done, didn’t affect this particular result. The third case in the spreadsheet (see my post) shows the failed RE statistics corresponding to the inset of Figure S4a(d) of Mann et al 2008. In my opinion, this proves apples-and-apples beyond any doubt. Every no-dendro graphic in Mann et al 2008 and there are 20 panels of them is affected by contaminated data – not just this one. .
But let’s say that you’re right and that Mann has not, after all, admitted that his 2008 EIV no-dendro reconstruction collapses without contaminated data, merely failure under the very slightly different setup of Mann et al 2009. So much the worse. The need for retraction of Mann et al 2008 is even greater if knowledgeable readers like yourself do not regard the information in Mann et al 2009 as constituting any form of concession in respect to Mann et al 2008. Your misunderstanding on this point will be an excellent argument supporting retraction and I hope that you express your concern to PNAS on this point.
Steve,
As I said further down to Tilo, I don’t accept the “very slightly different setup of Mann et al 2009”. The difference between constructing a global mean (M08) and a global spatial distribution (M09) is very great. You would be well aware of the spatial issue from the Antarctica work.
You need not only enough data, but good coverage. Mann09 refer to that very issue in the SI:
“This additional test reveals that with the resulting extremely sparse proxy network in earlier centuries, a skillful reconstruction is no longer possible prior to AD 1500.”
Sparsity is the problem – much more for M09 than M08.
I looked through many responses of Gavin, and he almost always correctly attributed the failure before 1500 to the M09 SI. Once, to Nierenberg, he spoke of M08 in a confusing way, speaking of the M08 network (which is of course very like the M09 network). Gavin is not an author of M08 or M09, and it would not be surprising if his off the cuff responses were occasionally imperfect.
It’s true that a spreadsheet has turned up in the M09 SI area which does appear to be from an EIV analysis of a type likely to be used in M08. But there isn’t much labelling info, and to me it doesn’t make much sense. I recognise your right plot as being from L54:72, but I can’t see what data gives the plot on the left. And the two cases which would seem to be, in order going down, all and “7 bad removed” don’t seem to be significant before 1500 either – in fact, maybe worse than the no dendro no “bad” case. You’ll probably have some conspiratorial explanation – for my part, I think we just don’t know what the file is.
“The first case in the new spreadsheet precisely matched the corresponding case in M08.” Which Fig in S08? Matches L6:24?
“The need for retraction of Mann et al 2008 is even greater…” You’ve really lost me here. You said the need for retraction was based on something in M09 disproving something in M08. Without that, what do you base the need for retraction on? And how could it possibly depend on my misunderstanding?
Re: Nick Stokes (Jul 9 06:05),
It doesn’t “depend” on your misunderstanding. Your misunderstanding is simply proof that Mann’s “hide the pea” strategy surrounding the Tiljander issue has fooled even a man of your, shall we say, inestimable intellect. Obviously, those many who have cited M08 have been fooled also. Ergo, the need for a retraction.
Nick,
Nick, please understand that if I say that data matches, it probably does and that I have a reason for my opinion. Surely by now, even my critics would concede this. Just because you say “you can’t see” that the data is precisely the same as comparable data in Mann et al 2008 doesn’t make it true. The top section of the speadsheet
http://www.meteo.psu.edu/~mann/supplements/MultiproxySpatial09/EIVResultsWitheldProxy_had.xls is a perfect match to spreadsheet 31 in http://www.pnas.org/content/vol0/issue2008/images/data/0805721105/DCSupplemental/SD3.xls.
It’s a perfect DNA match. If you don’t know something, there’s never any harm in asking before arguing the contrary. In the following comment, I’ve provided a script showing the perfect match. I’d appreciate it if you would run the script and then confirm to CA readers that the two spreadsheets are, as I said, a perfect match, thereby confirming my original point. Abject apologies are unnecessary, but would not do any harm either.
The basis for asking for retraction of M08 is that central conclusions do not hold up once the contaminated data is removed (contrary to the claims of the Mann et al Reply). I believe that this is established by the information in the SI to Mann et al 2009. However, even if the Mann et al 2009 SI does not categorically prove this (as you say), it remains my belief that the central conclusions of M08 do not hold up without the contaminated data, from *what-you-regard-as) evidence from a closely related study.
Indeed, as I said before, your doubt on this matter makes it all the more urgent that PNAS address the question of retraction. I can picture Gavin Schmidt or Mann trying to say that a retraction isn’t necessary because they addressed the problem in a “subsequent paper”. My view was that the correction or retraction needed to occur in PNAS and that trying to cooper the situation through disclosure deep in the SI of an article using apparently different methods did not suffice. Your arguments on this point are very instructive as to why authors should take responsibility for correcting their errors themselves, since innocent parties, such as yourself, can easily be confused or deceived by their failure to make require corrections and/or retractions.
library(xlsReadWrite)
parse=function(test) {
x=strsplit(test[,1],”-“)
test$from=sapply(x, function(A) as.numeric( A[[1]]) )
test$to= sapply(x, function(A) as.numeric( A[[2]]) )
test=test[,c(20,21,2:4,7:9,12:14,17:19)]
name0=c(t(outer( c(“earlym”,”latem”,”mean”,”adjusted”), c(“RE”,”CE”,”r2″), function(x,y) paste(x,y,sep=”_”) ) ))
names(test)[3:14]=name0
test[,3:14]=round(test[,3:14],3)
return(test)
}
##ALL-PROXY STATS IN M09 SI VERSION MATCH PNAS 2008 VERSION
#all-proxy case is identical i.e. apples and apples.
#this confirms that nhglful in the updated to M09 SI matches the PNAS SI on this point
#original
download.file(“http://www.pnas.org/content/vol0/issue2008/images/data/0805721105/DCSupplemental/SD3.xls”,”temp.xls”,mode=”wb”)
class0= rep(c(“character”,rep(“numeric”,4)),4) [1:19]
X=read.xls(“temp.xls”,from=5,sheet=31, colClasses=class0)
X=X[1:19,]
base=parse(X)
base[1:2,]
# from to earlym_RE earlym_CE earlym_r2 latem_RE latem_CE latem_r2 mean_RE mean_CE mean_r2 adjusted_RE adjusted_CE adjusted_r2
#1 1800 1855 0.287 -4.942 0.006 0.589 -1.571 0.001 0.438 -3.256 0.003 0.699 -1.346 0.140
#2 1700 1799 0.125 -6.291 0.017 0.525 -1.972 0.010 0.325 -4.131 0.013 0.699 -1.346 0.140
#2010
download.file(“http://www.meteo.psu.edu/~mann/supplements/MultiproxySpatial09/EIVResultsWitheldProxy_had.xls”,”temp2.xls”,mode=”wb”)
# “d:/climate/data/mann_2009/EIVResultsWitheldProxy_had-1.xls”
class0= rep(c(“character”,rep(“numeric”,4)),4) [1:19]
test=read.xls(“temp2.xls”,from=5, colClasses=class0)
test=test[1:19,]
new=parse(test)
# from to earlym_RE earlym_CE earlym_r2 latem_RE latem_CE latem_r2 mean_RE mean_CE mean_r2 adjusted_RE adjusted_CE adjusted_r2
#1 1800 1855 0.287 -4.942 0.006 0.589 -1.571 0.001 0.438 -3.256 0.003 0.699 -1.346 0.140
#2 1700 1799 0.125 -6.291 0.017 0.525 -1.972 0.010 0.325 -4.131 0.013 0.699 -1.346 0.140
range(new- base) # 0 0
Thanks – the R code pinpointed it. Sheet 31 isn’t so obvious, though.
But I’m still puzzled about what it means. The RE values are, starting from 1800-1859: 0.44 0.32 0.31 0.56 -0.41
Hardly significant to 1500, and then negative in 1400-1499. Do you know which case it corresponds to, as described in the SI?
Nick, as you observe “Sheet 31 isn’t so obvious, though”. No, it isn’t. That someone as knowledgeable as yourself was misled on the matter surely points out the necessity of a retraction. So that the point is recorded in the literature of record.
Steve, what’s less obvious to me is why you think this is so significant. A spreadsheet has turned up on the M09 SI site that clearly has material relating to M08. It’s unclear why, as the sheet doesn’t seem to be the one described by the text with the link. And you’ve established that the first part of the sheet is identical with sheet 31 on SD3.xls. But what does sheet 31 refer to?
As you’ve shown in the right bar plot in your post, the no dendro no lakes analysis loses significance at about 1500. But if you compare with the other data in the M09 sheet, it didn’t have much more with the lakes, or even with everything. So to make the argument, you really need to say what case you think the sheet is calculating.
Nick, it would be nice if you even pretended to read the posts before commenting. In the post, I stated of this data:
Sheet 31 is the data for Mann et al 2008 Figure S4a(d), the EIV no dendro land-and-ocean reconstruction,one of the central reconstructions of M08. The failed verification before AD1500 (not that Mannian verification really has any meaning other than in their own fantasy world) – but working on their internal logic – means that they no longer have a “skillful” EIV reconstruction for 1300 years, as I’ve been saying.
The thing really is a mess. And it’s the sort of mess that has gotten worse because they didn’t deal forthrightly with it right away.
Well, Steve, I’m sorry if I’m slow, but it’s not for lack of reading. How do you make that connection? Sheet 31 is labelled nhglful HAD. Sheet 32 is labelled nhglful-dendro, and the numbers in L6-24 do look much more like the numbers in the plot of S4a(d) that you have shown (although it seems, to my eye, not exactly). But in sheet 31, as I said above, the 1400-1499 number mean RE is negative, which clearly contradicts the pic.
Further, the heading in sheet 32 (line 2) identifies it with S4a(d) – NH-dendro. Sheet 31 just says NH.
So it seems to me that Sheet 32 is the likely source for Fig S4a(d). So the question remains, what is sheet 31? One would expect it to be the NH vs land/ocean series for which the RE values do not seem to be plotted in M08, nor the SI.
Steve: yes, sheet 32 (not sheet 31) corresponds to S4a(d) – the nodendro case. I used the incorrect sheet number in my late night comment, but not in my analysis. Sorry about that. Sheet 31 corresponds to M08 Figure S4a(c) -the with-dendro (with-bristlecone) case. This in turn matches to the first section of the August 2010 spreadsheet. you can track the name of the spreadsheet tab into an entry into a gridcell in the August 2010 spreadsheet as extra confirmation.
The no-dendro case of sheet 32 corresponds to the third set of information in the August 2010 spreadsheet – this is the variation with big changes.
See inline comment addition.
In another post, you note that the inset shows a running max. This is correct (and was noted in my post.) This – to say the least – is an ad hoc procedure unknown in statistical literature and deserves commentary on its own. At a minimum, the benchmark statistic for significance for the running max of multiple tries is going to be higher and perhaps considerably higher than individual tries.
This “test” is also not unrelated to Tiljander and the bristlecones. Because both are relatively long series, they are more influential in the early reconstructions than some of the later ones.
The sudden improvement in RE stats around AD1500 is also interesting. M09 says that they did not use Luterbacher gridded data as proxies (data that incorporates instrumental data and thus not a “proxy”). However, the identity of the RE stats in the Aug 2010 spreadsheet with sheet 31 (with-Luterbacher) strongly suggests to me that either the claim in M09 is untrue ( a possibility) or, at a minimum, confirms that M09 Figure S8 was done with the M08 (with-Luterbacher) setup. If it were done without Luterbacher data, my estimate is that the RE statistics at 1500 AD would not undergo such a jum – since there are no other series entering at that step that could result in such an impact.
I’ve been looking more at the S09 spreadsheet, and I agree now that the description they give in the M09 SI makes sense for the file, and that it is the M08 analysis for the EIV method, NH, calibrated against the HADCRUT land/ocean index rather than CRUTEM. They describe it thus:
“Additional significance tests that we have performed indicate that the NH land+ocean Had reconstruction with all tree-ring data and 7 potential “problem” proxies removed (see original Supp Info where this reconstruction is shown) yields a reconstruction that passes RE at just below the 95% level (approximately 94% level) back to AD 1300 and the 90% level back to AD 1100 (they pass CE at similar respective levels). So if one were to set the significant threshold just a bit lower than our rather stringent 95% significant requirement, the reconstruction stands back to AD 1100 with these data withheld. Recent work by Saltzer et al [ Salzer et al, Recent unprecedented tree-ring growth in bristlecone pine at the highest elevations and possible causes, Proc. Nat. Acad. Sci., 2009] suggests there is little reason to withhold tree-ring data however. “
I couldn’t identify the numbers quoted at first, but they are the “running max” figures. So the 94% refers to the RE level in L59 of the spreadsheet, and the 90% to L61.
Nick Stokes (Jul 10, 2011 at 6:13 AM) —
> I’ve been looking more at the S09 spreadsheet…
Thanks for posting that follow-up report on what you have found.
I believe that you are agreeing with the interpretation that SMcI offered on the relationship of Mann09’s findings in its SI Fig. S8 to Mann08’s findings in its SI Fig S8. That they share underlying data sets. And that they are highly relevant to one another.
Steve,
The running max makes sense to me, though I agree that some allowance for it should be made in the the attribution of significance. The figure of merit, whatever it is, should be monotonic. It shouldn’t tell you that your information about the 13th C is more reliable than about the 14th C, when in C14 you have all the C13 proxies available.
I would expect that it reflects more than just a revised test. If in that case the extra proxies available in C14 reduce the RE significance, then it would make sense to use the C13 subset. In which case the running max is actually correct. Of course, because you have made an a posteriori choice, that reduces the significance that can be claimed.
Amac,
No, I’m not agreeing to that. I’m agreeing that the spreadsheet that turned up in the M09 SI is info about M08. That doesn’t extend to any calculations using a spatial reconstruction by CFR, as in M09.
Nick Stokes (Jul 10, 2011 at 8:44 AM) —
> Amac, No, I’m not agreeing to that.
OK. Sorry for misinterpreting your remarks.
Nick Stokes (Jul 9, 2011 at 6:05 AM) —
You now understand that the Tiljander data series were contaminated by non-climate-related factors in the past two centuries, which renders them uncalibratable to the instrumental temperature record.
That’s progress.
You acknowledge that severe mis-calibration can lead to correlations that are upside-down with respect to the interpretations of prior authorities, as happened with lightsum and XRD.
That’s also progress.
And this comment (6:05 AM) leaves side issues to the side, focusing instead on whether Mann08’s reliance on the uncalibratable Tiljander data series has rendered Mann08’s conclusions unreliable.
That’s significant progress.
It is helpful to see your interpretation (Jul 9, 2011 at 8:13 AM) of McIntyre’s response, where he expands on the post’s assertion that the Mann09 SI spreadsheet contains data that are identical to data in the Mann08 SI. Having run McIntyre’s R script, you got the same results as he reports? How do you interpret these results: do you now think that Mann09’s SI’s methods and results are relevant to Mann08?
.
Meanwhile, in discussions with me and others, you took up Schmidt’s, Vermeer’s, and toto’s cause of “it doesn’t matter.” That would be “The inclusion of the Tiljander data series doesn’t matter as far as the Mann08’s important conclusions with respect to the reconstructions.”
In a comment last night (Jul 9, 2011 at 1:18 AM), I walked through the strong evidence in Mann09 SI Fig S8 (shown, recolored, in the body of McIntyre’s post) that the inclusion of Tiljander’s data “matters” a great deal.
1. The inclusion of Tiljander’s contaminated, uncalbratable data series improves the statistical validation of the reconstruction.
Compare the dashed portion of the Green line (no-Tilj/no-dendro) to that of the Blue (yes-Tilj-no dendro).
2. The inclusion of Tiljander’s contaminated, uncalbratable data series changes the shape of the reconstruction.
Compare the Green (no-Tilj/no-dendro) and Blue (yes-Tilj-no dendro) traces.
3. The inclusion of Tiljander’s contaminated, uncalbratable data series brings the shape of the no-dendro reconstruction into alignment with the yes-dendro reconstruction.
Compare Green (no-Tilj/no-dendro) and Blue (yes-Tilj-no dendro) to Brown (no-Tilj/yes-dendro). The Green trace is very different from the Brown trace. The Blue trace is similar to the Brown trace.
Do you think that the Mann09 SI Fig. S8 indicates that the Tiljander data series “matter” to the reconstructions of Mann09? Of Mann08?
Re: amac78 (Jul 9 08:43),
Well, Amac, the morning always brings progress. Unfortunately where I am, it’s now midnight, and if I keep going, we’ll probably go backwards. However, it’s good that it’s morning in America.
Just a PS – I haven’t run the script. I read it to locate the sheet, and established by eye that the numbers were identical. I’m now more interested in what they mean.
Nick, one important lesson of this one reconciliation is that it shows the importance of detailed archives for reconciliation. Archives that unfortunately are seldom available. In this case, the ability to show 3-digit matches was essential to reconciliation of a point that otherwise could be argued forever without resolution – like far too many climate science issues.
Amac,
I’ll just note that I still don’t agree that you can apply results from S8 of Mann09 to M08. The spatial analysis of M09 is likely to mbe much more sensitive to data loss. I agreed with Steve above that the spreadsheet from the M09 SI applies to M08, so the significance drops below 95% pre-1500 when both debdro and the 7 doubtful series are removed (for NH with HADCRUT), but there’s nothing that can be used to say what happens to the global average computed by EIV.
Nick Stokes (Jul 10, 2011 at 6:21 AM) —
> I still don’t agree that you can apply results from S8 of Mann09 to M08. The spatial analysis of M09 is likely to mbe much more sensitive to data loss… there’s nothing that can be used to say what happens to the global average computed by EIV.
I’ll await further comments on this issue, though it seems at this point that you are arguing a very weak case. M09 EIV is so irrelevant to M08 EIV that… Gavin felt he had to bring it up?
If the authors of M08 want to make this case, they can do an RC post that explores your claim. That would mean showing clearly the M08 version of M09 SI Fig. S8. The base case versus the Tiljander case; strip out the rest of the clutter. That’s No-Tilj/no-dendro (Green line) versus Yes-Tilj/no-dendro (Blue line). A second panel that shows No-Tilj/yes-dendro and Yes-Tilj/yes-dendro would be fun. While I’m making my Christmas list, how about if M08’s authors keep their colors constant, rather than swapping them around again. And it’d be nice if I didn’t have to recolor any of the lines to make sense of the figure.
This exercise would help them come up with a narrative that’s consistent within M08, within M09, and between M08 and M09. It would help you, too, I think.
I have a comment in the moderation queue (Jul 9, 2011 at 10:00 PM) that touches on this. It’s instructive to compare the recolored Mann09 SI Fig. S8 to the three versions of Mann08 SI Fig. S8. These are the original peer-reviewed 9/2/08 figure archived at PNAS.org, and the two informal updates archived at the PSU website (2/1/08 and 11/4/09). I reproduced the latter three images at the end of the “Null Hypothesis” post.
It’s a curious story, and your explanation doesn’t suffice.
Huh?
It would really be helpful if you recapped the state of play and the issues at hand.
conversations are getting unfollowable.
Steve- try re-reading my post. It is a pretty thorough re-cap of issues. IMO nothing in the comments changes the argument in the post. Nick Stokes has tried to argue that concessions in M09 do not entail comparable concessions for M08, notwithstanding Gavin Schmidt’s comments apparently doing so (which Nick describes as unauthoritative.) My position in reply is that, if the M09 concessions cannot be seen clearly to withdraw claims in M08, then all the more reason why Mann et al need to issue a corrigendum or retraction of M08 – a retraction if, as I surmise, the central conclusion of M08 to have achieved a skillful EIV reconstruction withotu tree rings for the past 1300 years does not hold without the use of contaminated data; a Corrigendum, if as Nick surmises), without the use of contaminated data, the M08 claim to have achieved a skillful EIV reconstruction without tree rings for the past 1300 years still stands (with only M09 being affected.) In my opinion, there is not a shred of evidence for Nick’s proposal, which I, like AMac, regard as nothing more than Racehorse Haynes arguing for the sake of exhaustion – done politely, but still done without evidence or a reasonable interpretation of the information.
Re: Steve McIntyre (Jul 10 12:32), Having not working with the data, its devilishly hard to keep all the papers, figures, methods, claims, counter claims, clear in my head. Amac’s recent post starts to unravel that.
There is one way of telling the story: the chronology, who said what where when. That’s a pretty confusing tale to retell.
Then there is another story: the analytical findings, that has a different flow.
But I’m starting to wrap my mind around it.
Steven Mosher (Jul 10, 2011 at 10:33 AM)
> It would really be helpful if you recapped the state of play and the issues at hand.
I’ve put up a post that summarizes what the various reconstructions “say” about the effect of including the uncalibratable Tiljander proxies. That’ll be it for now… Pattern Recognition.
Blogspot seems to be having troubles right now, so it may be inaccessible at the moment.
That’s a good start for seeing clearly what the issue is
Didn’t you read all of ClimateAudit?
How about you try reading all of AMac’s blog?
My summary:
Tiljander’s paper has colder values with bigger numbers. Mann used the data, but his program has warmer numbers bigger, so the proxy is upside-down.
Mann’s paper uses two algorithms, CPS & EIV.
EIV version is blind to sign of proxy, as it will flip something that is negatively correlated. However, you have the added problem that Tiljander proxy shows modern times to be cooling not warming due to non-climate reasons of farming or some such. So EIV takes this cooling to mean warming and also uses the data upside-down.
Other details:
There is more than one Tiljander proxy.
Main conclusions of the paper are that it’s warmer than ever, and we show this without treerings, and we have 1200+ proxies.
The treerings include bristlecones, which shouldn’t be used, as NAS agreed they are invalid.
Charts showing it’s warmer than ever were made with either bristlecones or Tiljander. Leaving out both brings the conclusion into question, and thus takes away the main thrust of the paper. Some of these charts are in the SI to Mann 2009, or on Mann’s website.
Gavin said the no-treering no Tiljander result doesn’t pass validation prior to 1500.
There is a 2009 paper which is EIV only.
Nick Stokes —
Taking your objections to McIntyre’s critique at face value, it seems likely that you don’t have a clear concept of how a proxy-based reconstruction works. So that could be a good place to start a conversation.
What follows is a thought experiment that has similarities with the circumstances of Tiljander’s lightsum data series. The issue being explored is miscalibration.
Suppose that we discover the records of the Qui-qui culture. Back in 1400, they started writing down the harvest of a particular field planted with maize, recording it in gur, a unit of weight. In that location, maize production is affected by summer temperature: cooler temperatures during the growing season lead to higher yields, all else being equal.
Happily, a weather station was established near the maize field in 1850, and temperature records are available from 1850 through 1995.
Since the record of maize harvest overlaps with the temperature record for 145 years, we can use this overlap period to try to calibrate the Qui-qui harvests with average summer temperature.
For instance, suppose that records for 1900 through 1909 ran as follows:
Qui-qui maize harvest (in gur) — 9.0, 8.5, 8.5, 8.0, 8.0, 8.0, 8.5, 8.0, 7.5, 7.5.
Weather station avg. summer temp (in deg C) — 15, 16, 16, 17, 17, 17, 16, 17, 18, 18.
This would enable us to establish that there was a very strong correlation for the period 1900-1910, where a one-degree rise in average summer temperature was associated with a 1/2 gur decline in maize harvest.
Let’s suppose that this relationship held true for the entire period of overlap, 1850-1995.
We could then propose the use of “maize harvest” as a proxy for temperature for the reconstruction period, 1400-1849. We would use the linear relationship
(temp in C) ≈ 33 – (2*(maize harvest in gur))
Of course, no proxies are this clear-cut, or this highly correlated to temperature. All valid temperature proxies have noise, and all have complex correlations to other factors (e.g. precipitation, wind, humidity) as well.
Methods of analysis will be much more sophisticated (e.g. Principal Component Analysis), but the relationship I’ve just described is at the heart of the idea of a “temperature proxy”.
During the calibration period, we have established that an increase of 1 gur in maize harvest is correlated with an decrease of 2 degrees C in average summer temperature.
This same relationship is presumed to hold true throughout the reconstruction period. This is the essential premise of the proxy-based reconstruction approach.
.
Now, consider Tiljander’s lightsum. You can visualize this data series via graphs here, and download the data as an Excel spreadsheet via a link from that post.
The numbers that follow are not real values; I chose them for illustrative purposes.
Suppose the annual lightsum thickness (in mm) 1941-1950 looked like this –
1.0, 1.0, 1.1, 1.0, 1.1, 1.0, 1.2, 1.2, 1.1, 1.2
While the CRUTEM3v gridcell temperature anomaly looked like this over these years (degrees C) –
0.29, 0.31, 0.34, 0.29, 0.36, 0.31, 0.41, 0.39, 0.34, 0.41
Clearly, lightsum thickness is increasing over this period. And clearly, the temperature anomaly is also increasing. The two are positively correlated. An increase of 0.1 C is correlated to an increase in lightsum of about 0.05 mm.
While this illustration used pseudo-data, you can inspect the actual lightsum and CRUTEM3v data for the 1850-1995 calibration period used in Mann08 at the link I gave. Overall rising lightsum, overall rising temperature anomaly.
By a more-sophisticated variant of this method, Mann08 established the positive relationship (positive correlation) of lightsum to temperature for the calibration period, 1850-1995.
This is the relationship that was used for the reconstruction period, 500-1849.
In the calibration period, Mann08’s authors found that warmer temperatures correlate to higher lightsum values.
Therefore, according to Mann08, during the reconstruction period, higher lightsum values signify warmer temperatures.
However, Mia Tiljander had a different interpretation. She suggested that a cold, snowy winter led to a deep snowpack and a vigorous spring runoff, bringing more mineral silt into the lake. A warm, less-snowy winter would cause a thin snowpack and a modest spring runoff, with less mineral material settling to the lakebed. Lightsum is the measure of the thickness of mineral material in a varve, with the remainder made up of organic material (darksum).
Colder and snowier winters lead to higher lightsum values, Tiljander claimed.
Post-1720, Tiljander cautioned that local human activities progressively overwhelmed this relationship.
More road-building, bridge reconstruction, and farming causes mud to wash into the lake, leading to higher lightsum values.
Prof. Mann and coauthors discussed this issue, but decided that Tiljander’s cautions could be overridden. They used lightsum as I have described.
In Mann08′s 500-1849 reconstruction period, the authors interpret higher lightsum values as indicating warmer temperatures.
This relationship is thus inverted with respect to the relationship that Tiljander proposed between lightsum and temperature.
More than that, the correlation that Mann08 established for the 1850-1995 calibration period between temperature and lightsum is invalid, irrespective of its orientation. During those years, the climate signal in lightsum was overwhelmed by the non-climate signals of road-building, bridge reconstruction, and farming.
Look at the graphs of lightsum and ask yourself what Tiljander asked herself: “How could I extract a climate-related signal from lightsum for the 19th and 20th centuries, given the massive contamination by local human activities?”
Her answer, expressed in her 2003 paper and her 2005 dissertation: It can’t be done.
Recall that Mann08’s answer was: higher lightsum indicates higher temperature.
Nick Stokes, what’s your answer?
Amac,
My answer is in this comment, and others. Yes, in the calibration period you regress the proxy against temperature, which establishes the co-variation. That’s all you have. You then extend backward the temperature from the proxy. If the calibration proxy wasn’t varying with temp but with other things, then bad luck. It’s a bad proxy for temp. If it happened to go in the wrong direction vs temp, the earlier proxy temp will come out “upside down”. But as you say, that just means you don’t have a correct calibration with temp. And there’s no way you can get one.
We don’t disagree on that. I’d prefer to resolve the issue of the Mann09 “admission”.
Nick Stokes (Jul 8, 2011 at 9:58 PM) —
> My answer is in this comment, and others.
You seem to have great difficulty in communicating simple concepts with straightforward words.
Your link “this comment” goes to your #297672, immediately followed by your #297692 and #297700. Both of the latter are continuations of your argument in #297672. #297700 begins, “With the help of Phil’s comment below, I’ll retract that last comment.” I cannot tell exactly which assertions you have retracted, and which you still hold to.
I also cannot tell what “and others”, supra, refers to.
In the most-recent comment (#297808; Jul 8, 2011 at 9:58 PM), you claim to understand how gross mis-calibration can lead to the upside-down orientation of a proxy. And yet this thread is littered with instances where you have challenged other commenters on precisely this point.
So it seems that you can understand and not-understand a concept, in near-real-time.
This is not a useful trait for advancing a technical discussion, if increased understanding is indeed an objective.
Amac – At this point, it is clear that Nick Stokes is just trolling. No need to feed him. The concepts surrounding the erroneous use (by Mann) of Tiljander’s data (likely good as a climate proxy until the mid to late 1800s and then rendered unusable as a climate proxy due to other anthropogenic activities) are not in anyway difficult to comprehend.
No, Amac, what I’ve challenged, eg with close, is:
“All I can see on that CA post you linked is umpteen ways of saying the result looks wrong. I agree. What I’m querying is your claim that “Mann et al made an a priori decision to invert it”. I see no evidence of that in what you have linked.”
ie people saying that Mann intervened to invert the fit. And I’ve consistently said that “The orientation in the proxy period is determined by the sign of the correlation in the calibration period. Not by any a priori decision of Mann.”
The link I gave takes me to where I’d intended. But if not, it’s my response to Jeff Id, beginning “Jeff, I think you should spell out”.
I don’t see any unclarity of language there.
My response to Nick Stokes (Jul 8, 2011 at 10:33 PM) is in my comment of Jul 8, 2011 at 10:51 PM.
Nick,
I have not had any time to read the rest of this thread. The headpost deserves a good deal of attention but spelling things out may take a bit.
Lessee,
The Air Vent wouldn’t even be a climate blog without M08 and no, I’m not doing the legwork for you.
A proxy is a thermometer so since you are the genius, you tell me how it works when the thermometer is upside down. Tell me how it is ok when the second (older) stage of M08 doesn’t pass the horsecrap anyway proxy-proxy stats used to justify it in the first place.
Sorry, not in the mood for games today, but also not impressed with your argument counselor.
Jeff,
I have from time to time been expounding my view that Mann did nothing special with the Tiljander proxy; all proxies went through the same calibration process, and it simply happens that the varves thickened recently, while the temperature went up. Probably anthropogenic, as Tiljander says. This inverse relation then carried over to the proxy period. So it’s not a good proxy. Mann said a lot about that, but stopped short of dropping it, which I think he should have.
But there seems to be a belief around that Mann deliberately turned the proxy upside down. I don’t believe that is so, and when Craig and close seemed to be saying that, I asked for evidence.
I also picked up when you said “retracting the use of sediment data upside down”. That’s more ambiguous, and maybe you didn’t mean that he actively turned it upside down.
Anyway, Amac says I shouldn’t spend time chasing that one down, and he’s probably right.
Nick, in my opinion, the original upside-down use of the Tiljander proxies in Mann et al 2008 occurred through the application of Mannomatic methods to contaminated data. In my opinion, most commentators have followed this assumptions.
However, subsequent decisions were made with knowledge of the problem. The decision not to acknowledge the problem in the Mann et al 2009 PNAS Reply was made despite knowledge of the problem. The continued use of the contaminated proxies in Mann et al 2009 (Science), recklessly ignoring the effect of the mannomatic on the contaminated data, was also done despite knowledge of the problem.
In my post, I intentionally stayed away from the conduct issues that you are trying to dredge up. My issue was that the central conclusions of Mann et al 2008 did not hold up with contaminated data and that the authors have an obligation to either correct or preferably retract the article, before it is inadvertently relied upon by third parties. And , as you have brought to our attention, it appears that a considerable number of people have relied on this article.
Re: Nick Stokes (Jul 9 06:37),
Mann deliberately used the proxy that was turned upside down by the method he used. He knows that he used that method, he knows the proxy was turned upside down, and he knows that no-dendro/no-Tilj is quite different from no-dendro/yes-Tilj.
Mann was responsible for choice of proxy and choice of method. Can’t blame the method without holding its user responsible.
“John shot Jane.” “No, John’s gun shot Jane.” “So what. John was holding the gun and John pulled the trigger.”
Do you really want to continue using the “John’s gun shot Jane” defense? It’s not doing you or Mann any favors.
The best that can be argued is Mann’s incompetence at the time. His continued willingness to pull the trigger indicates it is something more than incompetence.
Nick,
A reasonable reply.
I wonder if you are aware that a whole section of M08 was CPS based reconstruction which requires the operator to physically specify the orientation. It was used upside down there too. I don’t think Mann used it upside down on purpose the first time but he has steadfastly refused to admit error and has continued to use his reconstruction in subsequent papers – Kemp for instance. What Steve has shown here is that the no-dendro reconstruction cannot be statistically supported without the improperly inverted data. In other words, there still aren’t any known workable temperature proxies.
Getting deeper into the methods, you are right that in one method Tiljander was treated the same as the other data. The bad news is that the data is all (that is 100%) treated improperly. The big news here though is that the 800lb purple elephant Tiljander, which is obviously not temperature yet is used openly inverted in M08, without it the paper’s claims are reversed, and the climate advocacy world chugs right along as though it isn’t there.
Nick:
I will try this one more time but I think I am wasting my time as you don’t seem to be operating in good faith (your refusal to retract your bizarre comment “If contamination is the issue, and a retraction is warranted, isn’t it Tiljander who should issue it?” is indicative of your mindset here).
Let’s put this in basic terms:
1) Mann et al used a screening process for proxies
2) The screening test was one-sided in cases where a definite sign could be a priori reasoned on physical grounds
3) the sediment series were included in the cases where a definite sign could be reasoned
4) a one-sided test was used on the sediment series
5) the sediment series should have failed the one-sided screening test if the test was correctly designed (testing for the physical meaning the right way up)
6) It didn’t
Why?
I think you are mis-interpreting “one-sided test”. The term they used in the SI was
“one-sided significance criterion”. Basically, whether to test at 90% or 95%. It isn’t intended to sort out the sign of the correlation.
On the rhetorical question I raised (note the if’s) I was responding to the Grand et al analogy in the post. That was a case where scientists who originated data which was contaminated, in the correct technical test, withdrew their paper because of the contamination. I pointed out that this did not apply to the Mann situation; the experimenter who originated the data was Tiljander. Contaminated data implies that the experimental process went wrong. That is, of course, not the case here. My point was that the analogy was simply a bad one.
(Sorry- Original reply pasted in the wrong place)
Nick:
To quote John McEnroe: “You cannot be serious”. Your response is just bizarre. A one-sided test based on a physical interpretation makes no sense if you then ignore the physical interpretation when deciding on pass/fail. You use a one-sided test when you have a hypothesis about the direction of an effect. You can’t then ignore the direction.
Here’s Wiki on one-tailed and two-tailed tests – also called, as they say, one-sided and two-sided tests.
And wiki confirms exactly what I said: “and it is called one-sided or one-tailed if the null hypothesis is rejected only for values of the test statistic falling into one specified tail of its sampling distribution”
Note “one specified tail”
Yes, I now have to agree with you. It’s late here, and when I went to bed I realised that a one-tailed test indeed should produce the right tail.
However, their wording was one-sided significance criterion, and they previously spoke of 90%, so I do suspect that they just upped the test level to 95%.
However, there is another interpretation that you’ll probably prefer. They list the signs of correlation, and they include lake-beds as positive. Well, that depends on the measure, and they are probably thinking of xray density or some such. Which should actually be negatively correlated. So it’s possible that it passed because they had the wrong idea. Which I guess is what you have been saying.
As I’ve been saying, consistently, I think it was just a bad proxy, and whether it was upside down or not is irrelevant. You can’t fix it by inversion – it just isn’t varying with temperature, so getting the sign of the slope right doesn’t get the value right. It should have been omitted.
NIce to reach agreement on this point. I, for one, never suggested that the proxy be, for example, fed into the mannomatic CPS as a temperature proxy showing a modern ice age. As far as I’m concerned, the base case for M08 is running the entire script without the obviously contaminated data. This doesn’t mean that I agree that the project is rescued merely by removing the contaminated data. There are other problems some of which have been discussed here on other occasions and some of which we’re just scratching the surface of.
However, on their terms, this particular correction is required, as you now appear to agree. My own estimate of the consequences – as set out in the post – is that the consequences of re-doing M08 without the contaminated data are not minor as many statements from Mann and Schmidt and Vermeer say, but that the consequences reach to central conclusions of M08 and thus require retraction of the article under PNAS policies. Now that we’ve found common footing on one point, perhaps others will follow.
Nick Stokes (Jul 9, 2011 at 9:59 AM) —
> [The Tiljander data series] should have been omitted [from Mann08’s analysis].
Yes. That is correct.
Now you can start working through the implications. E.g. the major differences between the Green line (no-Tilj/no-dendro) and the Blue line (yes-Tilj/no-dendro) in the recolored figure in the body of the post. Discussion at comment of (Jul 9 at 8:43am).
I think we all finally agree on the major point – the Tiljander proxy should be excluded. The issue of how and why it passed a screening procedure that should have rejected it is little bit of a sideshow but the evidence continues to suggest that Mann et al simply made a mistake. I’m glad to see that Nick accepts this as a possibility.
Nick,
It is simply amazing how you will not get anything straight. It was just a BAD proxy. I am sure that Tiljander appreciates your confidence in the hard work they did creating high quality information. They documented sediments. For part of the series it is probable that it is useable and has some correlation to temperature based on their explanations and reality. Of course, as competent scientists, they documented ALL the series with the additional information that the later section was contaminated, the apllicable dates, and the reasons. This is not a bad proxy except for a particular man desperate to salvage a piece of garbage Peer Reviewed Paper. This is a reasonable proxy when used within the limits of its reality. The fact that a man could not do that is only that man’s fault. The fact that you would cast aspersions on the work of at least competent scientists to try and cover for those who aren’t reflects on the increasing desperation from your quarter to salvage something from a man’s mistakes.
KK,
Yes, it was a bad proxy and should have been omitted. Nothing new there. Here I am saying that over a year ago. How else do you think I should say it.
Nick Stokes (Jul 9, 2011 at 7:35 PM) —
Ah, thanks for that trip down memory lane. Yes, you did say that in February 2010, on that thread… among other things.
It was the twists and turns of advocates’ various defenses of Mann08 at Lucia’s Blackboard and at Michael Tobis’ blog that prompted me to write The Null Hypothesis. Unlike you, I thought it was clear that Mann08’s authors should have felt obligated to correct the errors in their paper… based on what was already known then.
When considering the questions I pose on the recolored Mann09 SI Fig. S8 (July 9, 2011 at 8:43 AM) and July 9, 2011 at 1:18 AM) — be sure to look back at Mann08 S! Fig. S8 as archived at PNAS (9/2/08), and as revised once at PSU’s website (2/1/08), and as revised a second time at PSU’s website (11/4/09).
The latter three figures are reproduced at the bottom of the “Null Hypothesis” post.
Taken together, these four figures tell a very curious story.
if (z(3,i)==9000 | z(3,i)==8000 | z(3,i)==7500 | z(3,i)==4000 | z(3,i)==3000 | z(3,i)==2000) &…
x(kk,i+1)>-99999 & x(kkk,i+1)>-99999 &…
z(1,i)>=ilon1 & z(1,i)=ilat1 & z(2,i)=corra
From the code for 2008 CPS. There are other cases which include abs(z(ia,i))>corra, and corra=.106, .128, or 0 depending on whether you are in case WHOLE, EA100 or LA100, or FULL.
Tiljander is case 4000.
Mann does not have to deliberately flip Tiljander. Simply using the data as is would cause it to be upside-down, as Tiljander uses colder = higher values, and Mann uses the opposite. It would be interesting to see if any other proxies wer properly flipped.
an intuitively named piece of code there. yuuk.
MikeN, I was amused by Martin Vermeer’s reaction when you posted up this code in a thread last year. From the way you had referenced this code, he first thought that I had written the code and made a sneering remark about the quality of the code. A little later, when he realized that Mann had written the code, he reversed his position and said that he thought that it was well written. 🙂
Interestingly of course mann makes the argument that eliminating the proxies makes the kemp paper better.
1. the proxy is contaminated.
2. eliminating it from the record improves kemp
3. it “makes no difference” in mann 08 (watever that means)
4. the flipping issue goes away if you drop the proxy.
5. nick says it should be dropped.
Siltdown mann. It’s hard to remove mistakes from science, especially when the science has been politicized.
MikeN, as I observed back in 2008, there are cases where series have different orientations depending on whether it is calibrated on the early or late period – surely an unattractive property of the method.
The impact is not trivial in some cases. It’s one of the reasons why different looking reconstructions arise from the application of seemingly different methods. martin Vermeer’s claim that he “knows” that flipping doesn’t matter does not, in my opinion, reflect authentic knowledge.
Nick Stokes (Jul 8, 2011 at 10:33 PM) —
> What I’m querying is your [Close’s] claim that “Mann et al made an a priori decision to invert it”…” ie people saying that Mann intervened to invert the fit.
With practice, you may do better at distinguishing important arguments from peripheral ones, and directing your remarks to the important ones.
[Close’s] claim that “Mann et al made an a priori decision to invert it” is not central to any point raised by McIntyre in the post. Did you provide a citation showing that is is or might be a claim of McIntyre’s? No, you did not. Can you provide that citation? No, you can not.
As you know or should know, the key point with respect to Mann08’s use of the Tiljander data series is that they discussed Tiljander03’s warnings, then disregarded them, proceeding to mis-calibrate them on the basis of non-climate-related local activities.
As you know or should know, there is no need for mind-reading skills. For instance, I assume that Mann and co-authors were not careful enough, and made a series of honest mistakes. The fact of their mis-calibrations of the Tiljander data series — and the effects of these mis-calibrations — remains.
Amac,
“…directing your remarks to the important ones…”
It’s actually something I try to do. But this claim was raised repeatedly, by Craig, by Jeff Id (I think), by close. And it’s what I think a lot of people here think is meant by “Mann used it upside down”. So yes, I do get drawn in.
But I agree with the ideal of thread relevance. That’s why I’m trying to guide you back to the issue of Mann09 and the “admission”. That is the central claim of the original post. Or at least what is supposed to be new that requires retraction of Mann08.
Re: Nick Stokes (Jul 8 23:18),
> That’s why I’m trying to guide you back to the issue of Mann09 and the “admission”.
OK. I can help you there.
The best way for you to guide me is for you to learn to say what you mean, and mean what you say.
For instance, did you say that Tiljander ought to retract her paper (Jul 6, 2011 at 5:56 PM ff)? Is that what you mean?
Did you say that Mann and coauthors did not ignore Tiljander’s warnings, and that after noting the issues in the SI, that they showed that inclusion of Tiljander’s data made little difference (Jul 7, 2011 at 5:32 PM)? Is that what you mean?
Did you say that “contaminated data” and “upside-down” are nothing more than attempts to start lame new memes, and that any fault lies with Tiljander, not Mann (Jul 6, 2011 at 8:03 PM)? Is that what you mean?
Those three illustrations are from a scan of the first part of this thread Plenty of others in the remaining two-thirds.
Again, most people who intend to participate in a meaningful technical discussion know better than to behave in this fashion.
Say what you mean. Mean what you say.
Amac,
I will follow your advice and “do better at distinguishing important arguments from peripheral ones, and directing your remarks to the important ones.”
You have previously accused me of careful speaking. I do try to speak carefully, and I did on this occasion. No, I did not say any of those things. You need careful reading, which will probably have to wait until tomorrow.
You are now putting up a smoke screen. Maybe we can try important arguments later.
> No, I did not say any of those things.
Wow.
OK, I’ll check in tomorrow.
Nick Stockes: “ie people saying that Mann intervened to invert the fit. And I’ve consistently said that “The orientation in the proxy period is determined by the sign of the correlation in the calibration period. Not by any a priori decision of Mann.”
Nick, you started out saying that if contaminated data should be retracted it should be done by Tiljander and you’ve slipped out from under that in order to retrench to every other argument that you can imagine. But this one is as bad as the rest. I assume that you are now trying to defend Mann against bad intentions by claiming that it was the algorithm that selected the sign, not Mann. This argument seems to fit with Mann’s “bizzare” defense. So let’s assume that it is the case. Let’s assume that it was the Mannomatic that made the mistake. Fine, no problem. But this excuse simply will not wash for the second and third time Mann published the data. At those points he could not say that it was the algorithm, because he knew by then what Tiljanders physical interpretation was. He knew that the later part of the data was contaminated. And he knew that he had used the data upside down. But he never corrected his error and he knowingly used the bad data again – just like he did with the bad bistlecone data. So spare me the potestation for the innocence of his intentions – they were not.
Nick Stockes: “If we don’t have that “admission”,”
But we do have the admission. Gavin made it. The 09 SI shows that Tiljander makes a difference when only good data is used before swapping Tiljander in and out. You are being absurd in insisting that the flavor of a method used, that changes very little, means that Tiljander can matter for one flavor of algorithm and not for another one that is very close. Mann doesn’t make that argument. Gavin doesn’t make that argument. You created it out of nothing. And it’s an irrelevant argument in any case. It doesn’t matter if Mann made an implied admission or if he made no admission. The fact is that the Tiljander data makes a difference and the Tiljander data is upside down and contaminated, so the papers conclusion are false and the paper must be withdrawn.
Tilo,
It’s not a question of flavor of algorithm. Mann09 was determining a global spatial temperature distribution. See their Fig 2 for the pattern for the MCA (850-1250ad) for example. It’s not high resolution, but you need a lot more data than you need for a global average. And when data runs low, the plot gets ragged.
That’s what you’re seeing in the Fig S8 shown here when all the dendro and lake varve data is removed. Before 1500 there isn’t enough to resolve a global spatial resolution well. That doesn’t mean there isn’t enough for a single global temperature.
Re: Nick Stokes (Jul 9 00:50),
> It’s not a question of flavor of algorithm.
Right.
With the Mann09 reconstructions, it’s possible to do the meaningful comparison — shown in the body of this post as the recolored Mann09 Fig S8.
The case without grossly mis-calibrated proxies is the Green line.
Even Nick Stokes has agreed that the Tiljander proxies were grossly miscalibrated (Jul 8, 2011 at 9:58 PM)
To see the effects of the addition of the grossly mis-calibrated Tiljander proxies, we look at the Blue line.
The trace of the Blue line is very, very different from the trace of the Green line.
Therefore, Tiljander “matters” for Mann09.
Nick Stokes doesn’t agree with Gavin Schmidt’s citation of Mann09 results in discussing the effect of the grossly mis-calibrated Tiljander proxies on the results of Mann08. So, to do the comparison, he will have to direct us to the proper figure.
Meanwhile, we can notice two very strange effects of adding bad Tiljander data to the other non-dendro proxies.
The first is that the Green line (no-Tilj/no-dendro) fails validation for all points prior to 1500. But the addition of the corrupted data improves validation statistics. The Blue line (yes-Tilj/no-dendro) only fails earlier than 700. Huh?
The second is that the Green line looks very different from the traces of reconstructions that include dendro information, and lack the Tiljander proxies (Brown line; hidden under Red and Blue for most of the Recon). But — addition of the corrupted data improves the resemblance of the no-dendro reconstruction to the yes-dendro reconstruction. In other words, Blue resembles Brown much more than Green resembles Brown. Huh?
Surely there’s something very odd afoot when the addition of data that is known to be corrupt has the effect of improving a reconstruction!
Don’t feed the troll.
Spot on Ray, don’t feed the troll, ignore it.
This thread has been as fascinating read. It exemplifies the human capacity for denial. Even the simplest arguments, requiring only a minimal amount of logic and common sense to understand and which would be readily accepted in other contexts, are denied. Mind-boggling, but also quite educational.
Nick:
To quote John McEnroe: “You cannot be serious”. Your response is just bizarre. A one-sided test based on a physical interpretation makes no sense if you then ignore the physical interpretation when deciding on pass/fail. You use a one-sided test when you have a hypothesis about the direction of an effect. You can’t then ignore the direction.
Nick Stokes: “That’s what you’re seeing in the Fig S8 shown here when all the dendro and lake varve data is removed. Before 1500 there isn’t enough to resolve a global spatial resolution well. That doesn’t mean there isn’t enough for a single global temperature.”
If S8 applied to Mann09 you would still have Mann’s statement:
“The differences between the hemispheric and global mean series are minor,”
But obviously it applies to Mann08. So what part of Steve’s statement don’t you understand?
“It’s a perfect DNA match.”
And this was also confirmed by Gavin.
So, let’s have it Nick. Tell us that Mann08 should be withdrawn because it clearly fails to support the claims that it makes. Just one lucid moment of honesty, Nick. Are you capable?
Nick says:
This just in– Hell has frozen over!!
Nick Stokes: “As I’ve been saying, consistently, I think it was just a bad proxy, and whether it was upside down or not is irrelevant. You can’t fix it by inversion – it just isn’t varying with temperature, so getting the sign of the slope right doesn’t get the value right. It should have been omitted.”
This is pretty obvious to everyone, Nick. Why do you think that it wasn’t obvious to Mann? The answer is that it probably was. But Mann was so desperate to use it, because of the strong hockey blade, that he didn’t really care about the rest of the lack of correlation with temperature. Once he had a non-tree proxy with a strong hockey blade he could play his games with the bristlecones and say, “see, I don’t need them to produce a hockey stick.”
“This is pretty obvious to everyone”. Except the peer reviewers, apparently.
What’s the point of offering ad hominems towards the authors of Mann08 or Mann09? State-of-mind is unknowable. Ascribing motives moves the discussion towards a guessing game, in which any assertion will be seen by onlookers as just as valid as any other assertion.
What matters is the work that was presented, its strengths and weaknesses. Inquiry focused on that subject can pay dividends.
My own belief is that the simplest and therefore best explanation for Tiljander-in-Mann08 is that the authors did not take sufficient care. They thought they did, by flagging the contamination issue in the text. But they didn’t.
If I’m wrong: I don’t know and it doesn’t matter. Heh. To coin a phrase. “It doesn’t matter” in the sense that, at this point, there’s no persuasive evidence, one way or another. Assertions about Prof Mann’s mindset are just examples of confirmation bias. To coin another phrase.
.
I’ve been critical of Nick Stokes’ debating style, even bringing up “Racehorse” Haynes (Jul 8, 2011 at 5:50 AM). But truth be told, by pushing back on technical points (he does read and understand the nuts and bolts, when he wants to), Nick Stokes has advanced the discussion more than the rest of the commentariat. Perhaps not always in the way he had hoped!
And Nick Stokes’s remarks are almost entirely free of ad hominem. That is admirable, and nearly unique in the context of debate on AGW-related issues.
Amac: would agree with your assessment that this interaction with Nick has had value…. I personally have a greater understanding (although still not yet complete) of what was done in Mann08 and 09 than i did prior to this thread. And, it IS refreshing to see discussion about the technical details without the use of ad hom responses, and he is capable and willing to actually review methods and run the data himself…sometimes doing so only after being openly challenged.
However, what is maddening about Nick’s style of debate is his select ability to be genuine… It seems his initial reaction to much that is pointed out here and elsewhere in the blogosphere is to be contrarian first without being initially thoughtful and objective.
I would hope that Nick stays around and keeps pushing back, but to do so in a more considered manner.
Bruce
snip – speculation on mindset
Steve: I intentionally refrained from discussing mindset in my post. There are technical issues to deal with. Prior to Climategate, I was very consistent about snipping discussion of mindset. Obviously post-Climategate, it’s impossible not to avoid. Nonetheless, there are advantages to not talking about mindset all the time and I’m going to require further comments in this thread to avoid mindset speculation.
In 2009, CA readers may Gavin Schmidt’s view on the urgency of correcting errors. CA readers may recall that on Superbowl Sunday 2009 I had mentioned that there was an error in the Harry station data at the British Antarctic Survey. I went and watched the football game. The next day, I discovered to my surprise that British Antarctic Survey had changed its data from what Steig had used – fixing the Harry error, but not leaving the original in place for those interested in replicating the Steig study.
I learned the next day at realclimate, that, by a remarkable coincidence, someone had “independently” discovered the error and had already reported it to the British Antarctic Survey – the BAS later thanking this other person. Gavin Schmidt criticizedg me for delaying even 12 hours in notifying BAS, stating at realclimate:
and again:
He explained that the error had been “independently” discovered by “people” Sunday night whose ethics obliged them to immediately notify BAS of the error:
See CA post here for a recap. Needless to say, the mysterious “people” who had “independently” discovered the error were none other than Gavin himself, who had learned of the problem from the Climate Audit discussion. See here. There was no “independent” discovery of this error. Schmidt had made use of ideas and information from Climate Audit without giving Climate Audit proper credit. His actions met the criteria for plagiarism.
However, I mention this incident for a different reason today. In the Harry case, Schmidt felt that it was so urgent that errors be corrected that he seemed to feel that a delay of even a day in correcting the error would be too long. (Though strangely he wasn’t worried about an identical error in the GISS database that remained uncorrected for some time afterwards.) Schmidt has been aware that M08 contains an erroneous claim about the skill of the no-dendro reconstruction for at least a year or so but, strangely, has taken no steps to notify PNAS of the error.
Borrowing Gavin’s words, his view seems to be that PNAS would be “grateful if someone told them about the data mess up”. The more that I think about it, the more inclined I am to comply with Gavin’s suggestion. If I do so, I will, of course, provide proper credit to Gavin for his important role in uncovering the “data mess up” of M08.
And, in relation to Mann09, once it was determined that the Hockey Stick could not be validated prior to 1500ce, Gavin then lost interest in the MWP, saying something to the effect that it really wasn’t important.
If paleoclimatoogy were an engineering discipline, there would be a theory of proxies and standards for acceptability of proxies. Is there a specific litierature on proxies and their acceptability for use?
I am only asking this because of the importance of AGW and paleoclimatology in addressing it. If the IPCC is organized to provide an assessment on such an important issue, the question occurs to me why it is not, as part of its duties, organizing the research to make the field rigorous. Why is not initiating and sponsoring research to provide theories to answer these foundational questions? The Tijlander question could be resolved by use of a standard derived from a theory of proxies
Re: Tom Gray (Jul 11 06:06) —
> Is there a specific literature on proxies and their acceptability for use?…
Yes, there is an extensive literature on using data series as paleoclimate proxies, starting with tree-rings.
Recently, ecologist Jim Bouldin kindly offered a synopsis of the field’s approach in comments to my post, Voldemart’s Question.
As far as Mia Tiljander’s observations and analysis on the lakebed sediments she collected, they are contained in her 2003 publication in the journal Boreas. The PDF is available via Climate Audit, or see the References page on my blog.
> The Tiljander question could be resolved by use of a standard derived from a theory of proxies.
Yes. This brings to mind the aphorism “In theory, theory and practice are the same. In practice, they are different“.
The contrast is between Tiljander03’s authors and Mann08’s authors.
* Tiljander et al saw the data series describing the sediments as descriptors of physical properties of those sediments, related to particular climate-related and non-climate-related processes.
* Mann et al tried to do the same. However, in practice (in my opinion), they treated the data series abstractly, as a set of numbers. Their use of these data series was thus inconsistent with Tiljander’s analysis, and inconsistent with any reasonable interpretation that gets back to physical processes, e.g. sediments settling to the bottom of a lake in Southern Finland.
To my knowledge, the authors of Mann08 and its defenders haven’t offered any explanations that rebut my claim in the prior paragraph. Instead, the authors have focused on details of their computer coding. Defenders have also offered arguments that amount to “I don’t know, and it doesn’t matter.”
I think one quick route to evaluating this brou-ha-ha is to consider the question,
“Can the Tiljander data series be meaningfully calibrated to the instrumental temperature record, 1850-1995?”
I am essentially certain that there is a simple answer to that direct question. It is “No“.
A follow-on would be, “If the Tiljander data series cannot be directly calibrated to the instrumental temperature record, how can they be used in reconstruction methods that rely on direct calibration?”
Responses akin to “I don’t know, and it doesn’t matter” aren’t very compelling. In my opinion.
There’s a recent post by Gavin in which he criticizes Soon for having files on his personal site that show data at odds with his published paper.
Steve: http://www.realclimate.org/index.php/archives/2011/07/how-soon-is-now/
I sometimes agree with Gavin’s criticisms. In this case, he seems to be claiming that Soon concealed information about Churchill temperature, which should have been used instead of Frobisher/Iqaluit.
What offends me (and others) is Schmidt’s rampant and systemic hypocrisy whereby even worse conduct by his associates in widely relied upon articles is condoned or turned a blind eye to. Schmidt’s hypocrisy is astonishing.
If Soon was wrong to use Frobisher/Iqaluit after this was criticized by a reviewer, how much worse was Mann’s continued use of Tiljander?
Though in Soon’s case, when I consulted the original paper, it contained extensive discussion of Churchill temperatures. SO I don’t understand the premise of Schmidt’s criticism other than throwing mud against the wall.
Your link to the posting at “Voldemart’s Question” is not functional. Would it be possible for you to repost it.
Of course if there is an accepted theory on the suitability of proxies, the question naturally arises about why it is not being applied in this case. Contamination and calibratablity would seem to be obvious aspects used to evaluate proxies in such a theory
> Your link to the posting at “Voldem
aort’s Question” is not functional.Link, or search the web for “Voldemort Tiljander”.
I read the post at “Voldemort’s Questions” and read the comments from Bouldin and others. I saw some rules of thumb about proxies and proxy selection ( i.e. select proxies where one thinks that the parameter of interest will be predominant). However I did not see any theory or set of standard which would relate any measured parameter or sets of parameters to proxy suitability. It just seemed to be up to the judgement of the investigator. The difficulty in doing studies based on the analysis of proxies which the investigator is free to select or ignore is, to my mind, obvious.
There must be some means of eliminating this danger of bias. This could be as I suggested a set of standards that can be applied mechanically to proxy data. Or if such a standard is not possible then an external expert evaluation for proxy suitability analysis. Blind studies have been found to be essential in medical research. Having an investigator both chose and then analyze the proxy data in paleoclimatology seems, to my mind, to ignore the lessons that demonstrated the necessity of blind studies.
The analogies of paleoclimate bias to those identified in “evidence-based medicine” have been discussed at CA in the past, though not for a few years. Ontario is a bit of a center of the evidence-based medicine movement and one of its proponents (Guyatt) even comes from a squash-paying family,
Tom,
this brings us back to the question of what is science. One of the issues whether it is repeatable. I believe that this is quite important in the question of proxies.
Did Tiljander provide us with enough imformation so that we can repeat the collection and replicate his findings. I believe he did. In that case let’s make the assumption that he did an appropriate job of collecting the samples. We then need to verify the analysis and interpretation of the analysis. Again it would seem that Tiljander provided enough information to help us understand how to analyze the material and come to the same conclusion by providing the research into the changing conditions for which he determined that the modern period should be considered contaminated. We can repeat this data acquisition also. So in summary, we COULD repeat the process if we desired. Apparently there is no real questuion that what Tiljander did was appropriate or not. He probably did good science and his information should not be slandered without someone going out and attempting to replicate ALL the work.
Since what Tiljander did was appropriate and would appear to be in accordance with scientific principles, what is the problem? For anyone not a believer like Nick and others, it is obvious. MM used the proxie in a manner most useful to support his bias and not appropriate to what the proxie actually represents. This is similat to the Yamal rings and stripbark bristlecones. They MAY be useful information, but, the way they are used is NOT in accordance with the best scientifc principles as they are jammed into innapropriate studies using incorrect procedures.
Using the modern period to CALIBRATE the proxie can be appropriate, but, ONLY if the proxie can be considered to be relatively consistent in properties for the full period it represents. For many proxies this may be a reasonable assumption. For the Tiljander series prized by the Team this is NOT a reasonable assumption due to the documented KNOWN contamination during the calibration period.
There really is no issue with the proxy. There are a lot of issues with what MM [Mann] does with it.
kuhnkat;
Proxy. not Proxie, pls. Looks too much like “pixie”. 😉
As for Tiljander, he’s a she. Name of “Mia”. :p
Tiljander “did the right thing” with the bad data. MM didn’t.
pax Posted Jul 9, 2011 at 5:44 AM
Welcome to the Alice Through the Looking-Glass world of Climate Science, where everything is inverted, including logic.
There are three time series that were archived by Tiljander03 co-author Antti E.K. Ojala at the NOAA Paleoclimatology Program: Lightsum (Mineral), Darksum (Organic), and X-Ray Density (XRD) (FTP site). A fourth time series, Total varve thickness (Thickness) was also archived; I am not sure of the original location.
Mann08 and Mann09 use all four of these time series as proxies for paleotemperature. Even if these data series could be meaningfully calibrated to the instrumental temperature record — they can’t — the use of all four series as independent proxies could not be valid.
This is because two of the time series (Lightsum and Darksum) are calculated from the two measured time series (Thickness and XRD). I discuss this further in this post.
2 Trackbacks
[…] Dirty Laundry II: Contaminated Sediments […]
[…] […]