Rule N revisited

PCA was also performed on certain proxy sub-networks (spatially dense regional networks of tree-ring data available separately in different continents) as means of dimensional reduction of the predictor network. In this case, the procedure was performed separately for each independent step of the stepwise calibration/reconstruction procedure described in “3” below. A decreasing number of PCs of these sub-networks are retained increasingly further back in time, as dictated by application of objective selection criteria (consideration of results of both Preisendorffers Rule N and Scree test).

MBH98 Corrigendum

We employed a standard, objective criterion for determining how many PCs should be kept for each region.

Michael E. Mann in his book.

Let’s continue with another housekeeping post.

When Steve and Ross uncovered Mann’s flawed PCA, Mann’s defence was that MM had failed to use Preisendorfer’s Rule N in the selection of the PCs in the NOAMER AD1400 step (for the background of this story, see here or read it from Montford’s excellent book). Steve observed immediately that the use of “Rule N” was not mentioned in MBH98 in connection to the tree ring PCA. He then emulated the claimed rule convincingly showing that it was pretty impossible that the rule was actually used. Further, up to now Mann has failed to reproduce any code or documentary evidence for the supposed use of the rule. It is hard to imagine, even in the context of the Hockey Stick, any other argument with so little support but which is still alive and well with the usual suspects. In fact, this fairy tale now seems to be the official story line in Wikipedia (citing Mann’s book, of course).

When I got my hands in the MBH9X file archive contained in the Climategate files, among the first things I checked was that if it contained the code for the selection rule, or even a file indicating the use of such code. Nope. Later I observed a curious thing in the files. MBH9X is a stepwise procedure, and in every step (if there were enough proxies), one supposedly (as the Corrigendum statement indicates) calculated the tree ring PCs. However, there were quite a few calculation steps missing from the archive. For instance, Stahle & Cleveland Oklahoma/Texas (STAHLE OK) precipitation chronologies had a single calculation step (AD1700) although they were also used in the later steps (AD1730, AD1750, AD1760, AD1800, and AD1820). I then checked the actual MBH98 data, and vola, I noticed that instead of calculating the PCs for the STAHLE OK network at, say, AD1820 step Mann had simply recycled the 1820-1980 part of the AD1700 step PCs (1700-1980)! The correspondence between the archive and the PCs actually used was almost one-to-one: every “missing” step in the archive matched the reuse of the PCs from the previous step in the MBH98 data. In other words, contrary what is claimed in the Corrigendum PCs were not calculated for every step.

So what has the above to do with the PC selection rule? Well, Steve had observed that there are three (*) cases (SOAMER AD1750, STAHLE SW AD1750, and NOAMER AD 1500), where MBH98 retains more PCs than in the previous step although the network does not change (i.e., the same proxies are inputs to the PCA). But it was not only that the network was (supposed to be) the same … the PCs were exactly the same as in the previous step! Of course then, it is impossible for any “rule” to retain different number of PCs. The most striking example comes from the NOAMER AD1500 step, which is using PCs from the AD1450 step. Mann is keeping 2 PCs in the AD1450 step, but in AD1500 step 6 PCs are retained from the same PC set!

As Steve has said several times, MBH9X is the gift that keeps on giving! When now preparing for this post, I discovered yet-another Mannian mystery. There is a single exception to the above mentioned correspondence between the archive and PCs actually used. Namely, PCs are calculated for the SOAMER AD1450 step, but no SOAMER PCs are used prior the AD1600 step. The corrigendum text cited in the beginning continues

PCs were no longer calculated back in time once a given network contained fewer than 7 available series (with the exception that PCs were calculated for the ‘Stahle Southwest U.S./Mexico network’ with 6 series available). Thus, although some series may be available further back in time, they may not have been used to calculate PCs. For example,there are 110 series available back to 1400, but only 95 are used because PCs were not calculated on 6 Australian and 6 South American ITRDB series and 3 ‘Vaganov’ series.

SOAMER contains 7 series in the AD1450 step. Why were none of those PCs used?

(*) Actually there is a fourth case (AUSTRAL AD1750), which is immaterial here as it has the corresponding calculation step in the archive.

Who Wrote EPA’s “Myths vs Facts”?

The Mann Statement of Claim prominently displayed, as one of only two quotations from the “inquiries”, an extended quotation from the Myths vs Facts webpage, included as one of three Resources accompanying the EPA decision denying reconsideration of various petitions for reconsideration of the Endangerment Finding (though Mann’s Statement of Claim falsely cited the gazetted (Federal Register) denial decision itself as its source.) The Myths vs Facts document (as well as the Factsheet and Press Release, also linked as “Resources”) contained statements and assertions that were both untrue and which were undocumented in the actual language of the “formal” documents that they were supposedly supporting.

The identity of the authors (and reviewers, if any) is not disclosed in the supporting documents (RTP documents) for the denial decision – Jean S speculated a few days ago that Gavin Schmidt was involved in EPA’s supposed investigation and “exoneration” of Michael Mann; Schmidt has thus far refused to comment.

In today’s post,   I’ll discuss who wrote (and possibly reviewed) the Myths vs Facts document. Continue reading

Mannomatic smoothing: technical details

This post is rather technical, and it is intended mainly for the historical completeness. So unless you are very, very interested in the tiny technical details of the HS saga, you can safely skip this.

As most readers are aware, and stated in my post few hours after CG broke out, Mike’s Nature trick was first uncovered by UC here. He was able to replicate (visually perfectly) the smooths of MBH9x thereby showing that the smooths involved padding with the instrumental data. The filter used by UC was the zero-phase Butterworth filter (an IIR filter), which has been Mann’s favourite since at least 2003. However, there was something else that I felt was odd: UC’s emulation required a very long (100 samples or so) additional zero padding. So about two years ago, I decided to take an additional look at the topic with UC.

Indeed, after digitalizing Mann’s smooths we discovered that UC’s emulation was very, very good but not perfect. After a long research, and countless hours of experimenting (I won’t bore you with the details), we managed to figure out the “filter” used by Mann before Mann (2004)-era. Mann had made his own version of the Hamming filter (windowing method, an FIR filter)! Instead of using any kind of usual estimate for the filter order, which is usually estimated from the transition bandwidth (see, e.g., Mitra: Digital Signal Processing) and has typically the length of a few dozen coefficients at maximum, he used the filter length equal to the length of the signal to be filtered! As Mann’s PCA was apparently just a “modern” convention, this must be a “modern” filter design. Anyhow, no digital signal processing expert I consulted about the matter had ever seen anything like that.

In order to see how absurd the “filter design” is, consider filtering a signal of length 999 samples. According to Mann, you should design a Hamming filter of the same length. One should always disregard half of the filter length amount (i.e., 499 in our example) of filtered values from both ends, so in Mann’s case one would end up with a single smoothed value! In Mann’s implementation, however, one ends up with a filtered series of the same length as the original signal.

Another way to think of Mann’s “filter” is to consider it as a normal filter with a huge (half the signal length) zero padding to the both ends of the signal. This interpretation also gives hints why UC’s emulation was so successful. One can also speculate, if the similarity of the results between zero-phase Butterworth and Mann’s original filters is the reason Mann chose the Butterworth filter in the first place.

If someone wants to explore this topic further, I’ve place my Octave/Matlab implementation of Mann’s smoother here. The code includes references to the original Mann’s code I uncovered. Finally, the exact parameters of the trick in MBH9x were as follows. MBH98 has 50-year smoothing with padding of 1981-1995 instrumental. Additionally, the smoothing is cut back 25 samples (half of the “filter length”) from both ends. MBH99 used 40-year filtering with 1981-1997 (not 1998!) instrumental padding. The smooth is cut back 20 samples from the end but not from the beginning.

Who wrote the EPA documents?

Jean S writes (transferred from a comment with the addition of a few headings):

A question for the experts: is it known who wrote and who were used as experts in the EPA documents? If not, is that information considered public (i.e., obtainable under FOIA or similar)?

The reason I ask is that I get very, very eerily feeling when reading certain parts of the EPA decision, especially this one. For instance, I think there are very, very few people in this world from whom the following paragraphs could originate (considering style, content and astonishing familiarity with Mann’s work): Continue reading

Misrepresentations and the Tainted Narrative of Mann’s Complaint

In a recent post, I observed that Mann’s Statement of Claim contained a bizarre misrepresentation about the nature of Mann’s research, as it falsely credited Mann with being “one of the first” to document the increase in 20th century temperatures. Reader PhilH, a retired judge, observed that, on its own, the misrepresentation was merely odd and that it would have significance for the pleadings only if it could be connected to the narrative of the case. In today’s post, I’ll try to do exactly that.

casey-at-the-bat-1888-grangerThe “money quote” chosen by Mann’s lawyers as supposed evidence of his “exoneration” by the “inquiries” is an EPA statement that manipulation of “temperature data and trends”  is a “myth”. This quotation connects to the paragraph 2 misrepresentation, but not to Mann’s actual research.  It’s a massive whiff by Mann’s lawyers. Continue reading

The “Discovery” of the EPA Inquiry

One of the essential elements in Mann’s reliance on EPA findings is his assertion that his supposed exoneration by EPA had been “widely available and commented” on in the media and had been “read by the Defendants”:

All of the above reports and publications were widely available and commented upon in the national and international media. All were read by the Defendants.

The claim that all nine “inquiry” reports had been “read” by Steyn and the other defendants is surely a fantasy on the part of Mann and his lawyers. While Steyn seems to be a man of eclectic interests, somehow I can’t picture him poring through the dreck of the turgid “reports” from the various inquiries.

I am particularly dubious of Mann’s claim that Steyn (and Simberg) had read the EPA documents. Some climate blogs took notice of the EPA decision denying various petitions for reconsideration of the EPA Endangerment Finding when it was issued, but none understood it to supposedly be an “investigation” and “exoneration” of Michael Mann, something which would have occasioned great interest in July 2010, then only a few weeks after the Muir Russell and second Penn State reports. The EPA denial decision was first included in a list of inquiries in an unmarked Feb 2011 revision of a November 2009 post by SKS, but really was brought to public attention for the first time in Mann’s Statement of Claim itself. It seems very improbable to me that Steyn (or Simberg) were aware of EPA’s supposed findings in connection in Mann (not that Mann’s characterization is accurate, but that’s a story for another day), or why they would be obligated to be familiar with them. Given Mann’s allegation that Steyn, Simberg and others were supposed to be aware of EPA’s investigation and “exoneration” of Mann, the contemporary unawareness of this supposed EPA investigation – especially at SKS and Real Climate – is really quite remarkable.

Continue reading

EPA on Mann’s “Fraud” Invective

Michael Mann, now feigning sensitivity towards Mark Steyn’s use of the word “fraudulent”, used identical language in the Climategate emails against critics without the slightest compunction. Mann’s hypocrisy has been widely noted.

Unpublicized thus far is a discussion by EPA, in which EPA concluded that Mann’s accusations of “scientific fraud” were within the scope of “acceptable and appropriate” scientific exchange and that it is “entirely acceptable and appropriate for scientists to express their opinions and challenge papers that they believe are scientifically flawed” in such terms.

The EPA’s finding appears to be inconsistent with Gavin Schmidt’s recent tweet arguing that such language is “per se defamatory”:

Saying that ppl [people] are frauds is per se defamatory. Goes beyond disagreement/error/dislike.

Continue reading


During the past few months, the volume of comment spam from spambots has increased by an order of magnitude. Right now, this blog receives about 1 spambot comment per minute. Akismet and other spambot blocking services do an excellent job and without such software, it would be impossible for blogs to function.

Nonethelss, I’ve noticed that the number of spambot comments being passed through has increased by an order of magnitude in the past few months. Anthony is also experiencing this. Because I moderate after the fact, spam comments appear here until I come online.

The blog software permits some customized screening. The word “blog” seems to appear in many spambot comments, so I’m going to try adding the word “blog” to my customized screening as this word. So avoid using this word in comments and if you do, expect it to be caught in moderation.

Yet Another Misrepresentation in the Mann Pleadings

In today’s post, I’ll discuss another misrepresentation in Mann’s Statement of Claim, one in which Mann bizarrely misrepresented the nature of his own research, falsely claiming credit for being “one of the first” to “document” the increase in 20th century temperatures. This particular false claim was in the same paragraph as Mann’s false claim to have received a Nobel prize. While the latter false claim has received widespread and well-deserved derision, Mann’s false description of the nature of his research has thus far passed without comment, an oversight that I will try to remedy in today’s post, which is part of a series of articles on various untrue statements by Mann in his pleadings in Mann v Steyn et al. For previous articles, see here. Continue reading

ACLU and National Media Intervene in Mann v Steyn et al

I get the sense that the Washington libel community and U.S. national media have belatedly woken up to the potential threat of Mann v Steyn and that the tide is now starting to run strongly against Mann in the anti-SLAPP proceedings. The most visible evidence of this is an impressive Amici brief from the ACLU and an imposing list of 25 other media organizations (the Reporters Committee for Press Freedom, the American Society of News Editors, the Association of American Publishers, the Association of Alternative Newsmedia (The Village Voice et al), NBC Universal, Bloomberg News, the publishers of USA Today, Time, The Washington Post, The Chicago Tribune, The Los Angeles Times, The Detroit Free Press, The Seattle Times, The Arizona Republic and The Bergen County Record) filed on August 11, 2014.

In addition, Steyn’s own Amicus brief substantially upped the ante on a separate front. It repeatedly and directly accused Mann of submitting “fraudulent” information to the court and commented adversely on “the ease with which Mann lies about things that would not withstand ten minutes of scrutiny in a courtroom”.

Other briefs are by CEI; National Review, the District of Columbia, the Alliance Defending Freedom; the Cato Institute, Goldwater Institute, Reason magazine and David Horowitz’s Individual Rights Foundation;, The Daily Caller, PJ Media, The New Criterion and various Internet publishers.

The Brief by ACLU and 25 Others
On some points, the ACLU brief takes a pretty similar line to that previously taken by National Review and CEI, but it presents some interesting fresh nuances and authorities. I won’t attempt to precis the brief as it is well written and worth reading, though readers uninterested in the procedural details of anti-SLAPP litigation would do well to skip to section II of the brief (commencing on page 12).

They asserted that the “challenged publications bear all the traditional hallmarks of opinion” and that “permitting such a defamation claim to proceed will substantially chill speech that challenges scientific conclusions, as well as public policies based on them.”

They observe:

Mann essentially complains that the defendants accused him of manipulating data, including by molesting and torturing it, to serve a political agenda. … Because the statements are quintessential opinions about the validity of Mann’s scientific methods and conclusions, they are entitled to full constitutional protection.

The ACLU brief provides detailed commentary on the range of abusive commentary in scientific disputes that courts have found to be permissible, concluding that the commentary in this case was within permissible limits.

In subsection II(D) (page 20), the ACLU et al provide an interesting and, in my opinion, compelling argument against Mann’s claim that the defendants were obliged to accept the findings of the various government agencies – a topic that I’ve focused on in my previous commentary.

CA readers are aware that Mann’s claim to have been “exonerated” by Muir Russell, Oxburgh, NOAA and the UK Government are untrue (“fraudulent” is Steyn’s term), but ACLU’s argument is different: they say that “punishing defendants’ speech because Mann’s work had been backed by other scientists or governmental agencies is contrary to core First Amendment principles”.

They first counsel the court against getting embroiled in trying to resolve scientific questions, no matter how authoritative the apparent support for the scientific assertion – advice that any court would undoubtedly be willing to heed.

The ACLU et al also unambiguously said that any reliance given by the Superior Court to supposed vindications or exonerations by governmental agencies was an “error” and that such government agencies did not give Mann the right to “silence his critics in a defamation claim”:

Furthermore, to the extent the Superior Court credited Mann’s assertion that investigations by the EPA, the National Science Foundation, and Penn State, among other scientific and governmental bodies, “laid to rest” defendants’ questions regarding Mann’s research, Am. Compl. ¶ 24, this too was in error. See also July 19, 2013 Orders at 16 (suggesting that statements were actionable because “Plaintiff’s work has been investigated and substantiated on numerous occasions”). The fact that certain official panels backed Mann’s methodology – facts that were not only disclosed in the challenged publications but in fact formed the basis for them – cannot allow him to silence his critics in a defamation claim. Under the First Amendment, the government is not the final arbiter of truth with the power to foreclose further challenge to its policies.11

In footnote 11, the Amici unambiguously supported defendants’ right to disagree with the findings of government inquiries, describing the right to such disagreement as fundamental to the First Amendment. In doing so, they observed that the defendants had provided substantive criticism of the procedures of the investigative bodies:

11 Indeed, if the First Amendment and case law interpreting it stand for anything, it must be that disagreement with findings of government and quasi-government bodies are fully protected. Here, defendants criticized the investigative bodies as, variously, lacking independence, failing to interview a relevant witness and therefore being too limited in scope, and being overly reliant on evidence provided by Mann’s employers, who had “so much at stake.” See Am. Compl., Ex. A.

In the next footnote, they pointed out that Mann’s pleadings had omitted the relevant information that the commentaries had disclosed that the government agencies had sided with Mann (while disagreeing with the agencies) and had provided hyperlinks to the criticized agency reports thereby permitting readers to form their own conclusions:

In this regard, Mann’s description of the commentaries omits that they disclose that governmental agencies had in fact sided with him while also criticizing those official findings. That background is described in their text – and, in some instances, through hyperlinked sources – thereby allowing readers to formulate their own judgments about the opinions expressed. See, e. g., Boley, supra, 950 F. Supp. 2d at 262 (hyperlinking to an earlier article provided “the necessary context for the allegedly defamatory remark”); Abbas, supra, 975 F. Supp. 2d at 18 & n. 7 (finding hyperlinks were sufficient to disclose background for fair comment privilege); Agora, Inc. v. Axxess, Inc., 90 F. Supp. 2d 697, 704-05 (D. Md. 2000) (dismissing defamation claim based on facts disclosed through hyperlinks), aff’d, 11 F. App’x 99 (4th Cir. 2001); Adelson v. Harris, 973 F. Supp. 2d 467, 483 (S. D. N. Y. 2013) (relying on a hyperlink to a report about an official proceeding in dismissing a defamation claim).

The ACLU brief closes:

At bottom, a participant in the “rough-and-tumble” of public debate should not be able to use a lawsuit like this to silence his critics, regardless of whether one agrees with Mann or defendants. See Guilford Transp. Indus., supra, 760 A. 2d at 595-96 (endorsing Voltaire’s philosophy, “‘I disapprove of what you say, but I will defend to the death your right to say it,'” which “anticipatorily articulated the spirit of our First Amendment”). The “law certainly does not insist” that a speaker “look kindly on [his] subjects,” nor that a plaintiff “simply by filing suit and crying ‘character assassination!,'” may “silence those who hold divergent views, no matter how adverse those views may be to plaintiffs’ interests.” Underwager v. Salter, 22 F. 3d 730, 736 (7th Cir. 1994). Rather, as the Seventh Circuit eloquently put it, expressing a sentiment echoed by other courts: “Scientific controversies must be settled by the methods of science rather than by the methods of litigation. More papers, more discussion, better data, and more satisfactory models – not larger awards of damages – mark the path toward superior understanding of the world around us.” Id. (citation omitted).13

Steyn’s Brief
Steyn’s brief (noted up together with the other briefs in Steyn’s blogpost here is far more direct in its allegations against Mann than previous pleadings, repeatedly describing claims and assertions in Mann’s previous pleadings as “fraudulent”.

Steyn rubbed salt into Mann’s false claim to have been a Nobel Prize winner and the imaginary tort of defaming a Nobel Prize winner, describing Mann’s priod claim as a “fraudulent misrepresentation”:

Mann’s fraudulent misrepresentation of his credentials and academic standing later earned him a rebuke from Geir Lundestad, director of the Nobel Institute in Oslo. One can well understand why the exposure of Mann’s fraudulent claim should cause him embarrassment but it should surely not justify resetting the procedural clock back to the beginning on this case, which is what in effect happened.

Steyn repeatedly used the term “fraudulent”, also using phrases like “the audacity of the falsehoods in Mann’s court pleadings is breathtaking” and that it was “deeply disturbing that a plaintiff should make such fraudulent claims in his legal pleadings”:

In his later court filings, Mann has made equally preposterous and objectively false claims. For example, Mann has claimed that he has been “exonerated” by such bodies as the University of East Anglia, the U.S. National Oceanic and Atmospheric Agency, and even by the government of the United Kingdom, none of which have investigated Dr Mann at all, never mind “exonerated” him.

The audacity of the falsehoods in Mann’s court pleadings is breathtaking. For example, on page 19 of his brief below dated January 18, 2013, he cites the international panel chaired by the eminent scientist Lord Oxburgh, FRS as one of the bodies that “exonerated” him, whereas on page 235 of Mann’s own book, The Hockey Stick and the Climate Wars , he states explicitly that “our own work did not fall within the remit of the committee, and the hockey stick was not mentioned in the report.” It is deeply disturbing that a plaintiff should make such fraudulent claims in his legal pleadings.

Steyn added:

It is clear from the ease with which Mann lies about things that would not withstand ten minutes of scrutiny in a courtroom that he has no intention of proceeding to trial.

All in all, it seems to me that Mann is facing far more formidable opposition than in any of the previous hearings, both from the ACLU et al intervention and from increasingly formidable defendant briefs. Mann presumably chose DC as a forum because he thought that it would favor him, but appears to have overlooked the possibility of intervention by the ACLU and national media. I can’t imagine that the DC Appeals Court will want the quixotic decision by J Combs-Greene (mostly adopted by J Weisberg) to represent the public face of DC libel law and I anticipate a different decision.


Get every new post delivered to your Inbox.

Join 3,382 other followers