(This post is by Jean S.) A few days ago Steve discussed Raymond Bradley’s objection to use of the Yang Chinese composite reconstruction in the Mann et al Eos-response to Soon & Baliunas (2003). Bradley called the series “crap”, and demanded it to be removed from Figure 2 in the Eos article. It is not completely clear if Bradley was fully aware that the Yang composite was also brought in to the Figure 1 in the form of Mann and Jones (2003) (later MJ03) NH composite, which was substituted only 3 days earlier for Briffa’s long series (of the earlier draft) in an agreement made only between Mann and Jones. Nevertheless, Mann defended (#4207) the use of the Yang composite in MJ03 by saying it got “a moderate low weight” in the composite.
In our GRL article, Phil and I weighted the records we used with respect to their decadal correlations with the instrumental gridpoint surface temperature data for the same region (numbers in parentheses in attached figure 1 from the paper), so if a series is truly crap in an objectively determined sense, it got very low weight. The China series has a reasonable (r=0.22), but not great correlation–and it gets a moderate low weight.
Only a day later the weight was further downgraded form a “moderate low weight” to a “low weight” (CG1: 1056477710).
Phil and I have already discussed–we agree that the low weight given to the record in the Mann and Jones composite treats the record appropriately…
At the time it was impossible for Bradley and others to quantify what Mann’s “appropriate low weight” meant. Luckily, as pointed out by UC, the MJ03 code is available in #3499 (also CG1: 1092167224) and we can now calculate the weight of the Yang composite. Before doing that, let’s take a look at other proxies used in MJ03.
As Gavin observed there are “a few typos” in the Eos Figure 1.
I note there a few typos in the Eos figure 1 though (signs of fast turnaround perhaps). It should say 1856–1940 in the key for Briffa et al. for instance (as it is in Jones and Mann, 2004).
Another “typo” was, of course, calling MBH99 plus 0.5 sigma as “Crowley and Lowery” (yellow). I found a third “sign of fast turnaround” in the caption:
an extension back through the past 2000 years based on eight long reconstructions [Mann and Jones,2003].
The long NH reconstruction shown in Figure 1 is actually based only on six series as clearly stated in MJ03 (Figure 2a). There is also another NH series calculated in MJ03 based on eight series, but it only starts in AD553, and AFAIK it is not used anywhere. This shorter NH series has the same six proxies as the main reconstruction plus Jacoby’s Mongolia series (discussed by Steve yesterday) and Fisher’s West Greenland series, both shown in the Eos Figure 2.
A natural member of MJ03 portfolio is Mann’s own western North American PC1 shown on a top of Eos Figure 2 (“Western US”), and discussed by Steve here. This series is a splice of Mannomatic PC1 of six chronologies extending back to AD200 (up to 1700) and the infamous “fixed” Mannomatic PC1 (based on 27 chronologies) from AD1000 network in MBH99 (after 1700). How exactly the splicing was done, as far as I’m aware, remains a mystery today. However, it is interesting that Mann did not “extend” his “fixed” AD1000 PC1 by splicing the new PC1 (based on six chronologies) to the end (i.e., for 200-999 period), which would have been more inline with his stepwise approach in MBH9X. Instead, the cutting point was 1700. We’ll probably never find out the reason for this odd selection, and I only remark that “fixed” AD1000 PC1 has relatively high 11th century values compared to those in the extended AD200 PC1.
As a side note, around time of Steve’s MJ03 post (see comments) we began to understand the effect of this ridiculous “CO2-adjustment” (Mannkovitch Bodge) in MBH99: it adjusted the verification RE statistic and affected the 1000-1850 linear trend (“Milankovitch cooling”) as plotted in the Hockey Stick. It took two more years and ClimateGate files to fully understand what had been done, see Bishop Hill’s treatment for details.
[Update: Dec 5, 2011. Steve wrote me that it is likely that instead of the bodged Torneträsk series some type of mixture of Torneträsk, Taimyr and Yamal was used as the third series, see Fig. 1 in MJ03. However, Taimyr and Yamal are not used in Jones&Mann (2004) (see Fig. 1), which shares the code with MJ03, so it is hard to tell what exactly went in without actually seeing the file “torny.dat”. Moreover, the decadal correlation value in the code (0.32) does not match either the one given (0.47) in Fig.1 of MJ03 or the one (0.54) in Table I of JM04 …]
Third series in MJ03 is Briffa’s Torneträsk, again shown in the Eos Figure 2. I suppose only few even among regular CA readers know that also this series was “adjusted”, see Steve’s discussion of the topic from the early days of CA. Fourth series to enter the MJ03 portfolio is the Chesapeake Bay Mg/Ca proxy, also present in Figure 2. This series has declining temperatures since late 19th century. Finally, the last two components of MJ03 are rather surprising (not-so-suprisingly these are not plotted in the Eos Figure 2): two Greenland borehole reconstructions from Dahl-Jensen et al (1998) (pdf). Both D-J series have very high MWP values relative to the present (see Figure 4 in the article).
Now that we know the six proxies going in to MJ03 composite, we can return to the weighting issue. Recall that Mann claimed to Bradley (and others) that the weighting in MJ03 was done “objectively” by their decadal correlations to the local temperature, and that the Yang composite had a “reasonable (r=0.22), but not great correlation” and thus it obtained a “moderate low weight”. What Mann “forgot” to tell is that the weight in MJ03 is calculated not only based on correlation but also on the area.
Composite series were formed from weighted combinations of the individual standardized proxy series, employing weights on the individual records that account for the size of the region sampled, and the estimated reliability of the temperature signal as determined by comparison with the instrumental surface temperature record [Jones et al., 1999].
From the code it is seen that this areal weighting is actually cosine(latitude)*dof, where dof is an “estimated number of temperature gridpoints represented by record”. The total weight given to a proxy is then obtained by multiplying this area weight by the correlation. As the Yang composite has the highest dof (4) while both Dahl-Jensen series has 0.667, it is not hard to guess what the final weighting looks like…
…and here are the results of the relative weighting of MJ03 proxies:
extended NA PC1: 33%
Yang composite: 30%
Torneträsk: 17%
Chesapeake: 10%
D-J (DYE-3): 6%
D-J (GRIP): 4%
So in the end of the day, MJ03 composite so prominently presented in Eos Figure 1 is practically just an average of three series, “crappy” Yang composite, “extended” and “fixed” Mannomatic PC1, and Briffa’s “adjusted” Torneträsk. Also the definition of “low” in the Mannian dictionary is likely “anything below 1/3”.
Climate Audit 
40 Comments
Steve,
O/T but FYI
http://strata-sphere.com/blog/index.php/archives/17701#comments
I don’t follow this. The dark blue line with half-tone in the Eos Fig. 1 is identified as Mann et al 1999 with uncertainties, presumably MBH99 plus and minus 2 (probably underestimated) sigmas. The fourth (yellow) line is identified as “Crowley and Lowerey”. Are you saying that MBH99 is in there twice? The two lines do look remarkably similar, though the vertical distance between them seems to shrink over time, and the wiggles in the yellow line are a little more attenuated.
Steve – yes, it’s MBH plus .5 sigma. We discussed this a few years ago. Tim Lambert, of all people, was the person who identified the problem. I knew that there was something wrong with the series, but hadnt thought that it was MBH in disguise.
Re: Hu McCulloch (Dec 4 16:32),
see the update in “Hide-the-Decline Plus” or directly here.
OK — But the yellow “Crowley and Lowery” line has been smoothed a little differently than the dark blue “MBH99” line, and the gap between the two distinctly declines over time — at the end there is hardly any difference between the two (except for end treatment). How odd that lead author Mann wouldn’t recognize his own reconstruction! 😉
Both also follow the CRU instrumental series closely at the end, but this may be a consequence of the fact that MBH99 was tuned to fit CRU with essentially a multiple regression?
Mann uses butterworth smoothing which is a method developed in frequency applications rather than noisy red noise series. To calculate, there is zero padding at the ends after some reflection or normal padding. My guess is that the displacement by 1 sigma affects the end effects which boils back throughout the series.
Ahah! — The fact that “sigma” gets smaller toward the end is then why the two series converge!
Seems like a deliberately constructed “faux-corroborating” series, and not just a mixup…
What a bunch of party-poopers you are!
Everyone knows the party will be a dud if the punch is not spiked up with some secret sauce!
RR
It was also Tim Lambert who exposed the equally bogus “Dr. Thompson’s Thermometer” in Al Gore’s AIT!
Wow. Endless games.
Jeff,you beat me to it.
And they excoriated me for using 18 proxies equally weighted for my reconstruction. I guess when Mann uses basicly 3, it is still a higher octane result than mere mortals can get.
Everytime I read one of these threads I think back to Manns own description of his papers as being ‘Skilful’ and cannot stop laughing.
I have difficulty seeing how what Mann and co. do with some of these papers can be called ‘scientific’…..
Vague undefined terms like ‘moderate’ and ‘low-weight’ while the exact recipe is kept deliberately opaque…. And the details turn out to be highly disputable if not risible. This all brings to mind alchemy and astrology, not rigorous science….
This is giving the phrase “It’s worse than we thought,” an entirely new meaning.
What were the dof, cos(latitude)*dof, and R2 used to obtain these weights? Dye and GRIP are relatively close to one another, though almost as far from Tornetraesk as Chesapeake is from the US SW.
I’m not getting exactly the same numbers.
I followed the code logic and got these:
Series Mann & Jones Percent of
Weighting Total
china_series1 .7421845 25.86%
itrdb_long_fixed .8250875 28.75%
westgreen_o18 .1125318 3.92%
torny .4195594 14.62%
chesapeake .2475770 8.63%
mongolia_darrigo .2727993 9.51%
dahl_jensen_grip .1033563 3.60%
dahl_jensen_dye3 .1465809 5.11%
Here are the intermediate steps requested by Hu:
Series A) dof B) cos(lat*pi/180) C) A*B D) r^2
china_series1 4 .8434 3.3736 0.22
itrdb_long_fixed 2 .7934 1.5867 0.52
westgreen_o18 0.667 .2250 0.1500 0.75
torny 3.5 .3746 1.3111 0.32
chesapeake 1 .7986 0.7986 0.31
mongolia_darrigo 1 .6820 0.6820 0.40
dahl_jensen_grip 0.667 .2924 0.1950 0.53
dahl_jensen_dye3 0.667 .4226 0.2819 0.52
[Jean S: Richard, your numbers are correct! However, as pointed out below, you calculated %:s for the shorter (8 series) composite. Using your numbers but excluding Jacoby’s Mongolia (starts AD252) and Fisher’s West Greenland (starts AD553), I get exactly the numbers I gave in the post. So thank you for conforming that I didn’t make any skrew up 🙂
As a minor note, I think Mann is using the correlation coefficient (r) not the square (r^2).]
That header on the first result set is supposed to read:
“Series — Mann & Jones Weighting — Percent of Total”
Also just FYI, here are the percentages of Earth’s surface that
Mann’s code assigns to each series. This is the percent-of-total of each value in my Column C above.
(In my opinion, the following percentages that Mann’s code assigns may be the most asinine thing about this entire paper.)
china_series1 — 40.26%
itrdb_long_fixed — 18.94%
westgreen_o18 — 1.79%
torny — 15.65%
chesapeake — 9.53%
mongolia_darrigo — 8.14%
dahl_jensen_grip — 2.33%
dahl_jensen_dye3 — 3.36%
Sorry, I should have said “of the NH” rather than “of Earth’s surface”.
RTF
Richard — You have 8 series, apparently for the index Jean says starts in AD 553, and which was not used for anything. Do you have the values for the 6 series index?
I’m just going by the code in e-mail #3499, since that seems to be the only code that Jean S has referenced. Perhaps I’ve missed something, but I can’t see what.
Are we absolutely sure that the e-mail code was _not_ what was used? Or is that just an assumption that we’re making here?
RTF
Richard, Jean points out that the eight proxies are not for the main graph. That is based upon only six. West Greenland and Mongolia are used in AD553 series (figure 2), not the full construct. Figure 1 (or Figure 2a in MJ03) is based upon the other six proxies. Here’s the quote:
“The long NH reconstruction shown in Figure 1 is actually based only on six series as clearly stated in MJ03 (Figure 2a). There is also another NH series calculated in MJ03 based on eight series, but it only starts in AD553, and AFAIK it is not used anywhere. This shorter NH series has the same six proxies as the main reconstruction plus Jacoby’s Mongolia series (discussed by Steve yesterday) and Fisher’s West Greenland series, both shown in the Eos Figure 2.”
Looking at the code, it should be simple to remove the sections that refer to Mongolia and West Greenland and the run again. That should give you the revised values.
Responding to Keith W. and also particularly to Jean S’s inline comment which was:
—–
[Jean S: Richard, your numbers are correct! However, as pointed out below, you calculated %:s for the shorter (8 series) composite. Using your numbers but excluding Jacoby’s Mongolia (starts AD252) and Fisher’s West Greenland (starts AD553), I get exactly the numbers I gave in the post. So thank you for conforming that I didn’t make any skrew up
As a minor note, I think Mann is using the correlation coefficient (r) not the square (r^2).]
—–
First, thanks for the correction on r^2; I hadn’t realized that was different from the correlation coefficient, though I probably should have remembered enough to know that. Sorry for that.
On the other matter, respectfully I think you are both missing something. The “dof’s” are indicated as being Mann’s estimate of the proportion of grid squares in the northern hemisphere which are attributable to a given series.
Jean S, in order to just take out any series, you have to know to which other series (singular or plural) Mann has assigned those grid squares, and in what quantities. This is of course almost completely subjective, which is what led me to make my comment about that being the most asinine thing about this entire paper.
So while I agree that you are on the track, and it’s a very good thing that you did this analysis and post, perhaps a little caveat is in order, to the effect that we cannot at this point really know the exact percentages of weighting that were assigned, even if we assume that this same code logic was used for a six-series reconstruction, because we don’t know the exact dofs that were assigned?
Thank you for your consideration.
Here is the relevant quote by Mann in his code, where he defines how he comes up with his area weightings:
“% Estimate Area represented by each proxy record based on latitude of
% record and estimated number of temperature gridpoints represented by record”
RTF
“on the track” = “on the right track”
RTF
Also, Keith W., apparently you replied to my questions:
“Are we absolutely sure that the e-mail code was _not_ what was used? Or is that just an assumption that we’re making here?”
by pointing out that the main graph only contains six series. I was aware that it does, but that does not prove that the exact e-mail code in #3499 was not used.
As I wrote above, if that exact code _was_ used with exactly those dofs, then it inarguably gave bad results, because to use the same dof’s for the six series that were used, regardless of whether the other two are included in the weighting calculations cannot possibly be correct.
To assume that that exact code was _not_ used is, in my opinion, to give Mann far too much benefit of the doubt at this point. It may have been, it may not have been. And if the code was modified, it may or may not have had the dof’s modified. But in any event, we should not be conceding points to Mann by default. At this point, he is one of the last people to merit such consideration.
Thank you for weighing in on my questions.
RTF
Richard, I’m not following your logic. This is the code used for both six series and eight series calculations. You just have to follow the code all the way to the end. The relevant parts being:
% determine min and max available years over all proxy records % minarray=[min(x1) min(x2) min(x3) min(x4) min(x5) min(x6) min(x7) min(x8)]; ... % perform reconstructions based on: % (1) the 6 proxy temperature records available over interval AD 200-1980 % (2) all 8 proxy temperature records available over interval AD 553-1980 istart0=200; istart1=200; istart2=553; nseries1=0; nseries2=0; weightsum1=0; weightsum2=0; for j=1:M if (istart1>=minarray(j)) nseries1=nseries1+1; weightsum1=weightsum1+weight(j); end if (istart2>=minarray(j)) nseries2=nseries2+1; weightsum2=weightsum2+weight(j); end end % calculate composites through 1995 (too few series available after that date) % As discussed above, persistence is used to extend any series ending % between 1980 and 1995 as described by Jones and Mann (2004). tend=1995; for i=istart1:tend unweighted1(i)=0; unweighted2(i)=0; weighted1(i)=0; weighted2(i)=0; for j=1:M if (istart1>=minarray(j)) unweighted1(i)=unweighted1(i)+standardized(i,j); weighted1(i)=weighted1(i)+weight(j)*standardized(i,j); end if (istart2>=minarray(j)) unweighted2(i)=unweighted2(i)+standardized(i,j); weighted2(i)=weighted2(i)+weight(j)*standardized(i,j); end end end unweighted1=unweighted1/nseries1; unweighted2=unweighted2/nseries2; weighted1=weighted1/weightsum1; weighted2=weighted2/weightsum2;Thanks for pointing out that part of the code to me. Well, my reason for thinking that you doubted that that exact code was used is because your six percentages add up to 100%, but the code’s six weights are as I calculated, and do not add to 100%.
The problem is simply that this code is doing something (else) that it shouldn’t be doing — taking out two of the eight, but not using a revised set of dof’s so that all the weights can add up to 100%. In effect, the six-series reconstruction is an estimate for _part_ of the NH, but not the whole thing. By doing this, he is upping China’s assigned percentage of the study area further toward 50% from 40.26%.
There are actually two study areas, but the paper is falsely claiming that there is only one — the entire NH.
Thanks again for your clarification. And thanks, Hu, for your comment which drew my attention to this.
RTF
I concur with Richard that by MJ03’s logic, they “should” have adjusted the rather subjective “dof” that went into their weights when they went from 8 series to 6, since deleting Mongolia would somewhat increase the area represented by Yang and deleting W Greenland would somewhat increase the area attributable to GRIP and Dye3. Yang already represents 9 sites scattered across China, whence its big “dof”.
But Jean’s point is not that they weighted correctly or incorrectly, but just that given how they did weight they ended up with 30% Yang despite Bradley’s misgivings about that series. 30% for 1 of 6 series can hardly be called “moderate”.
Hu McCulloch (Dec 5 10:14),
yes, and additionally they are claiming in the caption that the composite consists of eight proxies. Notice also that Mann does not say in his reply how many proxies were used, but he did attach the Fig1 from the MJ03 paper (which of course has 8 NH proxies marked). Attaching the figure for Bradley seems to indicate that Bradley had not seen the MJ03 draft.
If I was told that 1 of 8 series got “low moderate weight”, I would not expect anything more than 10% weight.
Bradley, behind the scenes, seems to have a level of honesty about him and his works.
Outwardly, he’s much, much different http://www.umass.edu/loop/talkingpoints/articles/132845.php
…Bradley, who directs the Climate System Research Center, with Mann and Hughes developed a hockey-stick-shaped graph that showed relatively flat temperature levels in the Northern Hemisphere for much of the last 1,000 years followed by a sharp upward turn in the 20th century. For this work the scientists were depicted by industry and government foes as publicity seekers and were accused of causing needless panic with a “sky is falling” message. They were harshly criticized by certain media and derided by some members of Congress.
In his book, Bradley says that many government and elected officials refuse to accept his and his colleagues’ evidence that human activities play a key role in global warming and many have continued to oppose any United States participation in international agreements to limit greenhouse gas emissions. Further, as the book describes, they tried to sow doubt about the scientific evidence for global climate change in the public mind and to undermine the credibility of Bradley and his colleagues.
“The book came out of the frustration and aggravation of my own experience. And my story is by no means the worst,” Bradley says. “Global Warming and Political Intimidation” is dedicated to Rep. Sherwood Boehlert, a Republican representative from New York who stood up to defend Bradley and colleagues as they faced a congressional investigation by fellow Republican Rep. Joe Barton of Texas…
Please, someone…. Please. Explain briefly to me how Pielke Jr. can make this assertion on Dec. 2, 2011?
http://rogerpielkejr.blogspot.com/2011/12/about-those-skeptics.html
“The debate over climate science is over and has been won by those who assert a human influence on the climate system. This then is what victory looks like. (For supporting evidence on the science and opinion, see chapters 1 and 2 of TCF).”
Steve: OT
snip – OT
I wonder what might be Mann’s definition of “reasonable”. According to his own calculations (see Table I in Jones&Mann(2004)), the decadal correlation of the Yang composite is not even close being significant at the 5% level. I have no details of his tests, but Thompson’s Guliya -series with r=0.45 is not significant either.
The regression F statistic for a simple regression with n observations is (n-1)*R2/(1-R2), and if there is no serial correlation and the errors are normal etc, this has an F(1,n-1) distribution, so that the significance depends a lot on the sample size. (In a simple regression, the regression F statistic is just the square of the t-stat on the slope, so the two tests are equivalent.)
In some of this literature (eg Thompson CC03, PNAS06), series are inefficiently aggregated with equal weight after scaling, and so this test is relevant for the significance of the aggregate’s correlation with temperature.
However, in JM03, the series are more efficiently aggregated according to their correlation with temperature. This is like multiple regression, in the special case where the regressors (proxies) are uncorrelated.
While this is more efficient, it must be remembered that the R2 of the correlation of temperature with the “predicted temperature” computed from a multiple regression is exactly the same as the R2 of the multiple regression. The significance of the simple correlation between temperature and the compound series therefore must be based on the underlying multiple, regression F statistic. When there are q regressors in addition to the constant term, this is F = (n-q-1)*R2/(q*(1-R2)), and has (q,n-q-1) DOF. If R2 is instead treated as if it arose directly from a simple regression, there will be an strong “data mining” or “wheelbarrow” effect that tends to make the correlation seem more significant than it really is.
A similar “data mining” effect arises if insignificant series are simply dropped from the predictive equation after pre-screening (i.e. given a weight of 0). This isn’t necessarily wrong, and may even be reasonable, so long as the omitted series are counted in the “q” of the final regression F statistic. But hardly anyone ever does this!
I must confess that it took me many years of teaching basic econometrics to figure this out — I at first thought that the “wheelbarrow” bias was due to the “-q” in the numerator, but this is minor in comparison to the “q” in the denominator!
(Since temperature is the exogenous variable and the proxies are dependent on it, it would in fact be appropriate to regress “predicted temperature” from the multiple regression of temperature on the proxies on temeperature, and then invert the regression line as in Classical Calibration Estimation. However, even though this beefs up the slope, the simple correlation coefficient and hence the regression F statistic is the same either way and so the significance of the slope is the same either way, abstracting from serial correlation.)
Correction — in a simple regression (q = 1),
F = (n-2)*R2/(1-R2)
and has an F(1,n-2) distribution, assuming normality, no serial correlation, etc.
Note also that MJ03’s procedure (which perhaps includes an atheoretic variance matching step) greatly attenuates the MWP/LIA signal that is present in Yang: The MJ03 composite as shown in the Eos note has only a roughly 0.2dC difference between the MWP and LIA, while Yang shows an approximately 1.0dC difference (See graph of Yang as smoothed by Craig in my SI for Loehle & McC at http://econ.ohio-state.edu/jhm/AGW/Loehle/SupplementaryInfo.pdf , p. 18.)
Yang’s series had already been calibrated to temperature by Yang, and hence there was no reason for MJ03 to re-calibrate it.
…..six geese a layin’…………
So a record which Bradley considered to be “crap”, and that has a truly woeful correlation with reality (!) of 0.22, becomes 1/3 of the final model result. I suppose the only thing you can say is that it was given a slightly lower proportion than the wholly excellent PC1 record :).
Wow, I realize that top chefs can whip up delectable culinary creations without exact numerical recipes, relying a lot upon taste, experience, and intuition…… but science is not cuisine (which is not to say there are no scientific aspects underlying cuisine, of course).
The output of Mann et al too often seem based upon hunches and guesses dressed up in scientific and statistical trimmings. It’s one thing to investigate based upon hunches, hypotheses, and even speculation…. but quite another to present one’s musings as reliable, well-tested “science” (relying in this case what Bradley had referred to as a “crap” proxy series).