Pension funds all over the world – even university pension funds – are clamoring for the services of Rahmstorf and Associates as pension fund manager. No more Fidelity or Berkshire Hathaway. Yesterday’s men.
Confused by the market? Worried about your investments? Stop your worrying. Rahmstorf and Associates’ portfolio managers will separate signal from noise using a proprietary smoothing method invented by two Arctic scientists.
No more inconvenient portfolio valuations. No more adding up the value of your portfolio at the close of the month. That’s so one-dimensional. Today’s sophisticated portfolio manager at Rahsmtorf and Associates will use an embedding dimension of 15 years to value your portfolio. Your portfolio value is “better than you thought” ( (c) The Team).
Rahmstorf and Associates: Consistently outperforming the market since 1990. Remarkably consistent market outperformance since 1990.
Figure 1. S&P 500 with Rahmstorf smoothing. Thanks to Ross McKitrick for data and idea.
Climate Audit 
53 Comments
Too late!
Now you tell me.
I’ve just lost all my money on T. Boone Pickens windpark venture.
Now we know Rahmstorf’s secret on how he intends to win his “big money” bet he made last year on global temperatures rising by 2020. Pretty sneaky…
This represents remarkable progress in the analysis of the Earth’s stock markets. It shows that the robust signal in the market is going up faster than economologists were predicting just a few years ago.
Extracting trends from data is a key element of many econometric studies; however, when the best fit is clearly not linear, it can be difficult to evaluate appropriate errors for the trend. I think the method of finding a data-adaptive nonlinear trend has significant advantages over, e.g., low-pass filtering or fitting by polynomial functions in that as the fit is data adaptive, no preconceived functions are forced on the data. (Like, for instance, that the nonlinear trend should go down just because the data go down.)
Re: Ross McKitrick (#3), even more important than mere money, I’m sure there are implications in other fields too – like medicine. “After smothing the vital signs of the patient, her condition has been upgraded from dead to serious.”
Why restrict these methods to stock markets only?
I’d think these methods for finding true robust signals could be of extreme value for certain political leaders in the US, and elsewhere. Especially for discerning hidden, yet evasive positive economic trends.
From hadcrut:
From: http://hadobs.metoffice.com/hadcrut3/smoothing.html
Applied to S&P
Re: Jeff Id (#5),
see also http://www.climateaudit.org/?p=2955 . Poor unfortunate smoothers 🙂
Re: UC (#8),
Awesome, I see it’s old news on CA.
Smoothing brought to you by Enzyte.
[self snip] – I was half done writing a commercial but I got this one Steve.
Re: Jeff Id (#5),
They forgot – “Pay no attention to the man behind the curtain! The Great Oz has spoken!”
Re: Jeff Id (#5),
Heh, only misleading when it shows cooling. Funny how that works.
“My hobby: extrapolating”
http://xkcd.com/605/
What wonderful news! No need to complain to my broker. My money is growing faster than expected!
Priceless, simply priceless. My first laugh of the day.
Thanks. This is a big help in understanding Rahmstorf. I had a vague understanding that that was what he was doing, but I was totally baffled by the math discussions. If only someone would translate your posts into baby talk so the rest of us could understand the details. I know that’s not your interest, and it’s your blog, as people often point out, but your impact would be world-shaking if your amazing insights were made comprehensible to the masses.
And just last night PBS was telling me everything is fractals. Is there no-one you can trust!?
Very well done, Steve!
Sadly, this very sort of behavoir by a fund manager would most likely result in a jail sentence; whereas such behavoir in a certain brand of “science” results in accolades.
Where do you get this from, it clearly doesn’t agree with Fig 1?
Re: Phil. (#13),
Phil, you clearly need to either pay more attention to other posts when interpreting Steve’s humourous throw-off lines. 😉
Try looking at the Rahm-Centering: Enhancing “Successful” Prediction post .
Re: Phil. (#13),
You know Phil, one time a few years ago, I was commenting on a blog topic in my area of expertise, and the author of the post called me a “humourless pedant”. It was a bit of an epiphany for me because on reflection, he was right. You might think on this a bit.
As to figure 1, if I get a choice between the end of the red line and the end of the black line to value my portfolio, I’ll go with red. Even green is better than black, but the green manager, who was the star just a few years ago was apparently sent packing for lack of performance. I will leave you to draw your own conclusions.
Re: Phil. (#13),
Rahmstorf and Associates have moved on. They’ve changed their logo to Remarkably Consistent Market Outperformance since 1990.
Re: Phil. (#13),
I hope this doesn’t ruin the fun… perhaps this is a spoiler…
– black: actual data… but see, that’s got too much noise to be believable
– green: previous “noise free” portfolio balance
– red: new improved “more accurate” noise-free portfolio balance
As you can clearly see, the portfolio is improving far more rapidly than previously thought.
Such fun! With better statistical models we can more accurately see the real data trend.
This filtering can even substitute for viagra
Can you run a logarithmic version of that chart?
Wow. That’s gonna leave a mark!
Outperformance?
I need to upget and outwalk.
Too bad this all comes too late to save Bernie Madoff and his investors.
LOL. Nice summary of the issue!
Looks like a nice sell indicator. You have a nice rule. When the green line is under the red line, and the market falls below the red line, sell. Maybe you should package it and sell it to Wall Street. A further drop below the green line is a confirmation. Past performance does not guarantee future success, except in climate science.
I hate to be an arithmetical pedant in the face of this jollity. But the reason here for the different performance of M=11 and M=15 mostly isn’t to do with different treatment of recent market fluctuations.
The filtering is basically linear. You can think of the current trend estimate as either the derivative of a smooth or as an average of recent trends. Mathematically, you can differentiate through the filter. And with the latter view, it’s clear that the drooping of the M=11 curve is because the rise of the 90’s is passing out of its range, while it is still in the averaging for M=15.
So with R filtering, or any other reasonable filtering, you’ll see in the next few years a similar tendency to droop with M=15 as in turn, the late 90’s exits the range. It just happens later.
In fact, the hot 1998 plays a similar role in the climate plots.
Re: Nick Stokes (#28),
I wish I had the larning to be a pedant (humorless or not) but isn’t this the point about 1998 that others ahve been making about the R et al papers.
Re: TAG (#29), Well, you’ve been making that point. But contrast #30. The one constant is that R is a sinner. It’s pinning down the sin that varies. A lot of the scoffing could apply to any form of smoothing.
So to concede #30 first – yes, the caption to Fig 3 was wrong, has been partly corrected, and R is being stubborn about doing it properly, and is unclear in blog responses about filter width. But that doesn’t relate to this post.
On your point, 1998 happened and will cause some sort of ripples whenever it passes out of the scope of a finite filter. I guess your claim is that he’s upping M to put off that time. But it’s predictable – if he’s really trying to do that, he could have used M=15 consistently.
But what is lacking in all this R-scoffing is the scientific notion of a control. Plenty of people smooth. Is there any other smoothing which would do differently? Or does the scoffing apply to smoothing generally?
Partly answering that re this post – yes a more tapered filter, such as binomial, or the IPCC recommendation would reduce the effect. But it would still be noticeable.
Re: Nick Stokes (#31),
Nick, there are layers of issues here.
1. the interaction of Rahm-centering and Rahm-smoothing. This can’t be coopered up merely by labeling things. It undercuts his entire point.
2. (1) is an embarrassing problem because the problems with single-year centering were discussed at length in blog-world a couple of years ago in connection with centering on Hansen’s 1958. Gavin Schmidt is well aware of the discussion and has been noticeably silent on this dispute.
3. in my reading through filter literature, triangular filters are stone age filters and do not meet IPCC specifications (Chapter 3 Appendix A)
4. the seeming opportunism in the replacement of IPCC methodology with a methodology from the AGU newspaper, without any explicit justification or rationale, leaving them open to the criticism that the replacement was with a view to enhancing the result;
5. the embarrassing lack of knowledge of the proponents and users of this “novel” method.
It is one more illustration of a point made over and over here: the perils of Team climate scientists using home made statistical methods that they understand poorly in purporting to derive important applied results. This is hardly a new theme.
Re: Nick Stokes (#31),
I’ve been looking at several dozen land sites in the southern hemisphere, all plausibly rural enough to have no significant UHI, and I’m darned if I can find the 1998 spike. Study period is 1968-1998 inclusive. 1998 does not stick up in the Australian Antarctic, or Macquarie Island, or my home town of Melbourne. There’s a slight hint of a rise in 1998 Tmin in 15 other aggregated stations from all over Aust., offset by a low in 1998 Tmax. I’m working up another 24 stations and still have not seen it prominently. So far, all stations except a couple I’ve looked at have hotter years than 1998 after 1998. Not uncommonly, 1998 Tav is beow the 40 year average.
Yet, it sticks up in official Bureau of Meteorology Australian overall figures. There is a story emerging here. Perhaps you know the answer already, which would save me a lot of work. And please, don’t say it’s local weather as opposed to climate.
BTW, a fair portion of these sites show essentially no trend increase of Tav, some show Tmax diverging from Tmin, some are parallel, some converge. But most of all, there is so much noise that it’s incorrect to be definite about any level or trend.
Maybe this helps explain why the Australian Stock Exchange, while small on a world scale, is heavily traded from abroad. It does not matter so much how you smooth or end pad a straight and level line.
Roos McKitrick, you have a devilish mind. Congratulations.
Re: Geoff Sherrington (#39), Well, Geoff, if I’m not allowed to say it’s local weather, I’m fairly tongue-tied. I’m sure the BoM is using a reasonable spatial averaging formula, and their computers are accurate. Comparing individual records is tricky. You can get a situation where they all have maxima in scattered years since 1998. But if they don’t agree on those years, but do tend to agree that 1998 was warm, then the averaging will smooth out the scattered effects and enhance the agreed 1998.
Re: Nick Stokes (#41),
Thanks Nick, that was my initial thought, but as I work through more and more stations I simply find that 1988 was a normal year.
Maybe Steve might start a separate thread on the global locations that DO show an abnormal 1998 in case there is a pattern lurking there.
I regard it as fairly important in the context of how “global” global temperatures really are (given some problems with satellite coverage and assumptions for the last 30 years). It’s also important to attempt to explain a mechanism, like parts of the Antarctic missing it, because at the moment it needs mental gymnastics to link it to GHG. A month-by month 1998 analysis in different places might be one fertile starting point. Rate of change at a given place and time might also hold some info.
Re: Geoff Sherrington (#39),
Geoff,
According to the Bureau the 1998 Tav was the second highest (since 1910) at an anomaly of +0.81
2005 is the warmest at 1.06
Tmax was the 12th warmest at 0.54. It was warmer in 1928.
The 1998 Summer Tmax had an anomaly of only +0.05.
It was Tmin that showed the highest anomaly at 1.09 in 1998.
Re: Len van Burgel (#43),
Yes, I am aware of this. I simply have not encountered the stations to date that add to give Australia 1988 such a high. Do you know of any stations that have a really strong 1998 signal?
Re: Geoff Sherrington (#47),
I can only go by the climate data online, but it seems the minimum temperatures for central and northern inland Australia in 1998 were well above normal. Tennant Creek +1.2 Halls Creek +1.5 Balgo Hills +1.2, Yulara +1.5 Warburton +1.8 Giles +1.5. These are above the long term mean, so the anomaly from the 1961-1990 average may be a little different (especially Warburton). These stations are in a data sparse area and would account for a large part of the 1998 ‘record’ warm minimum anomaly of 1.09. As mentioned, the Tmax was nothing special that year. It was the Tmin which accounted for the high mean temperature anomaly.
Re: Geoff Sherrington (#39),
Geoff,
Alaska remains a contrarian state. The temperature data compiled by the University of Alaska (available at this link) show no 1998 El Nino response for most of the state. The only places where there is a pronounced reponse are Barrow and Kotzebue. The inland sites and the rest of the coastal stations are flat (or flattish) for 1998 but show a pronounced drop in 1999.
I compiled some non-NWS data from other Arctic sites to compare with Barrow. While they don’t show the prounounced decadal increase that Barrow does there is a common uptick for 1998. It would seem that the 1998 warming was local to the Arctic coast and didn’t influence the rest of the state.
By the way, the average temperature data for places such as Anchorage dramatically show the effect of the Great Pacific Shift circa 1976. For 1953 to 1975 the mean annual temperature was 34.8F, whereas for 1977 to 2008 the mean annual temp is 37.1F. Have a look at Anchorage temps and check out the 1976 shift.
Re: Earle Williams (#49), I think it could be very interesting to look at local temperature plots through 1998. I’ve suspected that the 2007 Arctic melting was a delayed effect of 1998, I seem to remember seeing recently a Morlet wavelet picture that fitted in with this idea. Sorry cannot locate for the mo. There was something right out of the pattern that happened in 1998.
Perhaps this should be for WUWT or Jennifer M not CA.
Re: Nick Stokes (#28),
I agree, there’s nothing wrong with the filtering. The only problem which could occur is if the filtering isn’t used with full and open disclosure of the method.
Oh yeah..
Congratulations Steve from a long time chartist.
I would have been out of the climate market very shortly after the 1998 peak!
Has anyone notived the double top? – 1998 and early in the 2000’s?
That’s a sure signal the the climate bull market is over and that we are in for a big (polar) bear market. It’s a bit hard to tell from the chart, which does not go back far enough to get the starting point, but as a rule of thumb, chartists look for a fall of 61.8% of the difference between the hi and the lo. (Fibonacci talk here!). To be concervative I would start with a 23.6% retracement and see if it gives any support areas.
I have a simplistic model that has picked all the really major highs and lows of the last 130 years in the Australian stock market. Probably just a coincidence, but it ignored the false November 2008 low and then picked the (appearently real) March 2009 low, so I have been putting my retirement funds on it. Silly me.
Seriously Steve, this is a marvelous, clear exposition of the current situation. Well done Dan.
Re: Ausie Dan (#32),
If you look back in old posts, I did a rather pretty “multiproxy” reconstruction using the PC1 of tech stocks and white noise to make hockey stick graphs with high RE statistics. They consistently outperformed MAnn’s bristlecone-based reconstructions.
Re: Ausie Dan (#32), A lot of temperature bulls went long the surface and lower troposphere when the chart broke its earlier resistance line in 1998, but the El Nino bubble was short lived. Once it blew off its parabolic top it completed a head-and-shoulders and then was channel-bound until about 07, at which point it breached its lower resistance line. From there it’s been air all the way down through the first and second Fib retracements of the 83 low. Despite rumours of GISS quants front-running the tape, there’s a bearish pennant forming on the TLT.
Brilliant, Steve.
I’ve had my joke. Now to be serious.
I know Steve Mc won’t allow politics here, but please bear with me for a minute.
This is important.
Surely some of you statisticans know one or more senators. Most are lawyers who cannot understans science, but everyone in power understands the stock market. This must be made known to them, before they consider the cap and trade bills.
I intend to pass it on to the independent Australian senator, Steve Fielding,who is leading the resistance in Australia. I wholely disagree with all his other policies but he is spot on with global warming hysteria and has gathered a small group of like minded scientists behind him.
Dan
Ross, if you visit the citation here here (#76), a history of time series analysis stated of smoothing methods:
And to think I’ve been beating my head against the wall for twenty years trying to get a little edge in the world of trading and two arctic climate guys crack the whole thing wide open without even a little bit of sweat. Ouch.
Whoaaa, there should be a health warning on this post!
.
I’ve been with Rahmstorf and Associates’ for a while but needed cash. When I closed my positions I got back 50% of the value of my portfolio!! When quizzed they explained that the underlying fiscal realities not necessarily having a linear relationship to the portfolio value, and this was clearly made in an email a friend of theirs might have gotten with the valuation software from some high latitudes experts, but if that was the case they hadn’t seen it and weren’t aware of it. Nice guys though so whilst I’m now destitute at least I won’t need so much money for heating according to recent projections.
Lovely smoothing. Brilliant.
But it is also a serious lesson to be learned from this.
Is the climate best described by global average temperatures? Is the natural climate changes best described in global temperarture changes of tenth of degrees?
I dont thinks so.
The climate for mankind are best descibed by the extreme conditions. Not by the average conditions. The variance may be more intresting than the average.
A proper smooting will make more sense for the extremes than for the average.
Will you please add an IPCC filtered line to the graph?
Geoff Sherrington
Along the west side of NZ the effect of an El Nino tends to be increased west to southwesterly wind,numerous depressions and cold fronts welling up from the Southern Ocean, cool surface temperatures and substantially increased precipitation. There is a “teleconnection” but the sense in inverted. This same inverted teleconnection occurs during La Nina when the west side of NZ is less windy, slightly warmer, and a little drier.
Perhaps the signal you should look for in Australia is not the classical strong upward temperature spike. In fact when one thinks about it, if the BoM mean is showing such a spike, it might be evidence of a doctored record.
Cheers
Rob R
Regarding #49 (Earle Williams) I looked at various Alaska data sets some years ago, and noticed the climate shift – which is not a difficult task! However, on careful analysis it seems to have occurred immediately after October 1976, the upward step being obvious in the November data for places like Anchorage and Fairbanks. Several other minor steps can also be found but the Oct/Nov one is the most fascinating.
What was the cause of this phenomenon? Are there any references or links to papers that attempt to explain why it happened? It seems to be a feature of south coastal and interior areas, but not for the far north west. Mysterious!
Barrow does not show this step, but two others can be found. July 1977 and 1993. Since then there has been no discernible change
Robin
One Trackback
[…] Your Portfolio is “Better than You Thought” […]