RegEM has reared its ugly head again in Mann’s review of Burger and Cubasch.
An EM algorithm was one of the very first things that I tried when I started doing this a couple of years ago. When I tried to replicate MBH98, I got stuck on the temperature principal components even before the tree ring principal components. Both networks have problems with missing data. Mann had said that he had selected gridcells with continuous data records to enable PCA, but this simply wasn’t the case. Four gridcells in the HadCRU2 data set had no data whatever. (In passing, in light of recent Greenland temperature discussions, these 4 gridcells are all in Greenland and the difference in data versions bears examination. They were “nearly continuous” in the earlier HadCRU version used in MBH98 and had no data in HadCRU2 – what happened?)
I sent an email to Rutherford and then Mann asking for clarification without any success. I saw an iterative procedure for doing PCA in an image processing context on a data set with missing data and applied it to the MBH data set. The EM operation seemed to converge. This was the first mathematical thing that I’d done in almost 35 years, I was still learning my way on how to use R and I was quite proud of myself that it actually seemed to work. The answers were somewhat different from MBH PCs. I didn’t pursue it because of all the convoluted issues on the proxy side.
Later in the Corrigendum SI, Mann archived the original temperature data set, which is an interesting record of an earlier generation (HadCRU1 ?) data (and the only such record that I’m aware of). He also said that he interpolated missing data. From the earlier data set and using interpolation, I was able to pretty accurately replicate his temperature PCs. So my EM exercise seemed to be wasted as Mann had not used anything so complicated to fill in the missing values and derive temperature PCs.
However, it seems that the exercise was not totally wasted as RegEM appears to be a variant of this process using both proxy and gridcell data.
Let’s step back and consider the multivariate problem of relating a large data set X of (say) m=1082 temperature gridcells available over n=79 years in a calibration period to a proxy population Y of p=22 proxies available over n+N=581 years with the final objective of estimating an average NH temperature. Cook et al 1994 (the et al including Jones and Briffa) consider, as a base case, the OLS regression of each temperature cell on the proxies (Note that I’ve got the X and Y variables in reverse of the usual equation because, after all, tree rings do not cause temperature:
(1)
In our calibration period above, when you had 22*79=1738 proxy measurements, this would result in the calculation of 22*1082=23,804 coefficients. While Cook et al did not doubt the miracle under which each measurement could generate nearly 14 parameters (loaves and fishes, so to speak), even though they were climate scientists, they were able to see some risk of overfitting in such circumstances. After calculating 23,804 coefficients, the NH average would be calculated containing only 79 values – so, intuitively, it seems like a more parsimonious model should be available.
Cook et al 1994 then discuss using principal components to reduce the populations of both X and Y. This strategy is followed in MBH in only a piecemeal fashion. Temperature PCs are calculated with 11 of the first 16 retained. On the proxy side, Mann used PC methods to reduce some tree ring populations, but the majority of series in his Y matrix are used in their raw form. In the AD1400 network, 19 of 22 proxy series are original and only 3 are PC series. For now, let’s treat the Y matrix as garden variety proxies with the temperature gridcell matrix X being represented as principal components as follows:
(2)
Then under a Cook et al methodology, they would calculate a multivariate inverse regression (OLS) of temperature PCs on proxies as in equation (3).
(3) 
As regards the number of generated coefficients, in the 22-proxy and 1-PC AD1400 network, Mann would thus calculate only 22 coefficients (instead of 23,804) from the 22*79=1738 measurements (mutatis mutandi for 11 PCs calculated with 112 proxies in the AD1820 network.)
Now MBH98 did not simply do a multiple linear regression of PCs on proxies, but, as I’ve argued elsewhere, their “novel” method was, in effect, PLS (partial least squares) regression, a known technique in chemometrics (although they did not know that that was what they did and this is one of many things that I need to write up formally.) The differences between OLS and PLS are not as much as you’d think in the early networks anyway as the proxy networks are close to being orthogonal (some signal??). Thus
(4) 
Burger and Cubasch plausibly characterize distinctions between things like OLS and PLS (the latter, unfortunately, inaccurately characterized by them – this inaccuracy detracts from, but does not invalidate their results.)
Now my take on what’s going on with RegEM as described here is that it contains many of the problems faced in the Cook et al situation where there were an implausible number of different coefficients calculated on the back of a rather limited population of actual measurements, except that, instead of using OLS regression, they use ridge regression (RR). Thus, instead of (1), we have:
(5) 
I don’t guarantee that I’ve diagnosed this correctly, but it seems that in the AD1400 network, a grand total of only 1738 (79*22) measurements are used to yield 1082*22=23,804 regression coefficients plus 1082 ridge parameters as well. I’m not 100% certain that this is what they’ve done. Their use of idiosyncratic methods always make it hard to tell exactly what they’ve done. In this case, code has been provided so that it should be possible to decode what’s been done more expeditiously than MBH98, although the method itself is much more complicated. [Note: Code was subsequently provided]
In his Review, if one manages to get past all the ad hominems and irrelevancies, Mann’s main objection to Burger and Cubasch is that RegEM is “correct” and comparing results from other seemingly plausible methods is “spurious” or “erroneous” or “demonstrably incorrect” because RegEM is “correct”. Me, I like seeing what happens under other methods, all of which seem equally plausible a priori, relative to a method used by no one except Mannians.
But let’s see why Mann argues that RegEM is “correct”. His main reason(see S142) is that
“RegEM employs both a rigorous, objective regularization scheme and explicit statistical modeling of the error term”.
Now Jean S and I have spent a fair bit of time trying to decode Mannian confidence intervals and I will no doubt be forgiven if I think it prudent to investigate how the “explicit statistical modeling of the error term” is done in RegEM. As far as I can tell, in Mann’s own calculations, the “explicit” modeling is simply two sigmas (but I have to re-check this.)
My main point here is to draw attention to the consideration of error terms in Schneider 2001 (Analysis of Incomplete Climate Data, available online at Tapio Schneider’s website), who proposed RegEM. In his section 6, Schneider tested estimated errors from his methodology against a simulated data from a climate model (page 866). In this case, Schneider considered data set consisting entirely of temperature data in which:
In each of the nine data sets, 3.3% of the values were missing.
Under these circumstances, he reported that his error estimates were biased too low:
The estimated rms relative imputation error was, on the average, 11% smaller than the actual rms relative imputation error.
The underestimation of the imputation error points to a general difficulty in estimating errors in ill-posed problems. Error estimates in ill-posed problems depend on the regularization method employed and on the regularization parameter, but one rarely has a priori reasons, independent of the particular dataset under consideration, for the choice of a regularization method and a regularization parameter. In addition to the uncertainty about the adequacy of the regression model (1), the uncertainties about the adequacy of the regularization method and of the regularization parameter contribute to the imputation error. Since in the estimated imputation error, these uncertainties are neglected, the estimated imputation error underestimates the actual imputation error.
…
This underestimation of the variances is a consequence of using the residual covariance matrix of the regularized regression model in place of the unknown conditional covariance matrix of the imputation error (cf. section 3a). The residual covariance matrix of the regularized regression model underestimates the conditional covariance matrix of the imputation error for the same reason that the estimate of the imputation error in the appendix underestimates the actual imputation error: the error estimates neglect the uncertainties about the regularization method and the regularization parameter. To be sure, the traces of the estimated covariance matrices, on the average, have a relative error of only about 1.8%, but for datasets in which a greater fraction of the values is missing, the underestimation of the variances will be greater.
Now let’s compare this to MBH98. In the 15th century network we are estimating 1082 gridcells from 22 noisy proxies. In other words, Mann is going from a method that under-estimates error with 3.3% missing data to a situation where over 98% of the data is missing. Worse, it’s not “missing” data in the sense that you have actual measurements, but “proxies” whose connection to temperature is itself unproven in many cases.
If one actually reads Schneider’s exposition of RegEM, his caveats seem entirely consistent with the “flavor” problem of Burger and Cubasch. Having said that, I’m inclined to agree with the critics that the flavors are not as felicitously laid out as desirable, so it’s far from being the last word on the topic. I’ll re-visit the question of RE statistics on another occasion.
Climate Audit 
51 Comments
Honestly, the more I hear of Michael Mann’s actions, the more disgusted I become. So, now he’s pooh-poohing Burger and Cubasch for not fully divining his model?
First, he produces papers which re-write accepted history.
Then it becomes apparent that he hasn’t provided either the data or methods necessary to independently prove his “extraordinary” claims.
Next, when asked to provide that data, he refuses.
Finally, when others do their best to try and deconstruct his methods, he sits back taunting and criticising them for failing to properly replicate his methods!
Really, these are more akin to the actions of a magician or a grifter, when dealing with someone trying to guess how they performed an illusion or pulled off a con. They are most certainly NOT the actions of a serious, reputable scientist.
At the end of the day, it doesn’t matter whether you agree with Mann’s thesis, or not – surely no scientist can condone the way in which he has behaved these past years.
I would treat anyone who continues to defend him with an abiding suspicion, given what I now know of his behaviour.
You should ask Lee about his theory of “dual causality”, because in this new theory when X causes Y implies that Y can cause X.
Steve M.: You finished first in the Canadian HS Math Contest and yet didn’t touch math for 35 years?? You have been wasting talent. That’s not a compliment; it’s a criticism. And if you have the integrity that you appear to have, then it’s a very serious criticism.
Re:#3
Value judgment, Sara.
Just because you’re good at something, it doesn’t mean you have an obligation to pursue it.
re 2:
Steve, your co-moderator is at it again. He said to me directly yesterday that he is actign on your behalf in these posts. If you want to hide behind this guy to drive away dissenting voices, be honest enough to say so.
JohnA, I said no such thing. Put a lid on your mendacity.
I pointed out that showing that A causes B is NOT evidence that B can not cause A. It of course does not imply on its own that it does, and I never said that.
What was being argued was that since CO2 preceded temp in the ice cores, that this is proof that CO2 does not cause temp rises at all. I pointed out that teh fact that temp can cause CO2 rise (consistent with the ice cores and no surprise) does not tell us anything at all about whether CO2 can also influence temps.
Lee,
I comment on my own behalf. If you have an issue with moderation then speak to Steve. If you have an issue because I argue with you on statements that you have made then address me and stop whining to Steve that a “co-moderator” is being beastly to you.
JohnA this is what you said yesterday. Stop lying.
—-
L – ” Why Steve continues to tolerate your behavior is utterly beyond me.
J – Because he knows that without me, he will spend an inordinate amount of time trying to correct absurdist statements from people who have more time to argue the toss than he has, and who turn practically every thread no matter what the content into a discussion about whatever it is they believe.
re #7 (#5): Hah:
So Lee, would you care to comment RegEM/the actual post also?
Lee,
I think you’re getting confused by the difference between a “post” and a “comment”. What John A says in a comment has no bearing on what he’s doing in any post by Steve. Further, I’m pretty sure what John was talking about was putting questionable comments into a holding queue until Steve looked at then, not that Steve had to ok what John posts under his own name. (Though Steve, as owner of the blog, does have the right to modify or delete posts by John if he doesn’t think they’re appropriate.)
That said, while John’s taking a cheap shot, it is actually the case that you do claim (and are also correct in claiming) that something can both cause and be caused by another thing. That’s just a short definition of a positive feedback loop. You need to stop being so sensitive about John that you complain when he gets something right, just because you think the “tone” is mocking. IOW, don’t be so obvious when he’s pushed one of your buttons.
Jean, you will notice that JohnA began this with an unprovoked and false attack on me, in the second frickin post of the thread. I notice that my defense of myself gets called out as being off topic, but not JohnA’s attack.
If y’all dant want this in this thread, then call off Steve’s co-moderator.
And here is what Lee actaully said on “dual causality”
So X causing Y does not disprove the hypothesis that Y causes X. But “dual casuality” is fundamental to the hypothesis that Y causes or influences X.
Those are your words. Instead of justifying them, you’ll whine to Steve that I’m pointing them out and being “mendacious” for doing so.
I’ll leave it to our readers as to who is being mendacious. You’ll make up whole hypotheses as if they were scientifically obvious and then bridle when someone calls on you to justify them with reference to any facts.
Right, I am claiming that ther are positive feedback loops – and JohnA is lying aobut what I claimed.
Adn no, the discusisn was not just aobut the holding queue. JohnA was defending his style of posting to me (and others, I notice) in tht post.
I have had some good and substantive discussions here – some with you. JohnA’s behavior is obstructing that. I will admit that when JhnA posts misrepresentations of what I said, it ‘pushes my buttons.’ When someone posts something that falesly puts me in a bad light, I wil defend myself. JohnA is by far the worst offendor on this site, an dhe is whether he or Steve admists it or no, represetnatie of this site. Hell, he posts new threads, an dhe jsut claimed taht steve allows this behavior because it represents a work savings for Steve. It is, IMO, dishonest on the part of Steve to not, at the very least, muzzle the wporst of his mendacity.
Lee: your whining is comical. You need to grow a thicker skin.
John A: You are being unfair to Lee. He is a good warmer, although the belief is very strong in him and he will remain a skeptic of good science.
but his point about dual causality is not wrong per se. Warming can cause CO2 to increase, and more CO2 can cause more warmth. However, in the field that I work in this is not known as dual causality but rather feedback, which Lee has also made reference to. To see which is the primary cause and which is the effect with feedback, you would need to use empirical causality tests that run the model both ways and compare results. It seems that the theoreticians have posited both directions of causality, so at this point it would have to be resolved empirically.
snort.
So I’m being mendacious when I point out the basic fact, which seems to have escaped JohnA, that the question of CO2 – temp positive feedbacks is central to the question of climate sensitivity to CO2, and that evidence FOR the temp-CO2 part of that is not evidence AGAINST the CO2 – temp part. Too funny.
BTW, John Hekman, I used the “dual causality’ nonstandard erminology becaeu causlaity was what was under discussion. People were pointing to the ‘temp – co2′ (using the word ’causes’) part as evidence AGAINST the ‘CO2 – temp’ part, and I was pointing out that they are not mutually exclusive effects. BTW ( long shot) are you the John Hekman who used to race 505 dinghies?
jae, my skin is thick enough – I have no problem with rough and tumble discourse. I’m still here, arent I? But I will defend myself against falsely discrediting claims from mendacious a*****s..
No it’s not… unless you let it. If John A is your Peter Hearnden, then ignore him, as I do Peter. Indeed, John’s actual point in the message you posted (just before my earlier message which assumed you were talking about another message), is that he takes on those who hector Steve with off-topic stuff so Steve can concentrate on discussion with his statistical peers. Actually that’s my major “role” here when it comes to posting. I’m not very deft with (though not ignorant of)statistical mathematics so I concentrate on discussions, calm or agitated, with my peers. I try to be less ascerbic than John (though, I think, more cleverly sarcastic), however, so I don’t get noticed much. Most of my deletions are of replys to those who want to discuss things Steve isn’t very fond of discussing.
BTW, I tried claiming the Ou thread for some of these discussions, but so far have had not takers so I don’t know if Steve will allow it or not.
Lee, John A is acting on a personal basis below the masthead. He assists with computer software issues from time to time and will occasionally (but not often) put comments in a moderation queue. I don’t endorse his comments. If I’m not interested in what someone says, I generally don’t waste time arguing with them. It doesn’t mean that I’m acquiescing in what they say.
John A, having said the above to Lee, we’ve set out blog rules which call for people not to impute motives to other posters. “Mendacious” falls into that category. The rules apply to you as well. I don’t know who said what to whom, but both of you or either of you please stop forthwith.
#3. Sara, life’s a funny thing and it takes you down different roads. When I was 15-20, I was a talented problem solver – I was dynamite in things like math contests – , but I worried that I didn’t have any original ideas (Galois is a bad example.)
I certainly wouldn’t be able to do what I’m doing now if I’d had an ordinary academic career. You learn things in business too.
Last year, I posted up the names of a few articles published by a couple of my classmates in Math at university (actually Scherk and I were at high school together, so we had the same classes for 9 years.) So the types of articles that the world might be missing would be things like:
Maybe I’d have done a little better, maybe a little worse. But that would be one projection.
More likely would be that I’d drifted into mathematical economics. Samuelson phoned me from MIT to offer me a scholarship. My mother remembers this, as one of my siblings (then a baby was screaming) and she had no idea who he was. Michael Spence, who attented my high school, a few years old than me (who I admired greatly) recently won a Nobel Prize in Economics. His Nobel remarks are quite charming – I’m sure he’s the only Nobel laureate who thought to mention his/our high school football coach in this remarks. I must confess to some envy when he won a Nobel Prize as he definitely wasn’t as good in math as I was. His type of mathematical economics was another projection. Spence has always been successful in other ways as well, as he has had strong leadership skills.
I think that I’ve stumbled into something that I’m good at and that it would have been impossible to arrive at in a conventional way. Lindzen once observed that very few smart students back in the 1970s or early 1980s went into climate. IT’s too bad as there are many fascinating empirical problems that are theoretically interesting. But it would never have occurred to me as a young guy to study something then as unfashionable as meteorology or climatology.
I doubt that I would have enjoyed it anyway. I like my business friends – they’re fun. I went to an annual meeting the other day of one of the little companies from my old office, we had a fancy dinner afterwards, we laughed and joked and I really had a good time. I still might have to go back and make some money. MAybe I could try to sell Mann and Bradley some stock in a new deal.
It’s funny. When I was younger, I worried that I didn’t have enough ideas. Now I can’t keep up with them.
Sweet post.
There are no Nobel Prizes to be won in climate science.
Re#20, I can imagine people suggesting Al Gore should win the Nobel Peace Prize for spreading of message of “making peace with the Earth” with his movie.
Al Gore should also win a prize for inventing the Internet.
Maybe they should give him all the Nobel prizes in the same year – fiction, physics, chemistry.
.
My interest in component analysis stems from this as well. The connections to the DSP world are interesting.
Mark
To be fair to my pal Al, he did not claim to invent the internet… he merely claimed that he was the one that funded it and was therefore responsible for creating it. I’m not sure which is worse, however.
Mark
BTW, I had Al Gore at my wedding reception in 1995, and later he and his wife had dinner with us and he was pushing the internet….
Of course this Al Gore wasn’t the politician one (Besides, his wife’s name was Ursula). But he was a bright guy, active in MENSA. In fact he was hosting a Mensa meeting at his house that night so we let him take most of the left-overs from the reception to have as treats.
Re #18: Excellent comment Steve.
“You learn things in business too.” Yes, you learn that being right or wrong has huge consequences. If you are wrong too often in the business world you will have a mediocre career and not achieve material success. In academia, once tenure has been achieved, you can be constantly wrong and end up being a chairman emeritus of the department. Look at all the expert alarmist claims during the original 1970 Earth Day activities. Not a single catastrophic claim for the year 2000 came true including the coming ice age. Many of those alarmist Professors went on to have long distinguished careers being constantly wrong.
The biggest problem with climate science is the insulation of the experts from the consequences of being wrong. It is an academic problem. Tenure should be abandoned and much more performance based system should be developed.
Reid,
Tenure is meant to ensure academic freedom, so I would think hard about the removal of tenure. The more immediate problem is not tenure but a consistent betrayal of scientific ethics. No academic institution it seems, want to deal with clear cases of fraud and no-one wants to admit that fudging data, hiding data, refusal to cooperate with independent attempts to replicate, are now widespread.
There is a clear imperative to publish whatever the quality or probity of the work, which is how academic success is measured. Thus the imperative for any researcher is to hide mistakes, issues and confounding data that get in the way of that goal.
I don’t have a problem, for example, with the idea that Mann, Bradley and Hughes really were making an honest fist of a difficult problem in the reconstruction of past climate. I do have a problem with the subsequent behavior, in particular the hiding of information that the reconstruction was not robust and the verification stats showed overfitting and statistical insignificance for the large part of the reconstruction.
Was UMASS or University of Virginia in the slightest bit interested in such ethics violations? No. Neither was the NSF. Nor was the NAS Panel willing to look under that particular slimy rock. I don’t know what it will take.
John A: “Tenure is meant to ensure academic freedom, so I would think hard about the removal of tenure.”
Intellectual fashion is more important in the current academic system than being correct. Some fields are more mired in fashionable nonsense than others. I found academia to be chock full of intellectual sheep with inflated opinions of their intelligence. Academia needs a strong dose merit based compensation. I would like to see true academic superstars who tangibly improve society get millions of dollars in compensation from universities. And I would also like to see Ivy league Ph.D.’s who are constantly wrong demoted to teaching at community colleges.
The incentives in academia today leads one to be politically correct versus being correct. The two may be the same on occassion but they are often opposed. The incentive in academia should be heavily weighted toward being correct. In the internet age everyone has “academic freedom” as M & M have demonstrated.
I’m with Reid on this one. Tenure was originally designed to be a way to prevent discrimination against unconventional viewpoints. However, as with any broad rule, it gets used by the existing academia, those that sit on tenure boards, to prevent unconventional views contrary to their own. I.e. it is used as a way to keep out “the opposition” from the rest of the herd. I don’t have answers to the problem, unfortunately. Merit pay is certainly a good idea, but in the end, there is still some committee deciding who is “right” and who is “wrong”, and when it comes to controversial subjects such as climate, the herd still may have an agenda to push.
Mark
29, 30–Reid and Mark: Yeah, and the problem continues down to kintergarten. The whole public schools system stinks and is failing, due to the lack of any type of a special reward system for excellence.
Re #131,
The whining, and the appearance of a thin skin, may be parts of a technique of argumentation, but for those who don’t get taken in by it, the whining can be comical.
Errata for #32, the 131 should have been 13.
I think it’s pretty fun in a high school, “fight, fight, fight” manner. Sic ’em Steve!
Steve : What a lot of twaddle you right when talking about yourself . Here in Australia an adult male who talks about themselves like that would be met with derision if not violence. Stick to the science.
Re: #35
What happens in Australia to an adult male who wrongly writes “right”?
Sara asked me a question about myself in #3 that I was answering. It’s just a blog, for crissake. I’ve also posted about my mother going to Moosonee, 24 and Lil’ Kim.
And of the above, Lil’ Kim is the most worrying.
Not near as worrying as how violent some Australians can be over such minor issues.
Thankfully when I was there I never met any like that. Though a few did want to cut my hair off.
Man, are they all named Peter?
Actually the image in my post of selling stock to Mann and Bradley is not a bad one. Louis Hissink would like this analogy. I could show them profit projections with, ahem, a hockey stick shape.
Let’s suppose that I said that I had a “novel” method of calculating ore reserves. I could even mention that since high grade ore was what we were looking for, the method emphaszied high grade ore. But I’d tell them that my method of calculating ore reserves was my own personal method and I wasn’t going to get “intimidated” into showing the method.
Mining promoters would love to work under the climate science system.
Hey Pete W. Yo Momma!
#37 Steve, I went to Moosenee(sp) in ’80 on a summer long geology field trip. We were more or less following the Devonian Equator. Since we collected tropical fossils there, I would say it has cooled quite a bit since then.
I remember (barely) the train ride back when they opened the caboose, with a keg of beer and a piano. I spilt more than I drank, but apparently my professor didn’t. We found him the next morning having rolled down a 20 foot slope overnight.
I will be homeschooling, jae. Oh, and in relatively non-political subjects such as engineering, the tenure system probably actually works rather well. Engineering professors don’t often make headlines, either. They (soon we) busily publish away hoping to be the next Viterbi.
Mark
Yes, but being correct in the academic freedom in the Internet Age pays really poorly. The incentive isn’t just in academia, but also in publishing – the more controversial and ludicrously wrong a proposition, the more books it will sell. It’s the Ann Coulter phenomenon.
I don’t doubt that quite a few academics abuse their tenure with ludicrous propositions, but the problem is not tenure, but the extraordinary relaxed attitude to ethics shown by the institutions.
Re: #31 – Jae: You say “The whole public schools system stinks and is failing, due to the lack of any type of a special reward system for excellence”. The premise is questionable — did you know that in the 1950 in the US, only 50% of adults had a high school diploma? That’s up to about 80% lately. Do you have any evidence for your premise? Since I’m pretty sure your premise is wrong, I won’t bother commenting on your hypothesized mechanism.
Re #31 and 45.
Much of the US problem with the public schools system comes from the teachers unions, which won’t allow higher salaries for people capable of teaching scientific and mathematical subjects.
Another US problem is the number of hours students spend in the classroom over the academic year. I’ve read that it is significantly less for US students than for those in other sountries, such as Japan.
Roger Bell
Maybe they are just smarter than us?
Stop this discussion please or take it elsewhere.
We’re too stupid and poorly educated. We’re from the US. We’re not Canadian Fields Medal winners. Oh…ok…for you Steve. Last transmission.
#35: Please stop embarrassing us Aussies! We have a terrific record of scientific achievement that we should be proud about and shout from the rooftops (without fear of being bashed). I enjoyed reading Steve’s little trip down memory lane, as it was similarish to my own and it’s always interesting to understand how people have come to the point they have. I competed in three Australian mathematics competitions (not winning but making top 3% each time) and ended up going into business for myself, which I ran for 15 years before selling it a few years back. I must admit that I am biased towards the views of people who have survived the rough and tumble of the commercial sector over those theoretical practicians who seem to make more noise.
Steve M (#18), thanks for such a considered reply! I see your point about learning things in business. Like the importance of outside audits, for example…. Kindly appreciated, Sara
One Trackback
[…] self-contained and so I re-visited some of our previous comments on Rutherford et al here here here here . I’m going to reprise two issues today: Collation Errors, a small but amusing defect; […]