Mission Impossible Team, here’s your assignment today. Unfortunately you failed your last assignment: replicating Mann’s claimed correlations. But that probably was impossible. Today your assignment is probably possible, but is a dangerous expedition into the dark underground – into the terrifying world of Mannian RegEM. Courage and perseverance will be required. You may not return alive.
Let’s start with something simple and seemingly innocent – a location map of Loehle’s proxy selections. This shows the locations of 18 proxies in various locations around the world. Three “regional” proxies – two in China and one in North America – are shown at their approximate centroid.
Next here is a location map for the 22 proxies in the Mann et al 2007 network (the network formerly known as MBH98). As with the Loehle network, the two North American PCs are shown in locations near their approximate centers of gravity.) It’s more land-oriented than the Loehle network, perhaps a little more North American-oriented, but it looks innocent enough.
Although Mann et al said (and Zorita et al 2003 unfortunately adopted this point) that it was impossible to allocate weights to the individual proxies, this is untrue. Vast quantities of linear algebra cancel out enabling the NH reconstruction to be expressed as a linear weighting of the individual proxies (some with negative weights.) In the graphic below, I’ve made the area of each dot proportional to the MBH98 weight of the Mann et al 2007 proxy. The visual impression is obviously entirely different. The point here is one that’s been very clear to me for a long time and I’ve tried various ways to convey this message, but I think that this image may finally enable a wider audience to fully understand what the “more sophisticated” (using JEG’s words) algorithm does.
The MBH98 network is essentially a combination of only 4 series: the North American PC1 (which is 95% Graybill bristlecone chronologies), Gasp” – a questionable series, which had an unreported extrapolation to get it into the AD1400 network (this is the sort of unreported accounting adjustment that sets off alarm bells in the real world although not in real climate; Briffa’s Tornetrask series (which was manually altered by Briffa in Briffa et al 1992 – an alteration reported but nonetheless unjustified; and Cook’s Tasmania ring width series. The other series are just nothings under MBH98 weighting. Some negative weighted proxies (e.g. ice core accumulation) have a plausible physical interpretation as inverted. In my opinion, these can be counted as positive correlations and arguably the negative of the proxy should be collated. One should not otherwise invert things like tree ring proxies after the fact.
Now you’ve all heard that Mann can “get” a HS without using PCs. Again I’ve observed in the past that this is really just a trick for using the bristlecones, but this point can be illustrated rather neatly using the graphic technique of the above maps. First here is a map of the locations of all 95 series in the Rutherford et al 2005-Wahl and Ammann 2007-Mann et al 2007 AD1400 network, with the size of the dots set so that the total dot area is equal to the other two cases. You don’t have to be Ethan Hawkes to notice that the no-PC network is predominantly in the U.S.
Now in the world of temperature histories that we’ve been exploring from time to time, we know that the U.S. temperature history in the 20th century has a different course than the ROW; indeed, in the US, according to NASA, 1934 was the warmest year of the century (inquiring minds want to know whether it was the warmest in a millennium or even in a millllll-yun years). But Gavin Schmidt, functioning both as NASA spokesman and as an able real climate spokesman, has told us, as has Hansen, that the US constitutes only 6% of the global land surface and 2% of the surface and is thus unrepresentative. One would presume that the weightings in the no-PC network would downweight the over-populated U.S. proxy network to reflect the lack of geographic balance introduced by abandoning PC networks – even if it meant downweighting the bristlecones.
OK, I was just teasing. I didn’t really expect you to think that they’d downweight the bristlecones. Here’s what the weights work out to in the no-PC network. They are – if anything – even more concentrated in the U.S. southwest (bristlecones) than the unweighted distribution, which was already heavily weighted to the U.S. The main HS-shaped Graybill bristlecone chronologies each strongly impact the final results, so that the early portion of the MBH reconstruction – whether modulated through Mannian PCs or in the no-PC situation – still is essentially the sum of the Graybill bristlecone chronologies. Mann’s incorrect PC method is not the only method of overweighting the bristlecones.
What caused the MM reconstruction to differ was that different weights were assigned to the proxies – with less weight to the bristlecones, other series, especially Tornetrask, Quelccaya and (oddly) Tasmania gained weight. It wasn’t that we believed tha a weighting of these series was specially wonderful as a reconstruction, but that the recon was unstable to bristlecone weighting.
You can immediately see the problems that the Ababneh update of the Sheep Mountain chronology pose for these weightings. The image below (previously posted) compares the Graybill Sheep Mountain chronology (the most heavily weighted series in the MBH98 network) to the Ababneh version. Merely substituting this version for Sheep Mountain in other networks already accounts for change. In this case, there are a few other highly weighted bristlecone chronologies, but all are by Graybill in the 1980s and the replication problems at Sheep Mountain should be setting off alarm bells for the other chronologies. (Of course, there is heavy security on the updated Sheep Mountain measurement data, security which will be too hard for even a Mission Impossible Team.)
Although this network was first used in MBH98, it has been applied subsequently in Rutherford et al 2005, Wahl and Ammann 2007 and most recently Mann et al 2007.
In each new guise, it has become harder and harder to determine what the weights of the individual series are in the final reconstruction. Mann et al 2007 says that it is impossible to know, but Tapio Schneider has observed that, while the algorithm is nonlinear, the regression coefficients can be extracted from the RegEM algorithm.
So, Mission Impossible Team, your task is to determine the weights of the individual proxies in the Mann et al 2007 RegEM version and to plot them on a map such as the above. Your instructor, the brave and resourceful JEG, has unfortunately gone missing, perhaps swallowed up in the swamp of Mannian pseudo-covariances. But before going missing, he left us with the message that the failure to show location maps was pseudo-science. (Yes, he didn’t say that the failure to show a weighted location map was pseudo-science, but he’d probably have agreed that it was a good idea.) So, Team, in memory of JEG, return with the weighted location map for the AD1400 Mann et al 2007 RegEM proxy network. The fate of the planet depends on you.
Climate Audit 
58 Comments
What happens if temperature is converted to heat, using the average levels of humidity and assuming that the troposphere is only 7 km high at the poles but extends up to 17 km at the equator. Does that have a big difference on the weighing of the proxies?
Craig Loehle’s straight-arithmetical averaging of his proxies looks a lot better than Mann et al’s shenanigans.
Now, with a few more SH proxies, especially in S. America, he really would have a global reconstruction…
Cheers — Pete T
Steve:
Given the known global variance in anomalies, doesn’t the paucity of the proxies argue for large error bands on any multi-proxy study? Your map just drives this home with the obvious unrepreseantive nature of the sample – independent of the fun and games with the weights.
Great maps Steve. They make the manipulations very clear to me, and I’m sure others like me, who are not masters of statistics.
Steve, it seems to me that you need to make the *area*, not the radius, of each dot proportional to the weight it order to properly visually represent the weightings.
Steve: Actually I did do that. The radius is proportional to the square root. I’ll edit the text.
How were these maps generated? In R Project?
Are the proxy locations long/lat co-ordinates?
It looks like awesome software if that is the case.
Steve: Using the package fields. Lat, long and weights in table wproxy.
It’s interesting to see again how very few Southern Hemisphere proxies are used, but then again, the Hockey-stick is a Northern Hemisphere only graph. It would be interesting to have more Southern Hemisphere proxy data, but I guess gathering it is too inconvenient.
Steve,
You forgot to add:
“This tape will self-destruct in 5 seconds”
Great maps, Steve. Now I can tell people where to see maps when they do not understand what I am saying.
Also – if the southern hemisphere was weighted like the bristlecones – I bet that would flatten the stick a bit.
My guess is that JEG wasn’t prepare to deal with real scientist that answer to criticism that quickly.
In less than a week all is concern were adressed. Data and graphics are now available. That it should have been done from the start become secondary when the situation is corrected.
And is now speechless to explain Mann’s behavior. Better go into ignoring the reality it hurts less.
# 27
Steve H,
Of course, we can find in the mud the chemical tracks of the chemical and physical conditions and variability of the environment. That’s the raison d’être of Dr. Loehle’s work. He didn’t ponder the proxies derived from tree-rings, but from isotopes, diatoms, foraminifera, etc., which makes his research consistent and trustworthy.
I was in the military for 20 years and Colorado Springs is a military town with many personal friends.
My question for Steve: If I can obtain those lake mud samples, would you be willing to publish the results?
No matter what they show?
Steve: Sure. I promised the dendro results whatever they showed. It’s no skin off my nose whatever they show.
shunt1@mchsi.com
Contact me personally before you delete my last post……
A principal author on denrochronology in Tasmania has been Brendan Buckley. I have written to him asking if he is willing to provide raw data for further analysis. An abstract of some analysis follows from the 1997 paper
A CHANGING TEMPERATURE RESPONSE WITH ELEVATION FOR LAGAROSTROBOS FRANKLINII IN TASMANIA, AUSTRALIA by B. M. BUCKLEY, E. R. COOK, M. J. PETERSON3 and M. BARBETTI
Steve: Buckley has archived data at ITRDB for example: Huon pines here. You check the existing archive first. ftp://ftp.ncdc.noaa.gov/pub/data/paleo/treering/chronologies/australia/ausl023.crn
#14 above did not reproduce the text in full, but the last para is mine, not from the quote. There is enough info to start with. Geoff.
i fear i ll have to repeat this again:
two of the Loehle proxies end in the 19th century.
http://www.climateaudit.org/?p=2382#comment-162999
they don t simply go “hockey stick flat”, but completely flat line for the whole 20th century.
wouldn t it be at least sort of fair, to give these proxies some special treatment? like the bristlecones?
or at least to mention the problem now and then?
Steve: You’ve mentioned it. This really doesn’t have anything to do with Mann weighting which was the thrust of this post. Do you have any problems with this being moved to a Loehle post where it belongs?
i have no problem with you moving it (and/or erasing this post)
but reading your post and comments, i am left with a feeling that the Loehle proxies are “better distributed” and with “less flaws” than the Mann ones.that is simply false.
i was talking about comments in this comment section.
My eyes! My eyes! (I still feel ‘as ill as ever’ reading sub-par grammatical execution like this, never mind any *possible* deceiving, mis-directing or ill-intent of the poster; I can only surmise that the goal here being an infamous ‘meddle* of freedom’ award for being voted off-the-island with continued nit-picking diversions that sap the energies (and patience, patience: the ability to endure waiting, delay, or provocation without becoming annoyed or upset) of posters that might otherwise better be directed elsewhere, and I speak as one who wants to look at Wein’s law/Stephan-Boltzman Law/spectral energy distribution in more detail under the influence of … but I digress)
*intentional spelling: meddle, http://www.thefreedictionary.com/meddled
Okay, I’ve decided it’s just me. I will refrain forthwith from any more commentary on this subject.
Re #10, Sylvain, JEG’s absence
In fairness, the last post at his own blog, http://thatstrangeweather.blogspot.com/ is Nov. 18.
Cheers — PT
Re #10
JEG was probably surprised and must have been disappointed that his review was pre-empted by my own equally critical review. But as an Aikido master, I think he was prepared to deal with this. It is not true that all of our concerns have now been addressed. We will need to wait to see what Loehle and JEG come up with in terms of bootstap confidence intervals. Last we heard from JEG he was running some Monte Carlo simulations. I suspect he’s busy. Good science takes time.
Steve,
I thought people might be interested in this excerpt from a paper by the late Jonn Daly regarding the provenance of the pine tree data from Tasmania.
The data used by Mann is, I believe, Cooks Johnson lake (Mt Read) material that in Mann,s data set starts in AD 900.
As opposed to Buckleys Lake Marilyn set that starts in AD 1058.Is this correct to your knowledge (just to avoid any confusion)?
People might also be interested in where the calibrating temperature records came from, considering that Tasmania is the size of Scotland and Hobart is the second driest city in Australia.
On a personal note; I would like to thank you for this blog. I live in a small country town in Australia and for one reason and another had to finish my formal education at the age of fourteen. The wealth of information presented on this site has kept my brain ticking over furiously for over two years now. Other posters have likened this site to an online university (yes) and that trying to assimilate the information is like trying to drink from a fire hose (I loved that one).
However, I think that in the tradition of a number of scientists that didn’t act in a condecending manner towards “trolls” , you have tried to present science in an accessible manner for the lay person without dumbing things down to the point where polemical points are obscured.
Thank you.
The `Hockey Stick’:
A New Low in Climate Science by John L. Daly
http://www.john-daly.com/hockey/hockey.htm
Steve: Thanks for the kind words. I find these matters interesting and am glad that others do as well. I don’t think that one can regard the CO2 fertilization as established. It’s controversial.
Fig 7 in #22: Look at that warming trend from AD900-1000. Still unprecedented?
Steve,
Re post 22
I understand your point about CO2 fertilisation and I agree with it.
I wasnt trying to endorse all of Dalys observations, but I certainly was interested in the locations chosen for the calibrating temperature records. Hobart is the second driest city in Australia and is about 100 miles or more away from Mt Read. It has about 3000mm less rainfall per year than Mt Read and is also a very large population centre – Curious – given Dr Mann’s weighting of these proxies.
I also understand that the numbering of the figures is a bit confusing; John Daly kept the numbering from the original authors in the the figures but numbered them differently in the context of his entire essay.
I was very reluctant to tamper with the original document to make the exerpt make more sense and can only suggest that people look at the whole article and make what they will of it. As I say, I was particularly interested in the provenance of the Tasmanian tree ring data that was related to the theme of this thread.
I indicated on another thread that I have quickly realised that the old saying that it is “better to say nothing and let people think you might be an idiot rather than open your mouth and leave no room for doubt” might have some substance. I will however still be lurking for a long time to come.
My tag by the way relates to something that Gould once observed – people struggle with the idea that the world is very old – but the top of Mount Everest is composed of marine shale.
Thanks Steve
hm. looks like 1°C increase over 100 years. the end of the curve seems to show a similar increase in 50 years.
Im sure JEG has plenty of things to do (like his own research). The 8001* class doubts that he has been swallowed into anyone’s swamp of pseudo-covariances.
*not 8100 – maybe this is why you havent received any responses… somewhere there’s an intro to environmental sciences class scrambling to decipher what Steve means by “weighted location maps.”
Hi Steve,
the elusive JEG popping in for a minute, after research-hours.
My oh my, you have been posting with Stakhanovist productivity ! So much so that people are right to state i cannot keep up with it, so i will make this brief.
I mentioned your assignment today to the class, which was surprised you were part of the staff at GaTech. But there are so many things we are ignorant of, that i do hope you will forgive us that one.
You are quite right that i never said “that the failure to show a weighted location map was pseudo-science”, so i wonder what argument you are making there. It’s not the first time you put extraneous words in people’s mouth : is this a proof by numbers ?
I do, however, agree that said maps are a good idea. A very graphic way of representing this set of weights. Congratulations on a useful piece of work.
Now to the “hard” questions you raise : since, in the case of MRWA07, the authors have so diligently made their data available to everyone, and since the RegEM code is public, how “Impossible” is it for a beautiful mind like yours to go figure this out, and plot a similar map next to the aforementioned ?
Why should you ask a class of students – undergraduates and budding graduates – to go do your own homework ? And why do you address them by the derogatory “Team”, as if their soul had already been corrupted by Mannphistopheles you despise so ? Are you denying them the right to come to their own conclusions re: the Hockey Stick debate ?
If so perhaps Prof Cobb and I should stop encouraging them to peruse your pages, which we had heretofore found quite informative.
You know, when i checked CA this morning i felt behind in terms of keeping up with your posts, but after this sort of plaisanterie i feel much less guilty. I was just chastising people on my blog for making McIntyre-bashing comments, praising your many scientific qualities, and how much you bring to the climate debate. I would hate to have to take these words back because you value petty vendettas over scientific investigations.
The real issue is this : i am but an insignificant pawn on this checkerboard, Steve, so if your Audit has as much substance as you contend, then your bright intelligence, tenacity, and flair for writing should destroy my arguments (and those of the entire Team) without recourse to small-change intimidation and cheap jokes. Please don’t let these tendencies over-run what is otherwise an excellent blog that could foster amazing progress in climatology.
Steve: JEG, we enjoy your company here. First of all, I don’t “despise” anyone in this – why would you think that? Don’t project the feelings of others onto me. I disagree with people but I really don’t despite them. At the NAS panel, I made a point of going and chatting to Malcolm Hughes – we discussed pleasantries like a comeback by the Liverpool soccer team. Had either of Mann or Hughes showed up for the NAS panel reception, I would have chatted to them. I chatted with Hegerl and D’Arrigo. At the House hearings, I made of point of going over to say hello to Mann and introducing myself. I saw him on the street at AGU last year and said Hi, Mike, but he walked by stonily. I bought lunch for Caspar Ammann at AGU in 2005. I disagree with them, but I don’t “despise” them. I don’t know that they have quite the same equanimity, but that would be their problem.
You ask the question:
Last time, I looked in the mirror, I was not enrolled in any courses for credit nor have I been for many years. So I don’t have any homework. Am I personally going to try to operate Mann’s code? Not right now. I don’t have Matlab; a couple of CA readers do. They’ve been trying to get it to work and I’ll wait for them. My guess is that the weights will be like MBH98 based on the similarity of the recons. After all, even the RegEM recon is just a linear combination of the proxies. For students that are trying to understand how these algorithms work, I thought that it was rather a good exercise. Sure I wrote it in a lighthearted way, but it is a good exercise for all that.
JEG, how are those Monte Carlo simulations working out?
#25 Yes, same level rise in shorter time frame, but coming out of the little ice-age. Now put some confidence intervals on those curves, sod. How high might the rate of warming have been from AD900-1000?
JEG, is the Tasmanian tree-ring work in #22 Fig 7 “pseudoscience”? It does not include confidence intervals.
JEG,
Exactly why do you assume your class at GT is the “Mission Impossible Team?” Why don’t you realize that much of the “homework” is rhetorical in nature, hence statements such as, “Unfortunately you failed your last assignment: replicating Manns claimed correlations. But that probably was impossible.”
As much as many posters here have mentioned your name and your class, this blog is not about you.
Hi bender,
Monte Carlo doing good. I’m thinking of something more “sophisticated”, however 😉
Steve,
I am happy to hear that you don’t despise any of the Team members. It’s not always obvious from your posts.
Now, i agree with you that if the class was about replicating Mann’s work, yours would be a great suggestion. Last i checked, however, you were not on the teaching staff at GaTech, hence a slight surprise to see you enunciate their assignments.
You have a very good point regarding the excessive weight of certain Colorado trees in the ‘NH’ reconstruction. That weight would be justified if it meant that they are somehow recording NH temperature better than any other proxy – but you and several dendroclimatologists have shown that the divergence effect is marring such records in the late XXth century, so their weight should be accordingly revised.
I regret that Matlab is not as freely available as R. Sadly, this does not make Tapio’s code as “public” as i claimed. Hopefully some CA readers will be able to help you there.
For different reasons, i have a hard time following your arguments sometimes, as R code is like Aramaic to me.
Again, a question of language…
Michael Jankowski ,
“Mission impossible team” was equated to said class EAS8100 (a typo for EAS8001) in the post’s title. Or am i jumping to conclusions ?
Of course this blog should be about auditing Climate and not auditing JEG . My work, yes, go crazy. Other things are simply inappropriate.
Hints?
#32. A couple of comments from a CA reader on his replication efforts so far with the archived code:
Secondly:
JEG, you say:
You’re mixing up a couple of different issues here. The “Divergence Problem” is the failure of ring widths and densities for the majority of tree ring sites that are supposed to be temperature indicators to keep pace with temperature.
The Graybill bristlecone chronologies were the opposite: they had growth increases that were higher than could be accounted for by temperature. California bristlecones are the main ones, Colorado is secondary. I sampled the Colorado ones only because I was visiting my sister in Colorado Springs.
Your position on the Graybill strip bark chronologies is that “their weight should be accordingly revised”. The recommendation of the NAS Panel was that they should be “avoided” i.e. their weight be reduced to zero. What is the basis for your rejection of the NAS panel recommendation or do you agree with it?
Prediction: Crickets.
Why back-pedal when you can dodge?
And weave! Serpentine… serpentine!
Mark
Ok, you got me there! I maintain the rhetorical aspect issues with the “homework” 🙂
JEG– I’m trying to teach myself R. It is, indeed, obscure! When I read the various guides, I wonder if a person is required to first learn oop (object oriented programming) before dealing with R (aka, ‘the beast’).
In about a month, I’ll probably know whether it’s worth dealing with the learning curve. Meanwhile, the simplest things like trying to read a file or take an average seem horrific.
On the irony & the class assignment. …. the Wall Street Journal recently ran an article on teaching people to understand those from other cultures. It appears that Americans, British and Danes use irony much more often than most Europeans or Asians. The article didn’t mention Canadians, but it appears Steve is someone inclined toward irony. Even if your class didn’t let on, if they are at all bright, they likely suspected irony in the title.
But you already knew that, right? 🙂
The homework assignment is for those here on CA that are so inclined, I beleieve.
Seems to assign it to the “EAS8100 Mission Impossible Team” is a bit of satire. Steve’s been known to do that before from time to time in his titles, and to have a bit of fun with it in the article. Makes the subjects a bit more interesting I believe. But that’s just me.
RE 38. Irony meets camp. The non scientific aspects of this amuse me endlessly.
RE 33. hints? I think he is going to try more gum.
I posted this on the wrong thread and while it is not really worthy of a second chance, I am growing impatient with what I see as JEG’s failure to engage as a climate scientist.
JEGs ability to turn smart phrases while verbally tap dancing his way around this blog is most appreciated and an unexpected bonus coming from what one might expect to be a rather matter-of-fact climate scientist. His approach, while smart-alecky in context, does allow one to take a personal broadside and still chuckle at his remarks.
My question remains: after this flamboyant display of JEG, the erudite, and the urbane, will we be privileged with the presence of JEG, the serious climate scientist, here to answer rudimentary questions (and particular those posed by JEG, the viewer of misconceptions by CA participants) like, for example, a simple explanation of how teleconnections have been validly applied to a temperature proxy.
RE 41. Kenneth, the hilarity of JEG is that he tries to be urbane and erudite. When he mentioned
Wittgenstein I knew exactly what JEG was.
@Kenneth Fritsch–
My take: JEG appears to be taking what is said here seriously. He won’t be back frequently because a) he has a job and b) he is actually computing stuff.
Yes. He has a certain manner of expressing himself and sometimes sounds like he’s doing an imitation of Hercules Poirot. But that’s kinda’ fun, isn’t it?
I’ve got no beef with JEG, he’s just a little too confrontational for my tastes. I’m thinking he’ll be back, and that’s cool.
RE 45. Most people who mention wittgenstein in non philosophical conversation have
never read wittengstein. They have read “about” him. They have perhaps heard the term
“language game” and think they understand one of the quirkiest thinkers of the early
2oth century. It’s an indicator of faux erudition.
#38. Lucia, irony is a tone that I try to achieve.
I couldn’t disagree more. You can take an average of a vector x by:
Or you can standardize series by:
What could be simpler?
There are several read functions that I use. All can read directly from internet url’s
read.table (“http://…”, ) is versatile. If you want tab-separated, use sep=”\t” as an option.
read.fwf (url, widths=c(6,8,5) ) will read from fixed format files. fill=TRUE is a useful option in both cases. skip= is also useful.
read.csv is handy
I love the read functions in R.
The only thing that is slightly troublesome are Excel tables but that’s true of anything. I save them locally in tab-separated format and then use read.table( …, sep=”\t”) to retrieve.
Re: #43
If Hercule Poirot were a climate scientist who had come to CA to comment, I would answer him thusly:
Then please shows us how it is applied to climate science.
Ah, truth that is what we seek here from within and/or without. Please show us how you do it re: climate science.
No need for further demonstrations on that matter.
Then please answer, mon ami, my simply query on teleconnections.
bender,
I think you will find that JEG has been doing his Monte Carlo experiments in relation to ENSO teleconnection modulations caused by variations in IPO, PDO etc, (very large numbers indeed JEG).
Perhaps he found that there was too much chance, chaos and statistical fluke involved and is looking for a better way.
Can you please correct me if I am wrong JEG.
@steveM–
Well… the files you download off the web must be much more conveniently organized than the one I downloaded to teach myself. I can definitely get the mean values of a vector– or whole column of numbers–, but I always find myself wanting to calculate conditional means and discovering a drop out because a thermometer must have broken. So, then I have to google how to do something to create the vector given the information actually in the file so I can later stuff in command like mean()!
(It’s never hard in the end, but… well.. googling to find out how to do things is not that efficient. Why in the heck did Rproject organize their help pages using frames? Did they want to make it impossible for search engines to send users to the useful pages?!
There are lots of cool things about R, but other programs that permit selecting columns etc. are easier. (Though, ultimately, horribly limited.) Mathematica is better documented for someone who doesn’t want to read extensive discussions about how the package creates objects and that about that things are classes and subclasses or something or other.
I’ve subscribed to the email list, but I clearly need some sort of decent book to flip through. (Preferably one with real examples printed out. )
It’s really really efficient with matrices. Learn the functions apply, tapply, sapply,,,
apply(X,2,mean) columnwise means; apply(X,1,mean) rowwise mean.
Data.frames are often ragged. tapply(hurricane$wind,hurricane$id,mean) – average across a ragged array.
Liuke everything, it depends what you’re used to.
Many functions are like Matlab – I can follow Matlab code without ever learning it.
The packages are an immense resource and indicate that a critical mass is being gained.
SteveM–
I’ll look those functions up, but…
Do the files you find always have precisely what you want and only what you want in a row or column? I grabbed a file with 83 years worth of daily temps. I’ll eventually want average over full years. (Done that.) Average over individual days of the year. And/ or apply other processes based on these conditions. (Though, I’m not entirely sure what I’ll do in the end.)
Maybe there is a user friendly resource out there, but I can’t say I’ve found it. (Of course I have the introductory manual. It’s not useless, but using that is a bit like trying to learn Italian by reading a dictionary!
A good book book with some title like “Learning R:” would be nice. I guess I should check Amazon.com! 🙂
lucia, R syntax closely follows S/S+. Many books on S+.
Also R-search site: http://finzi.psych.upenn.edu/search.html
R Graphical user interfaces and editors:
http://en.wikipedia.org/wiki/R_%28programming_language%29
see Productivity tools.
@all– Cool. Now I’m glad I whined.
If you’re not used to open source, it can be a little challenging to find support. When you get software free, it’s hard to demand your money back.
@Larry in 55– Oh, I agree that I shouldn’t expect it to be easy particularly as it’s free. I’m just saying somethings aren’t — at least not for me. 🙂
So, I’m glad I whined because now I can look up the book bender suggested and check out the productivity tools. (Hopefully, some can be used on a Mac. My new one arrives tomorrow.)
@lucia
I am also learning R and have found Using R for Introductory Statistics by John Verzani and The R Book by Michael J. Crawley to be most useful. One, I have to relearn statistics while learning R. If you have the statistics down, then the The R Book will be big help. I have found Time Series Analysis and it Applications with R Examples by Robert Shumway and Davis Stoffer helpful also, as they include climate and solar time series examples. Hope this helps.
Lucia,
R is not obsucure nor requires object oriented programming (what’s that 😉 ) knowledge ! R is matrix based just like Matlab. R is supported by many many manuals, books and dedicated forums.
You cant’ have simpler and more powerfull.