Update:
As noted in the comments below, GISS updated the GLB.Ts+dSST anomalies which show a large 0.67 degC value for March. This addition of March 2008 temperature data to the record caused a corresponding drop in annual average temperature for the years 1946 and 1903. According to GISS, 1946 is now colder than 1960 and 1972, and 1903 dropped into a tie with 1885, 1910 and 1912.
That’s really neat.
End update.
In February I wrote a post asking How much Estimation is too much Estimation? I pointed out that a large number of station records contained estimates for the annual average. Furthermore, the number of stations used to calculate the annual average had been dropping precipitously for the past 20 years. One was left to wonder just how accurate the reported global average really was and how meaningful rankings of the warmest years had become.
One question that popped into my mind back then was whether or not – with all of the estimation going on – the historical record was static. One could reasonably expect that the record is static. After all, once an estimate for a given year is calculated there is no reason to change it, correct? That would be true if your estimate did not rely on new data added to the record, in particular temperatures collected at a future date. But in the case of GISStemp, this is exactly what is done.
Last September I noted that an estimate of a seasonal or quarterly temperature when one month is missing from the record depends heavily on averages for all three months in that quarter. This can be expressed by the following equation, where 
In the above, T is temperature, q is the given quarter, n is the given year, and N is all years of the record.
One can readily see that as new temperatures are added to the record, the average monthly temperatures will change. Because those average monthly temperatures change, the estimated quarterly temperatures will change, as will the estimated annual averages.
Interestingly, application of the “bias method” used to combine a station’s scribal records can have a ripple effect all the way back to the beginning of a station’s history. This is because the first annual average in every scribal record is estimated, and the bias method relies on the overlap between all years of record, estimated or not. Recall that annual averages are calculated from December of the prior year through November of the current year. However, all scribal records begin in January (well, I have not found one that does not begin in January), so that first winter average is estimated due to the missing December value. Thus, with the bias method, at least one of the two records contains estimated annual values.
Of course, it is fair to ask whether or not this ultimately has any effect on the global annual averages reported by GISS. One does not have to look very hard to find out that the answer is “yes”.
On March 29 I downloaded the GLB.Ts.txt file from GISS and compared it to a copy I had from late August 2007. I was surprised to find several hundred differences in monthly temperature. Intrigued, I decided to take a trip back in time via the “Way Back Machine”.
Here I found 32 versions of GLB.Ts.txt going back to September 24, 2005. I was a bit disappointed the record did not go back further, but was later surprised at how many historical changes can occur in a brief 2 1/2 years.The first thing I did was eliminate versions where no changes to the data were made. I then compared the number of monthly differences between the remaining sequential records and built the following table. Here I show the “Prior” record compared to the next sequential record (referred to as “Current”). The number of changes made to the monthly record between Prior and Current is shown in the “Updates” column (this column does not count additions to the record – only changes to existing data are counted). The number of valid months contained in the Prior record is in the “Months” column. “Change” is simply the percent Updates made to Months.
On average 20% of the historical record was modified 16 times in the last 2 1/2 years. The largest single jump was 0.27 C. This occurred between the Oct 13, 2006 and Jan 15, 2007 records when Aug 2006 changed from an anomoly of +0.43C to +0.70C, a change of nearly 68%.
Wow.
The next question I had was “how often are the months within specific years modified?” As can be seen in the next chart, a surprising number of the earliest monthly averages are modified time and again.
I was surprised at how much of the pre-Y2K temperature record changed! My personal favorite change was between the August 16, 2007 file and the March 29, 2008 file. Suddenly, in the later file, the J-D annual temperature for 1880 could now be calculated. In all previous versions the temperature could not be determined.
But some will want to know only how this process affects the rankings for the top 10 warmest years. Because the history goes back to the middle of 2005, I explored this question only for the years before 2005. While the overall ranking from top to bottom does change from one record to the other, the top 10 prior to 2005 does not change much. However, the top two do exchange position frequently, as can be seen from the following table:
I will note that the overall trend in changes between now and Sep. 24, 2005 is very close to zero. If one compares the latest file with the one from Sep 24, 2005, it can be seen that the earliest and latest years are adjusted lower today than in 2005, while the middle years are adjusted higher. However, this is purely coincidence. If one compares the file from Aug. 2007 with the latest file, it appears the earliest temperatures have been adjusted downward, leading to an overall upward trend. Surely other comparisons will yield a downward tend. It is by pure chance that we have selected two endpoint datasets that appear to have no effect on the tend.
It is at this point I would like to ask, does anyone have a copy of the GISS monthly and annual temperatures – the equivalent to GLB.Ts.txt – from a date earlier than Sep. 24, 2005?
In the meantime, will the real historical record please stand up?
Climate Audit 
130 Comments
And here all along, I thought that all those temperature numbers shown to us (the general public) were actual data.
I’m also kind of surprised that none of those scientists working with these numbers and this formula didn’t spot this.
Steve,
Do you still believe that this issue is so important that we should never throw our hands up and declare the data corrupted beyond recovery?
Resisting the temptation to quote 1984, its still amazing to me how quickly and often recent climate history changes right before our eyes.
John, I’ve spent some time collecting GISS versions and have posted up a number of vintage versions here
http://data.climateaudit.org/data/giss/hansen.collation.dat . The collation includes the hardcopy data from HAnsen and Lebedfeff 1987 – yes, they used to publish actual numbers in journals in the old days.
You seem to be on a mission to totally deconstruct Hansen.
Good on you.
#4 Steve, thanks…looks like at least four versions are useful…I will need to explore the provenance of the “Soon_smoothed” data more.
A reasonable way to interpolate a missing month that can be computed the following year and requires no further revision is to interpolate between the preceding and previous month, but then to capture curvature by adding to this the average difference between the same month and its similar interpolation in the preceding and following years.
If T(t,m) represents temperature in year t for month m, this reduces to
T(t,m) = (T(t, m-1) + T(t,m+1) + T(t+1,m) + T(t-1,m))/2 – (T(t+1,m-1) + T(t+1,m+1) + T(t-1,m-1) + T(t-1, m+1)/4
If m = 1 (Jan), T(t,0) etc is interpreted as T(t-1,12). If m = 12 (Dec), T(t, 13) etc is interpreted as T(t+1, 1).
If one of the required adjacent or caddy-corner months is also missing, more thought is required…
They changed their file structure in 2005. You can find older files under this structure below.
http://web.archive.org/web/*/http://www.giss.nasa.gov/data/
I think the www helps for domains that old. Archive.org keeps timing out on me, but I got one.
I downloaded a version from a page marked last update January 22nd, 2001. Here it is.
http://asshelmets.com/jeez/GLB.Ts.txt
FYI, I got to the file above via the single 1999 link, even though the copyright on the page identified 1/22/01
Here’s the path from the Wayback machine.
http://web.archive.org/web/20010126234100/www.giss.nasa.gov/data/update/gistemp/GLB.Ts.txt
So again we see that, while the temperature of the planet may not be rising in the present, it most certainly is in the past?
jeez, thanks for finding the old file structure and link to the appropriate archive on the wayback machine.
Re #8; Love that CYA URL.
The massage is the message.
Oh NOW I understand why they call it “climate change”! All this time I thought it was what they were studying, not what they were doing.
I’ve been doing hypothesis testing on the data over the past two months. I used data in Feb. I used data in March. Had the monthly GISSTemp values changed? Yep. Quite a few recent monthly temperatures had changed.
I thought it was me at the time. . .
Clearly, analyses using this data must now not only cite the source, but also state the specific date when you downloaded the data!
SteveMc, how about a spagetti graph of Giss. just for kicks
Presumably, if they’re re-calculating the estimated temperatures every month (quarter?) as the new readings are added, it should be a relatively simple – if time consuming – task to “roll back” the file, one month/quarter at a time, to reveal each step along the way.
e.g. take the current data, knock off anything after the last month recorded in the Aug ’07 file, run the calculations on the estimated temps & you should, in theory, have the August ’07 file, reconstructed from the March ’08 file.
It would be interesting to see if that is the case: In which case the methodology is consistent, if flawed, or if there are differences – and if there are differences, which way the past goes – cooler or hotter…
What percentage of the changes are due to their tinkering with the UHI correction vs recalculating the estimated temperatures?
Let’s suppose ( or should I say hope) that the expression for Tq,n has been shown by GISS to be a good estimator. Tq,n can be re-written as
Tq,n= (Tb+Tc)/2 +(Tbar,a-Tbar,b)/6 + (Tbar,a-Tbar,c)/6. So these differences in long term monthly averages have supposedly been shown by GISS to be part of a good estimator. It’s the changes in these differences in long term monthly averages over time that result in changes to the history at each update. I believe it’s reasonable to expect these differences to become stable as sufficient monthly history is accumulated even when temperatures are trending. That is, as long as the monthly temperatures have the same trending properties, as they well could have for nearby months in a quarter. If GISS does not update the earlier records as the history lengthens, then it may be unnecessarily increasing the error in the historical records since the long term monthly differences should improve as the history lengthens.
Anyway, I’m not sure it’s a bad thing that the history changes. It would have to be demonstrated.
And by the way, I’m just being a devil’s advocate.
When I first looked at the equation, I was a bit puzzled as to how/why anyone would develop such an odd algorithm. But it does become quickly obvious that they are simply filling in the missing data based on the historical average of the missing month’s relationship to the other months. Averaged, that is, over the entire climate record. On the surface, that seems reasonable. However, it also seems likely to me that as climate changes (for whatever reason), so may the temperature relationship of one month to the others in that season. That is, it wouldn’t surprise me at all if March tended get warmer (or cooler) with respect to January and February, or some other seasonal shift that would not be captured.
Why is it so important to fill in the missing data for a site, anyway? If I am averaging over many sites, I can still compute monthly and yearly anomalies based on the aggregate of available sites. Why is it so important to fill in the missing data which ultimately is corrupting the data, even if the corruption is slight. It certainly is not reducing measurement variance. I don’t get it.
As an engineer, I find it absolutely incomprehensible that any data, once recorded, (and massaged), should be altered. Talk about 1984. It’s a bit like altering the height of the steps behind you as you climb some stairs. Surely this goes beyond averaging to absolute fraud. How can anyone even countenance the idea of permitting historical data to be altered?
Why would anyone do this, asked my wife? Buggered if I know, said I. (But I’ve got a good idea!)
I guess the question is this: “Is the raw data archived and available (to someone like Steve Mc)”.
If it is, this continued massaging of the data is less pernicious, since one can always go back and start again with a new algorithm, if necessary. However, if it is not, I’d have to say it would indeed border on the [snip]
I suspect it is archived, but don’t know whether it is available.
Pat Keating says:
GISS is processed data – I assume that all of the raw data used as inputs to GISS is available and not altered over time (Steve can you confirm this?)
re 21. Its important for hansen to infill data because some of his alogorithms for constructing adjustmests depend on having LONG RECORDS. the longer the better. So stations are merged and patched onto each other and holes are filled.
Nothing nefarious here, but it is a proceedure that has drawbacks.
Re:#14
Bill P.
As one who read and taught McLuhan for several years your word play was not just very funny, it was very, very insightful. Thanks.
R
“So again we see that, while the temperature of the planet may not be rising in the present, it most certainly is in the past?”
#11,
No, no, they cool the distant past to keep a rising trend even with no present warming.
The CRU is notorious for doing that : just compare global temperature graphs they provided to the IPCC in successive reports.
One of the reasons for change is that anytime stations get lost (are not reported/lag in being entered) for a current period, you can’t use that station for any past times. As station records catch up (enter the data base again) the are added back into the calculation. Thus it is not the same set of stations every time the calculations are done. Steve M has documented that many stations with data that were in the calc in the past have been dropped or are not yet updated. However, this should create RANDOM fluctuations in past mean values.
#28 Craig Loehle
I don’t believe that is quite true. If a station’s record exists for 1980 but not 2000, I believe it is used to calculate the 1980 global average, but not the 2000 global average. If a record later appears for that station covering the year 2000, that station will now be included in the calculation of the 2000 average. The record for 2000 will also have a ripple effect back in time if any of the values in the station’s history were estimated.
However, I don’t think we often see the sudden “discovery” of older records (>2 years). What we see is a delay in current records finding their way into the calculation. Thus, a number of the updates from about 2003 to present in the graph I show are probably attributable to this delay in reporting. But because I looked at global averages calculated in 2005 and later, the changes to the record prior to ~2003 should not be due to the introduction of new records.
As I have noted before in this blog, there is a very large amount of estimation that occurs in the splicing of scribal records and calculating yearly temperature averages for each station. This estimation is sensitive to the addition of current temperatures to the record. Thus, it is not surprising to see it affect the updates to the global history in the manner I have noted.
I can only imagine the amount of chaos that would ensue should the worldwide stations “lost” during the conversion to MCDW suddenly be found. It would not only affect the global averages from 1990 onward, but I would also expect it to have a large effect on the averages going all the way back to 1880. The reason is most of the estimation seems to have occurred in these records before they went missing.
Re #7, an even better method of filling missing months permanently that does not require waiting for next year’s data (or the completeness of next year’s data) is to interpolate between the preceding and following months, and then add in an adjustment for seasonal curvature based only on last year’s deviation of this month’s temperature from the similar average of adjacent months:
T(t,m) = (T(t,m-1) + T(t,m+1))/2 + T(t-1,m) – (T(t-1,m-1) + T(t-1,m+1))/2
Then if one of these 3 months is missing next year, it can just be filled in with the now “complete” data for this year!
Often a deficient month is not completely missing, but say just 60% complete in terms of daily readings. In such a case one could say use 60% of the recorded average and 40% of the interpolation from adjacent months and last year.
In any event, a metarecord should be kept of how complete the data really is, so that we don’t get carried away with entirely made up data!
My first reaction on reading this was to laugh. Then I got to thinking, how much made up data should be allowed? That is, how much data that is not actual readings should be allowed to be part of the official record? How much variance from the actual readings, in the form of corrections, is too much variance?
There are no official standards in place for determining such things. If there are no standards, then anything done to raw data is simply a guess.
I propose that a body of concerned scientists should form an official organization, with the sole purpose of deciding what should be standard procedure in dealing with temperature records and proxies.
Yeah, too bad the UN couldn’t start something like that, to look into the science of weather and climate related matters.
Obviously no method is perfect. Initially I thought a method like Hu’s would suffice, but it assumes climate behaves normally. We know it does not.
Example: Where I live, November of 2006 was normal, December 2006 was way above average, and January 2007 was slightly below normal. If I were missing the December 2006 data point, just about any scheme I come up with will grossly under-estimate that temperature.
Step forward one year later. November was slightly above normal, December was slightly below normal, and January was even colder than the previous year. If I were missing December of 2007 but had both adjacent months as well as all three corresponding months from the previous year, I would over-estimate the December 2007 temperature.
Note however that the method GISS uses does not use adjacent months – it uses seasonal months. The autumn season is Sep-Oct-Nov. If N is missing it is derived from Sep-Oct. You and I might think it more reasonable to derive it from Oct-Dec. Likewise for missing Sep. Only Oct would be derived from adjacent months. In fact, only four of the missing seasons would be derived from missing months. If an entire season is missing, the relationship gets really strange.
Estimation of values during the intermediate steps of determining an average global temperature should be avoided. This is particularly true of the earliest steps, because the uncertainty of that estimation only grows as more and more results are derived from it.
In this case, estimation begins very early in the process. One of the first steps is to take each scribal record and estimate seasonal averages when months are missing. Then missing annual averages are estimated if any seasonal averages are estimated or missing. These scribal records are then combined with other scribal records to form a station record, and the consequence of the estimation ripples throughout the resulting record. GISS then applies a homogeneity adjustment to this data containing significant numbers of estimates. After that GISS goes through gridcell calculations and then the final calculation of the annual global temperature.
The only purpose of the estimation seems to be to calculate an annual average for each scribal record prior to combining, adjusting, griding and averaging. I propose that annual averages are not important as intermediate steps. Instead of calculating an annual global average temperature, calculate a monthly average temperature. If a data point is missing, then it is not adjusted, gridded or averaged. It simply is not included. There should be enough other data points from around the world to calculate an average temperature with some degree of certainty (of course, it is desirable that this degree of certainty be published).
Seriously, what happens if you wrote a paper in 2002 based on that years version of GISS
and your results were Sig at 95%, and 3 years later somebody checks your work and downloads
the 2005 version and their math dont check
What this means is that you CANNOT just link to the data source. You have to have a copy
of what you worked with.
I used GISS! which vintage?
This makes the shenanigans of mining entrepreneurs pale completely into insignificance.
I wonder if the variation in the adjusted numbers is greater than the warming derived from those numbers……
GISS comes in with a whopping 0.67 degC anomoly in March:
http://data.giss.nasa.gov/gistemp/tabledata/GLB.Ts+dSST.txt
Up 0.41 from Feb.
March in RSS was up only 0.08 from Feb.
To diverge, or not diverge: that is the question:
Whether ’tis nobler in the mind to suffer
The slings and arrows of outrageous audit,
Or to take arms against a sea surface temp of lies,
And by opposing end them? To adjust: to twist;
No more; and by extrapolation to say we end
The heart-ache and the thousand natural shocks
That GISS is heir to, ’tis a consummation
Devoutly to be wish’d away. To converge, to archive;
To archive: perchance to dream: ay, there’s the rub;
For in that archive of data what dreams may come
When we have shuffled off this nonreplicable coil,
Must give us pause: there’s the respect
That makes calamity of so long life;
For who would bear the whips and scorns of skeptical bloggers,
The oppressor’s wrong, the proud man’s contumely,
The pangs of lost grants, the lawsuits delay,
The insolence of office and the spurns
That patient merit of the unworthy takes,
When he himself might his quietus make
With a bare bodkin? who would fardels bear,
To grunt and sweat under failed predictions,
But that the dread of something after warming,
The undiscover’d country from whose bourn
No traveller returns, puzzles the will
And makes us rather bear those audits we have
Than fly to others that we know not of?
Thus conscience does make cowards of us all;
And thus the native hue of decadal resolution
Is sicklied o’er with the pale cast of thought,
And enterprises of great pith and moment
With this regard their haline currents turn awry,
And lose the name of feedback action. – Soft you now!
The fair Mann! Nymphette, in thy orisons
Be all my sins remember’d.
#36 Raven
Between the February and March versions, the values of 30 months were decreased by 1 and the values of 8 months were increased by 1. Not huge changes, to be sure. But, note that the addition of the March 2008 temperature to the record was enough to cause the annual temperature of 1946 to now be colder than 1960 and 1972, and to drop 1903 into a tie with 1885, 1910 and 1912.
People find what they look for and the minute a scientist believes his own hypothesis, he’s a dead duck as a scientist.
My hypothesis is climate change is not man-made.
The hypothesis runs:
It’s not solar irradiance alone.
It’s not sunspots alone.
It’s not CO2 above 18 C.
it’s not water vapour alone.
It’s not cosmic radiation alone.’
But it may be cosmic and solar radiation modulated by solar magnetic activity subtly changing the cloud albedo of Earth.
Beware the unintended consequences of sequestering plant food during the famine.
” The newspaper has discovered that most people most of the time are
interested in some form of catastrophe: a plane crash, a railroad wreck, a
murder, a flood, a scandal, a fight of some sort.”
Harry Overstreet, 1949 – “The Mature Mind”
It is our job to continue to point out the Kyoto train wreck
Sam, I was thinking of something along the lines of IEEE. That is, an organization that is not funded by political money or motives. I would not consider anything that the UN is involved with to be exempt from political money or motives. If this is not done, then climatology will sink back into the morass of pseudo-science, along with astrology and dowsing. It has the potential to standardize many practices, to start accumulating good data which can actually be of some use in the future, and to stop making wild predictions. Right now is probably the turning point for this particular philosophy. We might as well be discussing how many angels can stand on the head of a pin, for all the good that the rhetoric and computer programs/models are doing.
Bottom line: Science is standardized procedures. No standards, no science.
The report on my imminent death is premature. I have been sloshing around in the basins on the crust for more than four billion years. I now cover 80 per cent of the planet. Since the last ice age I have lifted myself out of the basin by 120 metres and scared the tribes of Noah to the higher ground. During deep time I became the universal solvent for the volcanoes and the clouds. I have taken up as much salt as required by local circumstances and sometimes give it back in hot shallows and desert areas of my world. I have given man the salt in his blood. Your CO2 output is infinitesimally small. I have absorbed as much gas as I need to maintain balance with the organic world within me and on land. The exchange is so peaceful that science calls it equilibrium. I can absorb more CO2, if the plants do not need it, and it does not give me acid imbalance. My pH will remain basic no matter what you say. These variations you measure have come and gone many uncountable times on the planet and your baseline is too small to know the truth. What you do not get is that warming of the oceans releases CO2 and other gasses from my water, while cooling my water allows me to take up CO2 in vast amounts to nestle with the other molecules in my coldest most remote realms. I can absorb all that man can produce because your impact is feeble compared to my capacity.
Please watch me with humility for you cannot change me. I am the ongoing sink for the planet, and I am huge. Measure me here and there with your microscopes but know that I will never be that way in that place again. Open your mind to the infinite cycles of chemistry and physics and kneel on my beach. You can only hurt me by not respecting my infinite ability to change chemistry and temperature in all the corners of the seas. My CO2 feeds your plants and your plants provide all the oxygen you breathe. Your base line is infinitesimally small yet your mouth is wide open.
Sorry to clog your blog Steven – – I’m just getting things out of my system while I meditate about my sciatica at home with the kids. Yikes!
re 33. do the GISS spagetti graph
Raven:
I call BS on +.67
However, that still gives us a DJF of +.26 versus last year’s +.72
DJF 1880-2008 average -.03 Standard deviation 28.4 and median of -.08 Mean of min/max +.045 Mean of start/end 0
Selected interesting DJF
2008 +.26
2007 +.72
1994 +.08
1988 +.43
1985 -.04
1973 +.24
1972 -.19
1958 +.27
1951 -.27
1944 +.30
1937 -.19
1936 +.13
1926 +.16
1917 -.54
1893 -.63
1882 -.09
1880 -.26
“There has been a slight anomaly drop for the winter season as a worldwide average since 1944, when it was +.30 C over the base period, to +.26 C in 2008”
😀
well SAMU if you look at the difference between RSS and GISS.. I would throw a BS flag as well.
See where Hatcrew come in.
re 44. on the hand, maybe the pole is getting warm.
no jokes jeez.
damn
#43 What do you mean by “the GISS spagetti graph”?
Francis, that was eloquent. Pain and children have a way of forcing you to focus on anything else, to avoid direct contact with them, don’t they? Alternating hot and cold packs helps a lot for sciatica attacks, along with mild movement. If the doctor gave you exercises, you should be doing those, rather than posting here. (just kidding)
It can also be humbling to realize that dust storms carry further than pollution. Amazing to think that Galileo pulled the earth from being the center of the universe, only to have humanity put back there by the likes of Al Gore. We are not nearly as important, or as powerful, as we would like to think. That is a comforting thought.
Posted this over on Atmoz Blog, some interesting data to be sure:
Major divergence with the UAH MSU stats for Mar (http://vortex.nsstc.uah.edu/public/msu/t2lt/tltglhmam_5.2)
For Mar:
Global 0.094
NH: 0.433
SH: -0.245
Tropics: -0.489
1) The RSS data tracks pretty closely with the MSU data for Mar
MSU Global: 0.094
RSS Global: 0.079
2) RSS NH data has an interesting dichotomy in that the Cont US numbers are totally opposite the rest of the NH:
Cont USA: -0.877
Lat 20-82.5: 0.877
Interesting to say the least.
I guess the snow I saw on Easter Sunday in London never happened. I know, it’s one day…still.
Re #43
Don’t know if this is what he means but I’d say if you’ve got all of these data sets, put them on the same graph to see how bad the spread is (assuming they’re normalized using the same range.
Interesting stuff, thanks for this John.
You seem very interested in the number of historical changes; surely more relevant would be the average and distribution of the effects these changes had on the monthly and annual means, on the magnitude you tell us only the largest single value. Did you calculate the average change and could you share it with us in true climate auditor fashion?
Its interesting that 1998 and 2002 change places so often, but worth noting that the difference between these two years is only 0.01C, about a tenth of the measurement uncertainty in the anomalies.
regards
CChrome
Steve: (For you, not for posting) My Hadley Centre source says they regard the NOAA data set, and not the NASA/GISS data set, as the authentic US data set. Does this make any difference?
Why aren’t satellite data constantly changed throughout the record?
55 (Jimc) Well, I think GISS is concerned with reformatting their data to conform with their latest understanding of the interpretation of temperature, and the satellite researchers are too busy making sure their unique view of the subject is a reliable proxy.
===========================================================
Jimc, the satellite data does not go through the same averaging, estimation, and adjustment process as the station data.
A question about using the entire period of record for estimation. Let’s assume “for the sake of argument” that climate changes over the decades. I’m sure Hansen would agree with that assumption (giggle). This means that data from 2008 is probably from a different population than the data from 1903. In that case, wouldn’t it be invalid to use 2008 data to adjust/estimate 1903 data? I’d be interested to see Hansen’s defense of this.
GISS has heavily “adjusted” upward its temperatures after dec 2002. The 1980-2002 trend has mysteriously jumped 30%.
This process strikes me as being one of the craziest methods of reporting records that I have ever had the misfortune to see. It becomes totally meaningless to look at records which themselves are continually changing just beause the reported values represnt the difference between the annual average value for a given year and some contiuously changing average of all years yet recorded. The use of this “anomaly” definition seems to be typical of climate reporting and the above description highlights just how rediculous this is.
Instead of looking for anomalies relative to an average, surely the only sensible and consistent method is to take some FIXED baseline such as the average temperature between 1890 and 1910 which then DOES NOT change. It is then easy to scan across a consistent graph showing the differences between annual temperature and this fixed base.
No wonder the people who model climate change need 23 different models to obtain an answer, while knowing that since they all give different results, only one of the 23 could possibly be correct! This fact also demonstrates that the values found from the models cannot be relied on, since if it is clearly understood that 22 of the 23 models are demonstrably incorrect, what confidence can there be that any one of them can be correct? Since it just as obvious that the average taken from 23 models, only one of which may be correct with 22 incorrect, cannot possibly be correct. This simple logic just does not seem to get across to the people supporting the results found from the GCMs.
Added to this it is equally well demonstrated fact that the calculated climate – temperatures – from the last 100 years are not correctly found by any of the models nor by their average. That is a second reason to disown the results from the modelling since one could not then possibly claim that they will be correct in fifty years time. Even if the tweaking which the IPCC is so fond of, to try to get the models to fit known data, and the models then DID fit that known data Exactly, it would NOT be proof that the models could correctly predict the temperature 50 years hence. However, it would certainly provide more confidence in the models that could be currently afforded to them. One is left with the undeniable conclusion that anthropogenic climate change is a figment of the imagination of 2,500 “scientists”.
Tammy has a post up on the CET. He makes this observation:
Too bad he only notices these things when he thinks they support his pet theories.
Re #61, Raven, you must be referring to the sea ice extent data and graphs that had 30-40% of the data “filled in”. Not sure why Tamino just didn’t apply a wavelet filter to the CET data as he did to the sea ice data.
Phil B.
Phil B says:
I was actually thinking of Tammy’s favorite dataset: GISS
Raven where is CET?
never mind, found it
64 (steven,Raven): I also looked at CET over chez Tammy and took him to task for saying that the recent increase is unprecedented: “If we calculate the 30-year trend for every year and slide it along from 1659 to the present, then tabulate the trends and ask if the recent trend is highly unusual, the answer would be that it is not, as similar trends have occurred in the past.” Here is my graph of that:
Needless to say his ‘response’ was that my ‘statement is foolish’.
I think everyone including him is aware that the earlier readings are less accurate.
I just don’t understand why the obvious conclusion to that isn’t easy to reach and understand.
The readings are unreliable, the anomaly reflects the way the number is gathered, and the jump in the monthly anomalies starting in the late ’70s is simply an artifact of that. Coupled with some kind of (seemingly unknown unquantifiable/unqualifiable) impact from the way the base period is derived.
So, we should stop calling it the temperature.
Note that this does not prove nor disprove that energy levels are or are not rising, it simply proves the readings are not reliable one way or the other as a proxy for them.
Dr. S. I’m kinda dumbfounded by the tone of his replies to you.
Re #66
And he’s right, I suggest you read this.
DR. S BTW, that a great graph. I find it funny that when people look at 10 years trends
then certain parties will look at the exact kind of chart you depeicted and conclude that
10 year trends are all over the map. But when you do it for a 30 year trend and conclude
similarly, they call it foolish. Shrugs. go figure. Its best if they stick with 100 year trend
that way, there’s not much to compare it to, and a long ways off to test it.
Nostradamus tactic. make long bets, collect the cash now, die before the payoff date.
re: #69
Interesting discussions of micro- and macro-site issues in that paper from over three decades ago.
Also Leif you’re misrepresenting the exchange somewhat, what was said was:
Here are the plots in question.
re 67. recording to 1 degree F of accuracy in the USA was fine with tamino, when I pointed
it out to him. Law of large numbers and all
72 (Phil.) without trying to misrepresent Tammy, I was summing up what I think he will agree with is characterizing my several posts on the subject of CET. I had this to say about the specific red curve on the plot you just showed [and THAT was was he called ‘foolish’]:
Instead of us exchanging snippets, let the folks just go there and look for themselves.
68 (steven): me thinks I’m too close for comfort…
Phil.
No matter what you said If you visited my blog, my house, I wouldn’t call you
foolish. If I thought you were foolish, I’d ignore you. When Tammy makes such intemperate
remarks to visiting scienctists, he creates an enviroment. An enviroment that invites
diatribe. go ahead and Contrast the Dr S. threads here
with Tammy town.
None of this goes to the truth of matter at issue. Temperature.
BUT, whenever I visit Tammy town my sense is “gloves off.” especially If the host calls a house guest foolish.
Every day I read Dr. S threads I keep my mouth shut, and mind my manners. Why is that? Is that because the host is patient and sets a good example?
I dunno. Shrugs.
I know that I really shouldn’t, but I can’t help myself. Here is Tammy’s latest:
Re #78
Well he’s allowed one of his acolytes (bender) to call me a liar when I criticized Steve’s post on Hansen 88, and didn’t acknowledge that he had rapidly changed his post in response to my comment for quite a while. When others in the thread criticized bender for his attacks Steve actually encouraged him.
I don’t think that #79 above by Steve is the post of the ‘saintly’ host that some of you portray, in contrast William Connolley has been attacked because a poster thought he might have been the cad who described Steve as a mining engineer on Wikipedia!
Steve: If William Connolley described me as a “mining engineer”, then that’s an incorrect claim. I wouldn’t be insulted to be called a “mining engineer”, but I lack those particular qualifications. Connolley has been criticized for his Wikipedia efforts for intentionally trying to demean me by saying that “M&M” didn’t know the difference between radians versus degrees in a context where “M&M” would be perceived as including me, even though he knew that he had nothing to do with that programming error. Not that I claim infallibility. But that particular error had nothing to do with me and Connolley knew it.
AS to your observation about name-calling, I am not online 24-7 and this is not a pre-moderated blog. One of the few blog rules is not call other posters names. I’m not online 24-7 and often deal with breaches of blog rules after the fact. I don;t recall the particular incident – there have been over 100,000 posts on this blog, but to say that I encouraged a poster in calling another poster a “liar” is inconsistent with how I try to handle things.
89 (Phil.): Not good enough. You said that some of my comments were ill-informed and directed me to that [otherwise nice] paper. Clearly you must know exactly which comment was ill-informed and exactly which point in the paper it clashed with, so please tell me and stop the guessing game. If you choose not to, then I cannot give a reasonable response.
90 (Phil.)
This is what Tammy said:
“As for the first question, rather than make yourself look like a fool by saying “yes,” or retracting your foolish claim by saying “no,” you try to change the question!”
maybe you will now argue that ‘look like a fool’ is not like actually being one?
96 (jeez): We still have a long way to go here before we can live up to Tammy’s latest characterization of me 🙂 :
“Your understanding is feeble, your implications are unprofessional, your attitude is supremely arrogant, and your attempt to avoid responsibility for what you yourself have posted here is despicable.”
But, you are probably correct that most posts back to, say, #77 should be deleted as they do not bring anything of lasting value to the table.
Let’s stop this discussion of blog manners. I try to be polite to posters here and ask posters to be polite to one another – one of the few blog rules. Since the blog is not moderated in advance, I make an effort to deal with food fights after the fact and ask people to behave themselves but some slip through. But enough of this. I agree with Leif’s suggestion in respect to this line of discussion and have followed it.
Re #79 (post deletions)
No response required, except that you should read the paper.
83 (Phil.): I first became aware of Manley’s work in 1982 when I read H. H. Lamb’s wonderful book “Climate, history and the modern world”. And I’m fully aware of the difficulty with the early thermometers. Did you know that Isaac Newton also had defined hos own scale, setting the freezing point to water at 0 and the temperature of the human body at 12 degrees? Now, Manley concludes his paper with this remark: “we can say that back to 1700 there is a very comforting accord with other western European series”. In view of that, I would like that you identify exactly which of my comments were ‘ill-informed’.
84 (Leif): As you say, that book by Lamb is indeed wonderful. Jones and Briffa stated that the 1730s was the warmest decade in each of the CET, De Bilt and Uppsala temperature records until the 1990s occurred.
(Jones, P.D. and Briffa, K.R., 2006 “Unusual climate in northwest Europe during the period 1730 to 1745 based on instrumental and documentary data.” Climatic Change 79, 361-379)
I got into this blog late, but surface temperature data which I agree with the main article seems to change. The IPCC for example lowered the temp data for the 1950’s which makes more of a warming trend later on. Satellites are the most accurate in collecting temperature data that we know of, but space is a rough and hostile environment which creates some flaws. The sensors generally don’t last that long and always are in need of replacements. Also, satellites tend not to stay on the same orbit, but often times drift. As a result, the data gets tainted. And so more revisions are needed…lol…
“Rewriting History, Time and Time Again” a great piece of work! Well done!
I live close to Dartmoor in Southern England and have plotted all the CET figures every way you care to mention. There are several interesting things happening.
1) The first is that winters have overall become much milder, which has pushed up the overall yearly mean average, however some months are otherwise cooler than they used to be and others have moved only marginally upwards.
2) Many of the individual records for warmth are held by months in years well before modern times
3) Bearing in mind that these CET figures back to 1659 are ALL within the ‘little ice age’ yet have only moved marginally upwards-if at all- one can only conclude that we are still in a relatively cold period compared to ‘climate incidents’ in the past.
4) The results of these climate incidents can readily be seen close to my home on Dartmoor-an upland area reaching a maximum of 2000 feet high. It is the ideal place to see what has happened in the past with your own eyes. There are 3000 year old hut circle villages abandoned as the weather grew colder, then in the 12th century farmhouses were abandoned as the medieval warm period finished. The accompanying medieval field pattern where crops were grown are still there to see. The climate today is still currently too cold to allow all year habitation, let alone crops to be grown. This is living history-the evidence of what has happened in the past with our climate can be seen in a short walk.
As a researcher/statistician and analyst I am increasingly bemused by the high level of trust that is attached to extremely theoretical computer models that require dubious prioxies and anomalies to make them even begin to work!
Why is ‘real’ evidence so little regarded?
Why are the CET figures not seen in their true context-as a record of the little ice age period which demonstrates that our climate since then has only warmned marginally?
Tony B
As another late entrant to this debate, I’m curious as to what formal training in statistical methods the protagonists here have, and whether they truly understand the techniques they are using or whether they saw them somewhere and thought they looked good.
My interest in the (mis-)application of statistical methods was sparked when, one day, one of my favourite movies suddenly dropped right out of an IMDb top 50 list in which it had formerly been highly placed. After very little investigation, the drop had evidently been caused by a change in a parameter used by the IMDb’s Bayesian estimator (I won’t go into details here, but it was the “m” value – see here for more information). After a little playing with some randomly generated datasets, I discovered that rankings produced using this estimator are actually quite ill-conditioned with respect to the “m” value – in other words, a relatively small change in “m” can produce quite a large change in the rankings. Seeing that the value of “m” is chosen arbitrarily, this seemed to me conclusive proof that this estimator was completely unsuited for the purpose to which it has been put. Of course, when I informed the IMDb staff they denied this despite my sending them the proof in spreadsheet form. Since then, over a period of some years I’ve been watching the almost “viral” spread of the same Bayesian estimator. It crops up all over the place now when people want to produce ranked lists (even in academic work), and often with a description that’s virtually identical to the one used by IMDb. Clearly a lot of people who want to produce such lists are familiar with IMDb, and because IMDb (incorrectly) uses a Bayesian estimator then they do too.
To cut a long story short, (or, in the modern parlance, tl;dr), is the estimator used to interpolate missing data really appropriate for that purpose? The Internet makes it so easy to copy other people’s mistakes, and Google is a poor substitute for a solid grounding in any discipline. I’ll leave the last words to Alexander Pope: “A little Learning is a dang’rous Thing”…
Re my post #88, I note that there is another “Paul S” posting frequently here, so henceforth I shall post as “Paul Sh”
GISS posted the April 2008 anomalies two days ago. Here are the values for 2008:
J: 13, F: 26, M: 60, A: 41
Last month the anomalies for the first three months of the year were:
J: 12, F: 26, M: 67
Meaning that previously hot month of March, 2008 just got a bit cooler when April was added to the mix.
RE 90. ya I noted that. a 10% adjustment. how do you like your cherry pie?
A 11.7% march change if the “new” is better. But hey maybe next month, we will get even “better” new numbers. I like picking the cherries for my pie. 11.7% lol.
Does this mean that March is no longer the hottest month ever?
#93 MarkW:
March 2005 did and still does carry an anomaly of 0.70. There may be a month with a higher anomaly, but I have not gone looking.
Spoke too soon….noticed January 2002 is 0.71.
Alright, just noticed Jan 2007 was even warmer at 0.86 and confirmed that it is the warmest from an anomaly perspective. I am not being terribly observant today.
I also noticed that adding in the April 2008 data made 2007 and 2005 even warmer, by 0.01. Some of that may be due to the fact that GISS changed the location it takes USHCN data, although we have been told in the past that changes in US temperatures are insignificant on a global scale. From the GISS website: USHCN data will be taken from NOAA’s ftp site – the original source for that file – rather than from CDIAC’s web site; this way we get the most recent publicly available version. Whereas CDIAC’s copy currently ends in 12/2005, NOAA’s file extends through 5/2007.
1903 and 1946 also warmed by the same amount.
John Goetz; Yep, Jan 2007, currently at +.86 and lowest -.87 in Jan 1893 So a range of basically 0 +/- .9 or 1.8
Check out the 35 years from 1959 to 1993 over at the BB along with some other interesting tidbits.
How random is the historic temperature record?
may is scortching here in norCAL. PDO my sweaty ass.
Beware the unintended consequences of sequestering plant food during the famine.
Nicely put; I’ve been beating this drum among friends & family, but not nearly as eloquently. The current obsession w/ C02 has gone from silly to slightly frightening. The West isn’t close to famine mode yet, but it could be. In the meantime, less fortunate regions are going hungry while Western farmers plant their soybean fields with ethanol corn.
Try the good old Piriformis stretch for sciatica. Works for me every time (albeit, temporarily): Get on your hands & knees, cross the leg of the offending side over the other leg at the knee, and then slowly stretch that other leg back until the Piriformis starts to protest. Stay there until it relaxes, and then stretch a little more. I sometimes come close to having my chest touch the floor.
This is beyond alarming no? Rewriting history to shape the present and solidify the future? Is that what is going on here or am I just another crazy conspiracy theorist?
Re #102
Steve I’m sure this breaks the rules?
Steve: Yep. Thanks for pointing this out and not engaging,
RE 103. If Hansen compared trains that take coal to coal plants to the trains
that took jews to death camps would you finally and utterly denounce him?
or would you dance around the issue like Neville Tinkerbell?
Realize that this site has been posting comments relative to the post by John Goetz on history being re-written at GISS, but I have a related question concerning Hadley.
I have been following the monthly temperature anomaly values published by UAH, Hadley, etc.
Recently I noticed that the Hadley values I had downloaded as they were first published for the first four months of 2008 had subsequently been changed
I have noticed occasional minor adjustments after the fact in most of the records, but this adjustment covered four successive months and was not “minor.
Original record
J -0.105
F +0.039
M +0.430
A +0.250
“Corrected” record
J +0.054
F +0.192
M +0.445
A +0.254
The net difference is an average of +0.083C per month, so fairly significant in a record where annual changes are only a fraction of this amount.
So my question, “Has the Met Office changed its method of calculating the reported monthly values or has it started some ex post facto “corrections” to the monthly record for the first four months of 2008, in order to “mitigate” the current cooling trend?
I sincerely hope that the latter this is not the case.
Up until now, I have always assumed that it is only the GISS record that has been compromised (and is therefore out of line with the others).
Do we now have a similar problem with the Hadley record?
If anyone has any information on what has happened, I would appreciate hearing it.
Max
Re #104
The data doesn’t all arrive at once, as data arrives it’s input into the calculation.
You neglected to mention that the changes include late data PLUS an estimated variance adjustment. In other words, the temperature reported in the dataset is another artifact and not the real observed temperature.
“The data doesn’t all arrive at once, as data arrives it’s input into the calculation.”
“You neglected to mention that the changes include late data PLUS an estimated variance adjustment. In other words, the temperature reported in the dataset is another artifact and not the real observed temperature.”
The “adjustment” was significant, i.e. +0.08C average over 4 months. Seems an awful lot for “late arriving data” (all in the same direction).
This seems suspicious to me, since this was an unusually cold period (as confirmed by both UAH and RSS) and especially since Hadley had previously published predictions early in 2008 that warming would continue in 2008.
http://uk.reuters.com/article/environmentNews/idUKL0314515220080103
Predictions defending “continued warming” and after the fact adjustments that increase apparent warming (or decrease apparent cooling) are a strange and somewhat suspect combination.
Can the Hadley record be trusted or is it as D. Patterson writes “another artifact and not the real observed temperature”?
Can anyone clear this up?
Thanks.
See the Climate Research Unit, Answers to Frequently-asked Questions
Re #107
The estimated variance adjustment is a way of allowing for the variable number of points per grid cell and so wouldn’t effect the mean value. In any case it would have been applied to both sets of data you referred to so shouldn’t be the cause of the change. All the data with the exception of march were appreciably below 2007.
Click to access HadCRUT3_accepted.pdf
#109
Sorry, Phil, but you appear to be wrong on both counts. The link you provided contains a description of the variance adjustment procedure which you might wish to read. On p. 26 in Appendix A, they state:
Now interpreting their somewhat incomplete description, I take “detrended” to mean that the moving average is subtracted from the original value. If so, then the resulting adjusted value is a simple weighted average (with weights k and 1-k) of the original grid value and the moving average. So, yes, the “mean value” is definitely affected. They admit as much in the earlier part of the paper at p.14 ff. Note that this also affects the autocorrelation properties of the time series as well:
With regard to your second statement, they don’t describe how they do the “centered running average” calculation near the endpoints. However, it is reasonably clear that after a value for a grid square is calculated, for the next three years, it will be subject to changes as the latest temperatures become available. The running average is affected that far back by a new observation. From the last quote, my suspicion is that they do not use “variance adjustment” when calculating a global series, but who knows…
Jones and Briffa have a funny variance adjustment procedure to adjust for changing number of proxies, described in a Dendrochronologia article (which I’ve decoded into a working algorithm). Perhaps that’s what they’re using here.
This is quite a veer to the running discussion, but to go back to the main article (and #33): has anybody run the formula given for estimating missing data points — the T(q,n) formula — on non-missing data points? It would seem that you could get an idea of the accuracy of the method by comparing actual data points with the calculated estimates that would result from treating them as missing data points. It would seem that doing so for all the non-missing data points would to some degree quantifiably characterize the method, no?
For example, how would the magnitude of the estimation error (difference between actual and calculated values) compare with the natural variablity of the data? How would it compare to the magnitude of the warming trend?
Hi Steve,
Thanks for getting directly involved.
My question regarding the Hadley upward adjustments to their original record for January-April 2008 was pretty basic and maybe a bit naive.
So far the responses have been a bit ambiguous.
The question was simply “is the record … part of the normal process?” [or is it an “unusual adjustment”]
And “if it is just part of the normal process”, I would have three more questions:
“Is it ‘normal’ for an adjustment of this very high magnitude to be made (over +0.08C average over the 4 months)”?
“In the past, has such a significant adjustment been made to the record after the fact?”
“Have past adjustments been in both directions (warmer/cooler) or predominantly in one direction?”
Sorry to keep “beating this dog to death”, but I hope someone can help clear this up for me.
Thanks,
Max
Hi Steve,
Read your article on the Briffa-Osborn variance adjustment, which you cited (111).
http://www.climateaudit.org/?cat=13
Not being an expert in statistics I cannot say anything to what you refer to as “a funny variance adjustment procedure to adjust for changing number of proxies” or to the algorithm you derived. You suggested that perhaps that’s what they’re using here.
If so, could this variance adjustment specifically change (after the fact) the recorded average temperature in an upward direction during an unusually cold period, thereby introducing an artifact?
Thanks for any thoughts you may have on this.
Max
Message to D. Patterson
Thanks for your input, but I’m still trying to find out what happened to the first four months of 2008 in the Hadley record.
Checked out the Jones et al. 2001 paper cited in the Hadley FAQ section under the sub-title, “Why are values slightly different when I download an updated file a year later?”, which you cited.
http://www.agu.org/pubs/crossref/2001/2000JD900564.shtml
Quoting the Abstract:
“We develop methods for adjusting grid box average temperature time series for the effects on variance of changing numbers of contributing data. Owing to the different sampling characteristics of the data, we use different techniques over land and ocean. The result is to damp average temperature anomalies over a grid box by an amount inversely related to the number of contributing stations or observations. Variance corrections influence all grid box time series but have their greatest effects over data sparse oceanic regions. After adjustment, the grid box land and ocean surface temperature data sets are unaffected by artificial variance changes which might affect, in particular, the results of analyses of the incidence of extreme values. We combine the adjusted land surface air temperature and sea surface temperature data sets and apply a limited spatial interpolation. The effects of our procedures on hemispheric and global temperature anomaly series are small.”
The last sentence tells me that the effect of this variance adjustment on the global anomaly is small.
For Jan-Mar 2008 the adjustment averaged 0.08C over these four months.
With a total anomaly (compared to the 1961-90 baseline period) of only 3 to 4 times the amount of this adjustment, I would not call this adjustment “small”.
So my question (to anyone who has an answer) remains, what happened?
Max
Max Reur 104 and 115,
You give the upward “correction” for January as 0.159C, and for February, as 0.153C. I would have thought that the Hadley boys would have been uttering an array of unprintable expletives upon seeing the original low numbers! I would further think that this would cause them to:
a) Check everything over and over and over to ensure there was no mistake
b) If they were still waiting for late important data, declare the result as preliminary, withhold it altogether, or make a preferred estimate. (Like sometimes in their graphs they show the plot colour as green)
I would also have thought that since the “correction” is so massive, it might merit a comment from them. Steve has discussed another Hadley matter elsewhere, where they “adjusted” their 21-point smoothing method when they did not like a recent outcome. Hadley explained @:
http://hadobs.metoffice.com/hadcrut3/smoothing.html
See, discussion around figure 2
There is a strong interconnection between the two matters I think.
Sorry, I don’t have an answer to your question. It should be noted Hadley says they changed their methods by no longer including an incomplete year, such as 2008. But, I believe this has already been discussed and factored into the question?
On a side note, I noticed Hadley is prominently declaring that there is a continued increase in global temperature. Their current anomoly map for June 2008 displays an increase in temperature for areas of the Midwest where locals have been loudly complaining about late garden crops of tomatoes, green beans, and more due to the unseasonable colder weather.
Max wrote in part in 104:
If this is the first correction made in 2008, it would seem to be for some reason other than “late arrival” of data. Would that not be a progressive thing, running month by month, or maybe randomly bi-monthly-monthly, but not in blocks of four months
Based on the input that is coming from many sides, I am beginning to get the impression that Hadley was [making an unusual adjustment]
I still hope that someone out there can show that this suspicion is unfounded, and that Hadley is providing us unbiased and impartial information on global average temperatures.
Please, any supporters of Hadley (or Steve), come out now to show that this is the case, and that my suspicions are unfounded.
Thanks,
Max
Re#119 This seems to to be flouting your guidelines?
Steve: I’ve snipped some food fighting. I agree with Phil here and appreciate his measured responses. In business accounting, on occasion, corporations do “unusual adjustments”. Just because they are unusual doesn’t mean that they are unjustified, but because they are unusual, they need to be spelled out in baby steps and auditors always examine them in particular detail. Let’s use this term which has less contentious precedents; I’ve substituted this for overly laden terms marked in […].
In the case in point, it would be nice to know if Hadley is making unusual adjustments or not. If so, perhaps there’s a valid reason, which would be interesting to know.
Steve wrote: “In the case in point [Hadley upward adjustments for January through April 2008], it would be nice to know if Hadley is making unusual adjustments or not. If so, perhaps there’s a valid reason, which would be interesting to know.
I guess that was my point, but it looks like the transparency on what happened seems to be lacking.
The two satellite records (UAH, RSS) recorded a temperature anomaly of 0.019C (UAH) and 0.022C (RSS) for the first 4 months of 2008, compared to 0.344C (UAH) and 0.354C (RSS) on average for the prior years 2001 through 2007, for a net difference of 0.326C (UAH) and 0.333C (RSS) between the 2008 reading and the average reading for the prior seven years.
The original (unadjusted) Hadley record showed a difference between 2008 and 2001-2007 average of around ½ of this amount, or 0.165C. GISS showed a similar difference of 0.161C.
After “adjustment”, Hadley shows a difference of only 0.082C, roughly ¼ of the difference recorded by UAH and RSS and ½ the difference recorded by GISS.
So the question remains, what happened?
I believe it is important enough to resolve this discrepancy, in order to restore credibility to the Hadley record, especially since Hadley makes it a point to “predict” continued warming and then to state that the record confirms these predictions, when the other records seem to contradict this claim.
So, if anyone (including Phil, who seems to have faith in the Hadley record) can explain what happened, I would appreciate it.
Max
Max
Steve Mc’ added his comment to 120, saying in part:
However, it has all gone quiet and pregnant since July 23rd, 2008 at 11:52 pm , whereas, I would have thought an unusual adjustment of the magnitude reported would create a lot of interest.
Max shows an upward “adjustment” for January of 0.159C, and for February, as 0.153C, with lesser adjustments for March and April, all in one 4-month block-adjustment.
I would like to investigate this in depth, because I THINK it is VERY unusual, but maybe I am ignorant in these matters, and do not know where to start.
I would SUGGEST that Max is very disillusioned at the lack of interest, and also does not know where to go from here either, other than maybe go to some other website.
On the other hand, is Steve Mc’ saying in code that he is personally interested, and will apply his great skills at sniffing such things out? I do hope so!
Re #118
This can not be as you describe since in February Watts quoted the January Hadcrut anomaly to be 0.034
Watts
In March as being:
2008/01 0.056
2008/02 0.194
And in April as being:
2008/01 0.056
2008/02 0.187
2008/03 0.430
So it was clearly not done in blocks of 4 months but progressively.
Hi Phil,
To the adjustment of the Hadley record you concluded, “So it was clearly not done in blocks of 4 months but progressively.”
I just noticed that the values I downloaded as they first became available had changed after a few months.
I also noticed that this affected the first four months of 2008.
I also noticed that the change was not a “minor adjustment” (as I have witnessed occasionally in the past in all the records from time to time), but a major shift.
I compared the 4 records for the months of January through March 2008 with the average of January through March over the previous 7 years (2001-2007) and saw that:
· UAH and RSS correlated very well, with J-A 2008 around 0.33C lower than the average of J-A 2001-2007 (in other words a significant cooling was observed as compared to earlier years)
· GISS showed only around ½ of this cooling at 0.161C (2008 vs 2001-07 average),but still a significant cooling compared to earlier years
· The original Hadley values also showed cooling at 0.165C (2008 vs 2001-07 average), which seemed to make sense
· The revised Hadley record, however, shows very little cooling at only 0.082C (2008 vs 2001-07 average).
Whether this adjustment occurred as “a progressive thing, running month by month, or maybe randomly bi-monthly-monthly, but not in blocks of four months” is not as interesting to me as to why it occurred. Were there major errors to correct? Are there specific “variance adjustments” that are made as Steve suggested earlier. Are values that seem to be “outside the normal range” discarded as “outliers”?
I have no notion, and am just trying to find out in simple terms what happened.
This is not a “gotcha” finger pointing exercise. I am really interested in knowing what caused this major adjustment for this four-month period, where all other records showed a significant drop versus earlier years, but Hadley (after adjustment) did not.
Thanks for your patience, but I hope to get to the bottom of this, so please bear with me.
Regards,
Max
Sorry. The comparison is for the first 4 months, i.e. January through April (and not as I said in one sentence January through March).
Hi Phil,
In my post 124, I was basically trying to find out why the Hadley record (after correction) for the first 4 months of 2008 appeared to be out of line with the other records.
At first Hadley showed a similar cooling in 2008 versus the average of the previous 7 years as GISS, and around half of that shown in the other two records.
After adjustment, Hadley shows only half the 2008 cooling that GISS shows and only one-fourth of the cooling shown in the two satellite records.
To make this easier to see I have plotted it in a simple graph.
It is puzzling to me, but there must be a simple explanation.
Regards,
Max
manacker (126)
July 31st, 2008 at 5:58 pm
“Hi Phil,
In my post 124, I was basically trying to find out why the Hadley record (after correction) for the first 4 months of 2008 appeared to be out of line with the other records.”
Just a quick question: does the chart take into account the differences in zero due to the different averaging periods?
That would tend to lower the GISS peak. Then it would appear that GISS and the Hadley original would match…
Reply to henry #128
Sorry for delay in responding. The figures shown on the graph are the deltas between January-April 2001-2007 average and January-April 2008, so the different “zero points” are irrelevant.
Regards,
Max
I am about to push the “submit comment” button when I really have a question. Perhaps someone can answer it or refer me to a blog or site which addresses it.
In my limited and long-ago math and statistics studies, the serial autocorrelation which was common to time series data was an important factor in identifying statistical significance of differences. In my case this was differences in simulation results. The serial autocorrelation impacted simulation run lengths – similar to sample size problems.
Isn’t this issue relevent to analysis of short-term trends in surface temperatures? It would seem to be – and I assume it is being taken into account – but I do not see it referred to. In particular – the oft-sited observation – that 9 of the 10 highest temperature years in some period all fell into the 90’s – or whatever. Isn’t that quite expected – not surprising at all?
Re: Schoen (#131), good question. The serial correlation is indeed very important in climate time series, as it is often high. See the Kousoyiannis thread for more info, or google CA for “autocorrelation” (box in the top right corner of this page) for more info than a man could want.
As to whether having a lot of recent records is unusual, yes this is impacted by serial correlation. However, it is a curious corner of statistics (the statistics of record-setting values), so there is more in the mix than just correlation. In a time of generally rising temperatures (e.g. the last 400 years or so) we would expect to find high-temp records being more common in the recent past.
w.
How about that. I was wondering why the weather seemed cooler inspite of the alleged global warming. I was told local weather is a “single data point.” But melting ice in the arctic is global warming, even though it too is local.
10 Trackbacks
[…] McIntyre has been tracking the changes closely on his Climate Audit site, and reports that NASA is Rewriting History, Time and Time Again. The recent changes can be seen by comparing the NASA 1999 and 2007 US temperature graphs. Below is […]
[…] GISS GHCN adjustments also were observed to occur frequently. John Goetz in February 2008 found on average 20% of the historical record was modified 16 times in the prior 2 […]
[…] http://www.climateaudit.org/?p=2964 […]
[…] final adjustments to the gathered data before final analysis. These adjustments are in some cases frequent and undocumented. Examining raw data versus processed final data shows numerous examples where the […]
[…] present temperatures, but past temperatures change as well. For a brief introduction to this, see Rewriting History, Time and Time Again, Climate Audit, 6 April […]
[…] final adjustments to the gathered data before final analysis. These adjustments are in some cases frequent and undocumented. Examining raw data versus processed final data shows numerous examples where the […]
[…] the temperature could not be determined. from Climate Audit, the guys that found the Y2K bug Rewriting History, Time and Time Again Climate Audit __________________ if you stop blaming me, I will stop pointing out why you are at […]
[…] […]
[…] McIntyre has been tracking the changes closely on hisClimate Audit site, and reports that NASA is Rewriting History, Time and Time Again. The recent changes can be seen by comparing the NASA 1999 and 2007 US temperature graphs. Below is […]
[…] climateaudit.org […]