Climate scientists should think about data quality more often, says Jones

After unveiling the Hadley Center adjustment error that has been used in all temperature compilations for the past 20 years, Phil Jones stated:

Climate scientists should think about data quality more often, says Jones, so that there is no opportunity for incorrect data to sow seeds of doubt in people’s minds about the reality of climate change.

This is the same Phil Jones who said:

We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it. There is IPR to consider.

Peter Webster, like Hans von Storch before him, was nonplussed at this attitude and, at his request, I provided the supporting email in a comment on another thread, noting that von Storch, also in disbelief, had contacted Phli Jones directly for confirmation, obtaining such personal confirmation, which von Storch had then reported to the NAS panel, as I discussed here, linking to von Storch’s PPT.

During the past few years, I’ve posted progress reports on CRU’s obstruction of efforts to find out even the simplest information about how they do their calculations – things as simple as a list of stations in their temperature calculations. These progress reports are scattered through many posts and I’ve collated into a PDF online here, covering two topics:

1) efforts to identify the station data used in CRU temperature analyses, and, once that had been refused, efforts to obtain even a list of stations used by SRU. Two generations of inquiry are shown, first by Warwick Hughes in 2005 and then by Willis Eschenbach in 2006-2007, which after 3 years and countless attempts only resulted in a not quite complete list of stations.

2) efforts to obtain a list of stations used in Jones et al 1990, a prominent study purportedly proving that the UHI effect was inconsequential. Once this list was obtained, an examination of the list of Chinese stations by myself and Doug Keenan, showed that claims in Jones et al 1990 to have selected stations based on careful examination of station history metadata could not possibly be true, as such metadata did not exist, which led Keenan to file a complaint against one of the authors.

This collation draws on previous posts at Climate Audit and my correspondence files.

42 Comments

  1. DaveR
    Posted May 30, 2008 at 9:34 AM | Permalink

    What was the outcome of Keenan’s complaint?

  2. Jeff A
    Posted May 30, 2008 at 9:36 AM | Permalink

    Typo after the second blockquote, “Phli Jones” 😉

  3. Michael Jankowski
    Posted May 30, 2008 at 9:38 AM | Permalink

    With “25 or so years invested in the work,” Jones can claim he thinks about data quality frequently and give himself an out.

  4. Andrew
    Posted May 30, 2008 at 9:38 AM | Permalink

    Steve, von Storch’s presentation seems to be a dead link.

  5. Stan Palmer
    Posted May 30, 2008 at 9:58 AM | Permalink

    There is a error in the URL for the PDF. There is a comma after the www

  6. George
    Posted May 30, 2008 at 10:01 AM | Permalink

    Typo in the address of the PDF online.

    Looks like www,climateaudit…
    instead of http://www.climateaudit...

  7. Severian
    Posted May 30, 2008 at 10:08 AM | Permalink

    Notice the inherent bias in the statement:

    Climate scientists should think about data quality more often, says Jones, so that there is no opportunity for incorrect data to sow seeds of doubt in people’s minds about the reality of climate change.

    It’s not, hey, let’s think about data quality so we don’t come to the wrong conclusions or make mistakes. No, it’s the TRVTH is known, we need to be careful so we don’t sow seeds of doubt about AGW/climate change.

    This, to me, is as reprehensible a statement as the one where he refused to provide data. It speaks to a mindset and attitude that facts are irrelevant to science, we know the answer, we don’t need no stinkin data, but we need to not give the skeptics any ammunition.
    snip

  8. retired geologist
    Posted May 30, 2008 at 10:31 AM | Permalink

    With each passing topic on this blog I find myself more and more incredulous that the IPCC, the Hockey Team, Al Gore, and all the other “warmists” have any credibility with the politicians, the public, and the MSM. Is there any climate data at all that isn’t modified, adjusted, or mathematically tortured with odd algorithms before they use it to justify their apparently foregone conclusions. I simply have no faith whatsoever in anything they say. I know that Steve Mc’s work is finally having some good influence but it baffles me that AGW still has any legitimacy. Here is a simple question, Is there any data set at all that can be used “as Is” that bolsters AGW?

  9. stan
    Posted May 30, 2008 at 11:15 AM | Permalink

    #8! Yes, indeed. Credibility is the key. You nailed it.

    To be credible, climate scientists should long ago have done what Watts and his volunteers are now doing. Someone should have actually looked over Mann’s work before it was used to justify global alarm. The scientists Steve Mc has identified should have agreed to open their work for genuine review and critical testing. They should have followed proper forecasting principles. And enlisted the help of the best statistical modellers.

    If they had, perhaps they’d have some credibility. But they didn’t.

  10. Alan S. Blue
    Posted May 30, 2008 at 11:18 AM | Permalink

    After unveiling the Hadley Center adjustment error that has been used in all temperature compilations for the past 20 years,

    Is that true of Loehle’s paper? Because I don’t recall any of the modern instrumentations in there.

  11. Posted May 30, 2008 at 11:21 AM | Permalink

    Re #1, DaveR, my allegation is still being investigated. A web page with updates on the affair is
    http://www.informath.org/apprise/a5620.htm

  12. Craig Loehle
    Posted May 30, 2008 at 12:55 PM | Permalink

    Re: Alan Blue: no, Loehle’s 2 papers did not use Hadley data except to discuss.

  13. Gerald Machnee
    Posted May 30, 2008 at 1:13 PM | Permalink

    **Climate scientists should think about data quality more often, says Jones**.
    Does he mean “ALL” climate scientists including himself?
    The man speaks with two faces.

  14. Sean
    Posted May 30, 2008 at 1:14 PM | Permalink

    Can someone answer a stupid question from a non-scientist? What is “gridded data”? Believe it or not, I searched google and wikipedia for a definition and could find anything anywhere.

  15. Andrew
    Posted May 30, 2008 at 1:25 PM | Permalink

    14 (Sean): It refers to data which is sorted into latitude/longtitude “grids” usually.

  16. jeez
    Posted May 30, 2008 at 1:33 PM | Permalink

    http://www.cru.uea.ac.uk/cru/data/temperature/

    Both the component parts (land and marine) are separately interpolated to the same 5º x 5º latitude/longitude grid boxes. The combined versions (HadCRUT3 and HadCRUT3v) take values from each component and weight the grid boxes where both occur (coastlines and islands). The weighting method is described in detail in Jones et al. (2001). Land temperature anomalies are infilled where more than four of the surrounding eight 5º x 5º grid boxes are present, as discussed in Jones et al. (2001). Infilling doesn’t take place when the box is ocean, except when it covered by sea ice based on 1961-90 average conditions.

  17. adrian starks
    Posted May 30, 2008 at 1:40 PM | Permalink

    I get involved in court cases and we would have to provide full disclosure of documents and audit trail of what we have done. Yet the court actions I am involved in pail into insignificance compared to the economic damage that could occur from climate policies.
    I am happy to take the medicine if necessary but not just for the fun of it!
    I ride a mcycle most of the time, we got 54mpg out my VW the other day so I do care, yet when I look at what CA reveals I am alarmed.
    To me the summation might be, there has been a very slight warming during the last century maybe 0.3c when corrected, temperature rises have stopped for a decade, the sun is at its minimum output hence cool temperatures globally. To me if climate scientist cannot predict 10,15 20 yrs into the future how can they say what will happen in 100 yrs.
    I am happy to acknowledge I may be wrong so please tell me if I am and why

  18. MarkW
    Posted May 30, 2008 at 2:16 PM | Permalink

    pail into insignificance

    That would be “pale into insignificance”, I don’t want to start another flame war on bucket adjustments.

  19. Bruce
    Posted May 30, 2008 at 2:30 PM | Permalink

    Beyond the pale: Unacceptable; outside agreed standards of decency.

    Beyond the Pail: Unacceptable; outside agreed standards of scientific inquiry.

  20. David Jay
    Posted May 30, 2008 at 2:50 PM | Permalink

    Engine inlet temps are beyond the pail 🙂

  21. Earle Williams
    Posted May 30, 2008 at 3:02 PM | Permalink

    Re #14

    Sean,

    Most methods and tools for analyzing quantitative data work best when the data are sampled at regular intervals. For example it is much easier to process and interpret temperature data taken once every 24 hours over the course of a month than it is to pocess temperature data taken in the morning on the 1st, three times on the 2nd, none on the 3rd or 4th, in the afternoon and evening of the 5th, and so on. This irregularly sampled data is often recast into a regularly sampled form. You could calculate a value for each day at 12:01 pm by averaging the nearest values in time. For better accuracy you would want to weight the average so that readings taken close to 12:01 pm would have more weight in the average than readings taken in the early morning or evening, or even on a different day.

    Irregularly sampled spatial data are also recast into a regularly spaced form, aka the grid of coordinates such as latitude and longitude. There are a host of methods and caveats to apply when generating a grid data from irregularly or randomly sampled data.

    It is such a fundamental aspect of many fields of scientific and engineering endeavor that you would think some basic information can be found. Most of what Google yields is tied to specific software, such as this wiki page: http://en.wikipedia.org/wiki/Rockworks#Two-Dimensional_Modeling_.28Gridding.29

  22. Anthony Watts
    Posted May 30, 2008 at 10:05 PM | Permalink

    FYI I have a short review of what the instrumentation used for manual SST measurement from the era looks like:

    http://wattsupwiththat.wordpress.com/2008/05/30/buckets-inlets-sst%e2%80%99s-and-all-that-part-2/

  23. mccall
    Posted May 30, 2008 at 10:22 PM | Permalink

    buckets-inlets-ssts-and-all-that-part-1

  24. Philip Mulholland
    Posted May 31, 2008 at 1:37 AM | Permalink

    #14 Sean

    Data are numerical values of a variable environmental property (e.g. temperature) measured at a specific location at a given time. Data locations are control points used to generate contour maps that show the expected spatial distribution of the environmental variable across a given area of interest. Contour maps can not be generated by computer from data collected at randomly distributed control points (e.g. weather stations), so these data locations have first to be gridded into an array of nodes spaced at regular intervals measured using the X and Y axes of a Cartesian matrix. This process of gridding is performed by a mathematical technique that populates the array of nodes with values that fit the expected trend of data between adjacent control points (by interpolation) and also expand the data to the boundaries of the area of interest (by extrapolation).

    # 22 Earle

    Nice link, thanks

  25. Lawrence Beatty
    Posted May 31, 2008 at 4:54 AM | Permalink

    I’m not too sure if I’m off beam here Steve, but as I struggle with all the data and science a theme seems to have developed.

    1) Sloppy science
    2) Blatant non-cooperation and sharing of methods and data.
    3) An almost unwillingness generally by Nasa and the IPCC to query or question any science paper if it’s pro AGW.

    This all smacks to me of being driven by ideology and that trumps objectivity every time.

    Now I would guess any of us if trying to prove something we really believe and as a result have invested so much,
    in time and commitment, that we would feel rather devastated when evidence proves us wrong.

    But this now goes beyond the pale or bucket as in this case.

    It would seem to me that there is a whole raft out there of scientist, journalist, opportunists and especially politicians that want AGW to be happening regardless of the facts, and if they don’t fit well then make them.

  26. Mark_T
    Posted May 31, 2008 at 7:09 AM | Permalink

    Anybody know what “IPR” stands for in Jones reply to Steve M.?

  27. Carl Gullans
    Posted May 31, 2008 at 7:22 AM | Permalink

    I believe that he meant Intellectual Property Rights.

  28. Jim Hanrahan
    Posted May 31, 2008 at 7:26 AM | Permalink

    Re: #27

    I read it as Intellectual Property Rights.

  29. schlew
    Posted May 31, 2008 at 1:48 PM | Permalink

    adrian stark says

    To me if climate scientist cannot predict 10,15 20 yrs into the future how can they say what will happen in 100 yrs.

    It is even more amazing to me that climate scientists are having difficulty predicting temperatures 20 years in the past!

  30. henry
    Posted May 31, 2008 at 6:45 PM | Permalink

    Jones:

    We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it. There is IPR to consider.

    Question, then: If there are “IPR” (Intellectual Property Rights) to consider, then what kind of paperwork is required for another scientist (researcher, or student) to fill out? Do they get the raw data, or the “heavily adjusted” data?

    Also, if another researcher uses the data, and the journal or paper requires archiving or posting of the data, could Jones block that requirement?

    If a paper using this data surfaces, and needs to be sent to reviewers, do the reviewers have access to the raw data?

    All in all, how long can one man sit on data before others cry foul?

  31. Posted Jun 2, 2008 at 4:37 AM | Permalink

    Climate scientists should think about data quality more often, says Jones, so that there is no opportunity for incorrect data to sow seeds of doubt in people’s minds about the reality of climate change.

    This may be a good reason for a political activist to consider the quality of data that underpins policy decisions, but why would a scientist say such a thing?

    If data quality is a serious problem in the climate debate, surely the mind-set of some scientists who produce and interpret it is a far greater one. Phil Jones, one of the most influential players in the production of GMT estimates, seems to be focused on the propaganda value of research, not its scientific integrity.

    This quote confirms a deeply embedded political bias and suggests that advocacy, and not objectivity, is his priority.

  32. Sean
    Posted Jun 3, 2008 at 8:34 AM | Permalink

    Philip, Andrew and Earle,

    Thanks very much for the information on gridding, which I now understand.

    And thanks to everyone who posts/comments on this blog. I generally try to follow along and am definitely learning a lot from all of you.

  33. Posted Jun 5, 2008 at 1:37 PM | Permalink

    I live 15 miles from the Hadley Centre and take a special interest in their activities, especially as I come across some of their scientists professionally. My concern is that they are being paid by the British taxpayer to spread what appears to me to be propaganda, to British Schools and Businesses. Can people kindly look at my posting on ‘unthreaded’ under ‘Hadley centre’ and give me your expert comments on the presentation material being used. Many thanks in advance.

    Tony B

  34. John Link
    Posted Jun 6, 2008 at 4:56 PM | Permalink

    I wonder how Jones can cite Intellectual Property Rights without disclosing what that property is. After all, the whole idea behind patents is to disclose your invention to the public in return for a right to enforce others against making, using or selling it without your permission. Ditto copyright: to enforce it you’ve got to go to an infringer and say “Hey, I wrote, published and copyrighted that, and I can prove it!”

    But in both cases the patentee or copyright holder is asserting a right involving creations of the mind that have been prevously disclosed to the public, not data taken from instrument measurements. There’s no creativity in the latter, and keeping such data secret is completely antithetical to patent and copyright law—not to mention to the spirit of science itself.

    Imagine if Michelson and Morley had said “Yep, our data confirms the existence of the ether — but we’re not going to share it with you.” They would rightly have been regarded as charlatans.

    Wasn’t there a spoof called “The Journal of Irreproducible Results”?
    Maybe Jones can publish and copyright his data there.

  35. Jeff A
    Posted Jun 6, 2008 at 7:32 PM | Permalink

    But if the data were gathered/produced with government/public funds, Jones has no IPR claim.

  36. henry
    Posted Jun 7, 2008 at 10:55 AM | Permalink

    To me, the only IPR claim that Jones may have would be to the method he used to make his assumption, and not to the data used (he created the code, for example).

    Some of the data he uses comes from other countries, and part of the arguement he uses is that there are some countries that don’t want their data released. Yet he even refuses to list which contries these are, so that other researchers can approach them and ask to use the data.

  37. Tom Gray
    Posted Jun 7, 2008 at 3:02 PM | Permalink

    re 39

    The IPR issue is likely to be a non-disclosure agreement.

    It is possible that these agencies require purchasers of the data to agree not to pass it on to other parties. It is an issue for international standards committees. These groups used to finance themselves, in part anyway, by the sale of copies of the standards. Copying machines were an issue with this but the advent of the web was a particular problem. People expected that these public standards should be free to access. That was possible but how were the standards groups going to get the money to make them free to access.

  38. P.M.W.
    Posted Jun 8, 2008 at 9:50 PM | Permalink

    Intellectual Property Rights apply to something that has been created by the rights holder.

    Seems to me that, for Intellectual Property Rights to apply in this case, the clear implication is that the data is a work of fiction.

  39. Posted Jun 13, 2008 at 2:40 PM | Permalink

    “Climate scientists should think about data quality more often”

    so does this mean the opposite is true and data quality is thought of rarely?

  40. Chris
    Posted Nov 22, 2009 at 5:39 AM | Permalink

    And now Phil Jones’ misconducts are all exposed on the web!!!!!!!
    Read this:
    http://blogs.telegraph.co.uk/news/jamesdelingpole/100017393/climategate-the-final-nail-in-the-coffin-of-anthropogenic-global-warming/

    and then download the related zip file from one of these sites:

    http://www.megaupload.com/?d=003LKN94
    http://thepiratebay.org/torrent/5171206
    http://www.filedropper.com/foi2009

  41. Troy
    Posted Nov 23, 2009 at 4:46 PM | Permalink

    Phil Jones and Michael Mann, I have one thing to say,
    “YOU LIE”

  42. Richard Linsley Hood
    Posted Jan 30, 2010 at 5:34 AM | Permalink

    Steve, anyone,

    I have downloaded the Jones et al Model data for Oxford UK from “http://image.guardian.co.uk/sys-files/Guardian/documents/2009/12/08/uk.csv” (an extract from “http://www.metoffice.gov.uk/climatechange/science/monitoring/reference/All.zip”) and compared it to the Actual temperatures as recorded at “http://www.metoffice.gov.uk/climate/uk/stationdata/oxforddata.txt” for the relevant dates, 1900-1980.

    These two series are believed to be the same station.

    Model.

    Number= 038900
    Name= OXFORD
    Country= UK
    Lat= 51.7
    Long= 1.2
    Height= 63

    and

    Actual.

    Oxford
    Location: 4509E 2072N 63 Meters

    I have caclulated the tMonthlyMean values from the Actual data (tMonthlyMax, tMonthlyMin) and compared it to the Model tMonthlyMean figures on a month by month basis. These mostly show a +-0.05 difference (which is presumably due to some rounding errors to get to 0.1 degree published values either by me or by others).

    Can anyone tell me why, then, the last fews years of the Model data (1978-1980) differs so widely from the Actual recorded temperatures? The Model is out by up to 2.2 degress C and an average of 0.36 degress C of apparent warming when compared to the Actual temperatures for just these last few years. That is, the Model shows a unexpected temperature rise of up to 2.2 degrees C in Oxford if using the Actual figures.

    Actual – Model Oxford 1978-1980
    Year Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
    1978 0.15 0.35 0.35 0.35 0.35 0.15 0.25 0.40 0.35 0.25 0.05 0.45
    1979 0.15 0.25 0.35 0.35 0.30 0.15 0.25 0.40 0.90 0.25 0.20 0.05
    1980 0.20 0.25 0.70 0.40 0.35 0.15 0.35 0.40 0.40 0.30 0.15 2.20

    Anyone with any insights into this divergence of the Jones et al Model from Actual?

2 Trackbacks

  1. […] was measured. James Annan says “oops”, and points out that this has been discussed at ClimateAudit before. In fact, it was discussed almost two years ago. But how will this change the temperature […]

  2. By Religion, not science | Cold Fury on Jun 6, 2008 at 1:35 PM

    […] How “science” gets “settled”: We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it. There is IPR to consider. […]