Seems that relying on models instead of actual data is rife in the climatology community.

Deamin’, as we say here.

Instead, the program office relied on the results of a model run by the contractor to predict system reliability, rather than testing reliability.

I read the report first referenced in aurbo #44. Although the report is late 1990s, records taken in that period that relied on the ASOS system were not without problems. I have selectively quoted two of the worst parts of the report, which admittedly does have some better news sprinkled through it. ASOS costs at that time were $US351 million and procedures such as aircraft safety and flood prediction relied on it to a degree.

From the report:

Furthermore, critical failures and errors that were caught and corrected by weather observers were not included in these results. Once these errors are included, ASOS failure and error rates are 730 and 1,680
hours respectively.

(Text deleted for brevity here. Not all errors are temperature related. Some are precipitation, wind etc. A MTBF of 700 hours for a remote, automated, long-term system is simply deplorable.)

The reason that ASOS’ reliability problems were not discovered during testing and corrected prior to system deployment and operation is
that ASOS program management repeatedly chose to defer testing of
mean time between failures. Instead, the program office relied on
the results of a model run by the contractor to predict system
reliability, rather than testing reliability.

One might conclude that some current uses of USA observations are beyond the capability of the data because of errors that cannot be adjusted in hindsight.

A lovely feature about Climate Audit is that one can ask a question and be rewarded with information that would be near impossible to find on a search. I had wondered aloud a few months back if spectral response, especially IR, affected different types of temperature devices. Now I have a comparison of two. It would be interesting to see how these compare in a similar experiment with the historic Hg in glass thermometer. The closest I have got does not answer this question, but is worth a revisit. Work in Australia by Jane Warne, Bureau of Meteorology,

http://www.wmo.ch/pages/prog/www/IMOP/WebPortal-AWS/Tests/ITR649.pdf

From the outset, the original design was a disaster. An early document outlining some of these problems can be found here.

Many of the subsequent fixes in the original equipment (HO-83) and later the HO-1088 sensors were often patches rather than a total revamping. The original contractor for ASOS had never built meteorological instruments prior to their contract, and when deficiences in the product were noted by the ovseright committee, it was too late to start from scratch as budget and personnel requirements were already committed to elimtaing the NWS observer employee specialty and new money was not available.

One should also check our this detailed comparison of ASOS and CRN temperature measurements.

OT, some road traffic revenue raising devices here were calibrated using a music tuning fork and the judicial system accepted that as reasonable.

On thread, if you think air temperature measurement is hard to sustain with accuracy, think about ice core temperature reconstructions and the accuracy claimed.

Fascinating report. It just goes to show that measurement is hard and accurate measurement is harder.