Comments on: False Positives

By: David Graves

David Graves — Mon, 23 Jul 2012 01:26:09 +0000

Sorry, red, not white noise data sets.

By: David Graves

David Graves — Mon, 23 Jul 2012 01:22:34 +0000

I am interested in the notion of data set selection for the “white noise” for the McIntyre and McKitrick 2005 paper in GRL—how did that get resolved?

By: J Ashley

J Ashley — Thu, 12 Jul 2012 17:24:07 +0000

How times have changed. Remember the splash Alan Sokal’s “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity” made in 1994? Here’s the link to that classic:

http://www.physics.nyu.edu/sokal/transgress_v2/transgress_v2_singlefile.html

By: Weekly Weather and Climate News Roundup | Watts Up With That?

Weekly Weather and Climate News Roundup | Watts Up With That? — Sun, 08 Jul 2012 23:30:38 +0000

[...] http://climateaudit.org/2012/06/28/false-positives/ [...]

By: Jay

Jay — Sun, 08 Jul 2012 14:58:55 +0000

Mann and the pseudo-scientists at RealClimate should be forced to read the following which includes a damning statement…

“Furthermore, in my experience I saw many 3-sigma bumps go away as more data were accumulated.”

http://www.huffingtonpost.com/victor-stenger/higgs-and-significiance_b_1649808.html?utm_hp_ref=science

By: AntonyIndia

AntonyIndia — Sun, 08 Jul 2012 03:07:04 +0000

Some "bonus material" from the same interview of this Nature news article in here.

By: AntonyIndia

AntonyIndia — Sun, 08 Jul 2012 02:57:34 +0000

Nature published an interview with Uri Simonsohn called The data detective. Some quotes: "I was working on another project on false positives and had become pretty good at picking up on the tricks that people pull to get a positive result3. With the Smeesters paper, I couldn’t find any red flags, but there were really far-fetched predictions.The basic idea is to see if the data are too close to the theoretical prediction, or if multiple estimates are too similar to each other. I looked at several papers by Smeesters and asked him for the raw data, which he sent. I did some additional analyses on those and the results looked less likely." "Journals should be embarrassed when they publish fake data, but there’s no stigma. They’re portrayed as the victims, but they’re more like the facilitators" "Simply that it is wrong to look the other way. If there’s a tool to detect fake data, I’d like people to know about it so we can take findings that aren’t true out of our journals. And if it becomes clear that fabrication is not an unusual event, it will be easier for journals to require authors to publish all their raw data. It’s extremely hard for fabrication to go undetected if people can look at your data."

By: La confianza. « PlazaMoyua.com

La confianza. « PlazaMoyua.com — Fri, 06 Jul 2012 06:42:51 +0000

[...] False Positives [...]

By: Hu McCulloch

Hu McCulloch — Tue, 03 Jul 2012 04:49:49 +0000

GLP = Good Laboratory Practice:
http://en.wikipedia.org/wiki/Good_Laboratory_Practice

It’s too bad that Forest et al ( http://www.climateaudit.org/2012/06/25/nic-lewis-on-forest-et-al-2006/ ) didn’t comply with this data retention standard.

By: uppyn

uppyn — Sun, 01 Jul 2012 12:58:03 +0000

Being a quality manager in a high tech company, much of this sounds to me like basic knowledge in quality management:
1. Requirements for products and test should be decided on and documented up front.
2. Sample sizes for any data collection need to be determined up front and must be large enough to yield statistically significant results.
3. Parameters to be measured are also determined up front, e.g. through a process control plan.
4. All measurement results are to be documented (raw data) reported (summary data).
5. So called “outliers” are to be highlighted. A justification is to be provided if any outlier is removed from the data set. Ideally, an impact of such removal on the results is shown.
6. Any analysis method, if not standard, is to be described.

In a business setting, I find that the concept of risk assessment plays a more dominant role than in an academic environment. This may be recognized by readers who work in a business.

I remain surprised that what is considered to be standard rigor in business and engineering is the subject of current literature in academia and climate science.

Uppyn