Here are some thoughts from 70 years ago from the eminent statistician, R.A. Fisher, from the “Design of Experiments”. There’s a nice dig about “heavyweight authorities”.
When any scientific conclusion is supposed to be proved on experimental evidence, critics who still refuse to accept the conclusion are accustomed to take one of two lines of attack. They may claim that the interpretation of the experiment is faulty, that the results reported are not in fact those which should have been expected had the conclusion drawn been justified, or that they might equally well have arisen had the conclusion drawn been false.
Such criticisms of interpretation are usually treated as falling within the domain of statistics. They are often made by professed statisticians against the work of others whom they regard as ignorant of or incompetent in statistical technique; and, since he interpretation of any considerable body of data is likely to involve computations, it is natural enough that questions involving the logical implications of the results of the arithmetical processes employed, should be relegated to the statistician. At least I make no complaint of this convention. The statistician cannot evade the responsibility for understanding the processes that he applies or recommends.
My immediate point is that the questions involved can be dissociated from all that is strictly technical in the statistician’s craft, and when so detached are questions only of the right use of human reasoning powers, with which all intelligent people, who hope to be intelligible, are equally concerned, and on which the statistician as such speaks with no special authority. The statistician cannot excuse himself when the duty of getting his head clear on the principles of scientific inference, but equally no other thinking man can avoid a like obligation.
The other type of criticism to which experimental results are exposed is that the experiment itself was ill designed or, of course badly executed. If we suppose that the experimenter did what he intended to do, both of these points come down to the question of the design or the logical structure of the experiment. This type of criticism is usually made by what I might call a heavyweight authority. Prolonged experience, or at least the long possession of a scientific reputation, is almost a prerequisite for developing successfully this line of attack. Technical details are seldom in evidence. The authoritative assertion “His controls are totally inadequate” must have temporarily discredited many a promising line of work; and such an authoritarian method of judgement must surely continue, human nature being what it is , so long as theoretical notions of the principle of experimental design are lacking — notions just as clear and explicit as we apply to technical details….
In the foregoing paragraphs, the subject matter of this book has been regarded from the point of view of an experimenter who wishes to carry out his work competently and having done so wishes to safeguard his results, so far as they are validly established, from ignorant criticism by different sorts of superior people.
I hope that I’m fitting into the first category; you can probably guess my views on the rest of the story.
Climate Audit 
12 Comments
In the first case the data are questioned, in the second the scientist.
H,mmmmmm
Steve, carry on – yout fail both!
You’re criticizing the stats of others but not from a heavyweight position, for sure. And not with blatant comments like “controls are out of wack” but with specific objections.
sometimes, I worry that you do things like “find faults in the substudies that you care most about” rather then verifying if such errors occur in ones that favor you also (and so the errors might cancel…after all that is part of the reason for a meta-analysis). Or you find some individual proxy that was left out (and would help MWP), but I can’t tell if it’s representative of the overall set of available proxies out there, because you haven’t done a real survey of all the studies.
TCO,
You’re getting annoying! You’re beginning to sound like the warmer trolls. Are you actually one in drag? You seem to be unwilling to understand the role of skepticism and what amounts to a market of ideas. Yes intellectual integrety requires a person to point out or admit to his own errors, but not to spend all his time attacking those who agree with him. It’s the opponents who have the responsibilty of defending their own positions and pointing out the flaws of others. If you can’t understand this you’re at best a dilettante and at worst a charlatan
Yes I am in drag. I thought you liked it though…big boy! We were getting along…
P.s. read that part of the Feymnann essay where he talks about publishing results of an experiment whichever way it comes out.
P.s.s (repeating my point): when Steve (or worse John) comes up with some individual proxy (a stalagtite or whatever) that shows MWP, I can’t properly evaluate it’s importance until I know that it is not just the one thing that supports their point of view and a bunch others (and more of them) would support the other side. In a sense studies are like data points. you can’t base things off of examples. You have to have some way to integrate and account for everything to make it relevant.
From someone who has been doing exactly that to another person (yours truly) whos only dispute (ironically) is the amount of heat by RD and if it exceeds solar heat.
Hey! Down boy! No thread to thread battles. This is my playpen now. Sticking to hitting in the aptly named HITS thread.
Wouldn’t have said anything if he hadn’t been so on point. Though I don’t see your point in digging up months old threads.
No thread to thread battles!
But thanks for the defense…dahling. 😉
Ok, you two.
First my complaint against “sid” was strictly limited to what he said which was wrong. But he misconstrued that to mean I disagreed with where he was right and / or agreed with wrong things. This is silly. I say what I mean and I mean nothing more than what I say.
And TCO, I explicitly added the section to my post stating it to be the intellectually honest thing to do to point out your own errors and under that rubric I’d include pointing out negative results.
The point is Steve isn’t your galley slave and yet you march around from thread to thread telling him what to do. Once in a while; fine. 10 time a day is irksome. At least that’s the opinion of this observer.
Well there is something (3rd line) That we can agree on.
As to the first line I would repeat the sttement changing only the name in quotes to Dave. And then I would re-iterate what I said to Ed about stepping into the middle of a fight.
DAve, I agree that my kibbitzing is likely annoying.