Nick Brown Smelled BS

http://narrative.ly/pieces-of-mind/nick-brown-smelled-bull/ h/t Mosher.

94 Comments

  1. Skiphil
    Posted Oct 21, 2013 at 4:11 PM | Permalink

    WOW. Sokal has done some great service for science with his prior “hoax” paper in the journal Social Text. It’s nice to see these three authors come together in such a helpful way for this new project.

    There is also an interesting piece on science journal standards and procedures here:

    article in The Economist on problems of replicability in scientific research

    In testimony before Congress on March 5th Bruce Alberts, then the editor of Science, outlined what needs to be done to bolster the credibility of the scientific enterprise. Journals must do more to enforce standards. Checklists such as the one introduced by Nature should be adopted widely, to help guard against the most common research errors. Budding scientists must be taught technical skills, including statistics, and must be imbued with scepticism towards their own results and those of others. Researchers ought to be judged on the basis of the quality, not the quantity, of their work. Funding agencies should encourage replications and lower the barriers to reporting serious efforts which failed to reproduce a published result. Information about such failures ought to be attached to the original publications.

    And scientists themselves, Dr Alberts insisted, “need to develop a value system where simply moving on from one’s mistakes without publicly acknowledging them severely damages, rather than protects, a scientific reputation.” This will not be easy. But if science is to stay on its tracks, and be worthy of the trust so widely invested in it, it may be necessary.

    • Gphill
      Posted Oct 21, 2013 at 9:07 PM | Permalink

      this problem is systemic and occurs at the most basic levels

      “A major factor is the failure to validate methods carefully when
      they are first introduced or implemented. This failure can often
      be attributed to a lack of sufficient analytical training. Researchers
      require a large knowledge base covering a variety of disciplines. Among those, knowledge in statistics and analytical methods is often sadly lacking.”

      SN Young, GM Anderson
      Bioanalytical inaccuracy: a threat to the integrity and efficiency of research
      J Psychiatry Neurosci. 2010 January; 35(1): 3–6.

      http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2799499/

  2. Posted Oct 21, 2013 at 4:28 PM | Permalink

    I particularly resonated with this bit:

    “For me, the real question is not about Fredrickson or Losada or Seligman,” Sokal says. “It’s about the whole community. Why is it that no one before Nick—and I mean Nick was a first semester part-time Master’s student, at, let’s be honest, a fairly obscure university in London who has no particular training in mathematics—why is it that no one realized this stuff was bullshit? Where were all the supposed experts?”

    “Is it really true that no one saw through this,” he asks, “in an article that was cited 350 times, in a field which touts itself as being so scientific?”

    CA readers could no doubt fill in the blanks…

    “For me, the real question is not about ____ or ____ or ____,” Sokal says. “It’s about the whole community. Why is it that no one before ____ —and I mean ____ was a part-time ____, at, let’s be honest, a fairly obscure _____ in ______ who has no particular training in ____ —why is it that no one realized this stuff was bullshit? Where were all the supposed experts?”

    “Is it really true that no one saw through this,” he asks, “in an article that was cited 350 times, in a field which touts itself as being so scientific?”

    • j ferguson
      Posted Oct 22, 2013 at 4:04 AM | Permalink

      Ross,
      Maybe it touts itself as “scientific” because it isn’t. In the ’50s, answering “false to the question “Sociology is a science equal in stature to chemistry or physics; true or false?” got me an interview with the sociology teacher to have the correct view better explained to me.

      It was amazing that she taught the scientific method in this class yet didn’t grok that none of the social scientists employed it.

    • Stephen Richards
      Posted Oct 22, 2013 at 4:26 AM | Permalink

      Ross
      I think this was the uni I attended some 30 yrs ago. It wasn’t called Uni of E London then. At that time all the lecturers in physics had PhDs in physical subjects and I thought I was well taught. However, you are right that it is not well known.

      Steven Mosher, thanks for finding this and persuading SteveMc to put it on his great site.

      • Steven Mosher
        Posted Oct 24, 2013 at 10:19 AM | Permalink

        no persuasion required. I read it and was speachless.

    • Posted Oct 24, 2013 at 7:49 AM | Permalink

      Ross,

      I’m with you completely, I was going to ‘go off’ on that quote too – so widely applicable these days. I guess I dump it from clipboard now.

  3. Bernd Palmer
    Posted Oct 21, 2013 at 4:45 PM | Permalink

    Highly interesting and impressing! Any resemblance to other branches of science is purely coincidental.

  4. timothy sorenson
    Posted Oct 21, 2013 at 5:06 PM | Permalink

    Remember this: http://www.shapingtomorrowsworld.org/Lewandowsky_2013_Recursive_Fury.pdf with his analysis of McIntrye’s analysis, all of it was based on we are honest, competent, scientist working here!

    • GrantB
      Posted Oct 21, 2013 at 5:38 PM | Permalink

      If only Fredrickson and Losada had included some commentary along the lines of, “the recursive fury of the Lorenz attractor”, they might have got away with it.

      • Posted Oct 21, 2013 at 6:02 PM | Permalink

        Ha. The recursive fury of the Lorenz attractor becomes my phrase of the week.

  5. Posted Oct 21, 2013 at 5:14 PM | Permalink

    Wow! Amazing similarities – particularly, as Ross notes above, “It’s about the whole community. Why is it that …”

    This “sacred shield” known as “peer review” certainly seems to have been doing an awful lot of damage to itself, for quite some time. I sure hope that these silently acquiescent “communities” begin to wake up and take some concrete steps towards reversing this … uh … trend towards “mediocrity forever”!

    As an aside, back in the early ’90′s, somewhat tired of the proliferation of pseudo-psych books on the market, I started writing a parody, Calling a Spade a Spade: Life’s not a b*tch, it’s a bridge game in which I introduced the concept of “bridging”. From the intro:

    In the past 20 years, we have seen a proliferation of books, gurus, theories and fads to help us do our jobs and live our lives. Think about “The Games People Play”, “I’m OK, You’re OK”, “Dance of Anger”, “You Just Don’t Understand”, Tom Peters, Dr. Ruth, Ralph Nader, the power of positive thinking, Total Quality Management, empowerment, inclusiveness, team-building, political correctness, partnerships with stakeholders, making a difference, mediation – the list is probably endless!

    You would think that with all this – and of course, the self-help “recovery” groups that seem to sprout up to fix almost every personal problem that you didn’t know you had – living in the 90’s should be a breeze.

    Perhaps now’s the time to retrieve it from the back-burner, dust it off, update and complete it ;-)

  6. Posted Oct 21, 2013 at 5:25 PM | Permalink

    Reblogged this on CraigM350.

  7. KNR
    Posted Oct 21, 2013 at 5:40 PM | Permalink

    We been here before in Psychology , for years Cyril Burt’a work on intelligence was treated as ‘god like ‘ with this work being cited time and again with it going far as to become a standard text for many universities.

    Trouble was it was all lies , he never even did the claimed research his claims were based on , for instance he claimed he had research assistants that no one ever heard of and of whom there is no record .

    And what brought all this out was an outsider , Eysenck, looking at the maths and seeing for the BS it was at a rather basic level. Why for years had no one done the same thing , and why even now some will defend it , is a very good question.

    So its not just climate ‘science’ were academic BS can be turned into gold if the will is there and people cannot to bother to challenge the consensus.

    • bernie1815
      Posted Oct 22, 2013 at 12:02 AM | Permalink

      KNR:
      The Burt affair has many complexities. The issue of whether he in fact faked his data as far as I can tell remains unresolved. His assistants have been identified. Given the political and ideological perspectives of many of those involved at the time – including Kamin and Hudson – and more recently, I see no quick resolution. Gould, who in some ways reignited the debate, has been shown to have completely screwed up his PCA. Ring any bells?

      • charles the moderator
        Posted Oct 22, 2013 at 1:14 AM | Permalink

        Re: bernie1815 (Oct 22 00:02),

        Gould also opposed the Data Quality Act as an attack on scientists. I used to love to read Gould, but he was, in his own way, a part of the team.

  8. MrPete
    Posted Oct 21, 2013 at 5:40 PM | Permalink

    I also appreciated this:

    When you’re dealing with people like this—particularly with people like Losada, although I must say I was disappointed by Fredrickson’s reaction too—you have to kill every point stone dead.”

    aka

    When you’re dealing with people like this—particularly with people like ____, although I must say I was disappointed by ______ reaction too—you have to kill every point stone dead.”

    • Steven Mosher
      Posted Oct 22, 2013 at 2:18 PM | Permalink

      a horse is horse of course, except when its a racehorse

      • MrPete
        Posted Oct 22, 2013 at 9:40 PM | Permalink

  9. Posted Oct 21, 2013 at 6:04 PM | Permalink

    Thanks, Steve and Steven. Wonderfully written article.

  10. Henry
    Posted Oct 21, 2013 at 6:35 PM | Permalink

    “The paper had exceeded the two-month limit of which authors were allowed to respond to a target article” makes one wonder how science is supposed to develop. I somehow doubt this is unique to American Psychologist.

  11. Mole Cat
    Posted Oct 21, 2013 at 7:17 PM | Permalink

    Have to love a consensus.

  12. RoyFOMR
    Posted Oct 21, 2013 at 7:17 PM | Permalink

    Gosh!
    After 28Gate it seems we have a new prefix; 2.9013Gate.
    Granted it doesn’t have the same alliteration as its predecessor but it does seem to follow an emerging association with PNS and magic numbers.
    1.618Gate, 2.7183Gate and 3.1416Gate seem sure to follow.

    • Stephen Richards
      Posted Oct 22, 2013 at 4:28 AM | Permalink

      Doesn’t roll of the tongue quite as well as 28gate :))

  13. Posted Oct 21, 2013 at 7:40 PM | Permalink

    After reading a paper that showed that her positivity ratios were of little empirical value Ms. Fredrickson wrote,
    “It is important to recognize that considerable theory and evidence point to the continued value of tracking and raising positivity ratios.”
    That’s an A+ for chutzpah.

    • Frank
      Posted Oct 21, 2013 at 8:28 PM | Permalink

      In other words, “It doesn’t matter” She has the “right” answer.

      • David L. Hagen
        Posted Oct 25, 2013 at 11:46 AM | Permalink

        Could that possibly be the “Appeal to Authority” logical fallacy? Say its not so!

  14. Posted Oct 21, 2013 at 8:46 PM | Permalink

    Thank you for bringing this to my attention. Special thanks to Steve Mosher.

  15. William Larson
    Posted Oct 21, 2013 at 9:33 PM | Permalink

    I’ll channel Karl Popper yet once again: It actually is not SCIENCE until it (the paper) has been thoroughly checked over and the methods found to be sound and the results found to be reproducible. “Publishing the paper” is in itself merely the introduction to the science. And what needs to follow after the publication of the paper is often the most difficult part of the process called “science”. That is one large reason why CA–in my opinion, and probably in yours–is so very valuable. Nick Brown, credentials or not, you have done some true science.

    • Steven Mosher
      Posted Oct 22, 2013 at 2:19 PM | Permalink

      papers advertise science. the data and code document it.

  16. AntonyIndia
    Posted Oct 21, 2013 at 10:17 PM | Permalink

    Who’s Afraid of Peer Review? By John Bohannon, a science journalist. http://www.sciencemag.org/content/342/6154/60.full
    Few smelled BS or even smelled. It must be a hot topic even amongst the Establishment considering the 196 comments. “Mysteriously” he missed the opportunity to include the two biggest – but non Open Access – publications: Science and Nature: was he afraid of the outcome?

  17. ianl8888
    Posted Oct 21, 2013 at 11:34 PM | Permalink

    Thanks to SMc for putting this example up on a thread

    Been said quite often before: “There is little problem with being wrong. The grievous sin is not being published”

  18. Geoff Sherrington
    Posted Oct 22, 2013 at 12:17 AM | Permalink

    Egads! This makes one think of the Dr Strangelove defence:
    ……………………………
    (General in SAC)Turgidson: The duty officer asked General Ripper to confirm the fact the he had issued the go code and he said, “Yes gentlemen, they are on their way in and no one can bring them back. For the sake of our country and our way of life, I suggest you get the rest of SAC in after them, otherwise we will be totally destroyed by red retaliation. My boys will give you the best kind of start, fourteen hundred megatons worth, and you sure as hell won’t stop them now. So let’s get going. There’s no other choice. God willing, we will prevail in peace and freedom from fear and in true health through the purity and essence of our natural fluids. God bless you all.” Then he hung up. We’re still trying to figure out the meaning of that last phrase, sir.

    (USA President)Muffley: There’s nothing to figure out General Turgidson. This man is obviously a psychotic.

    Turgidson: Well, I’d like to hold off judgment on a thing like that, sir, until all the facts are in.

    Muffley: (anger rising) General Turgidson, when you instituted the human reliability tests, you assured me there was no possibility of such a thing ever occurring.

    Turgidson: Well I don’t think it’s quite fair to condemn a whole program because of a single slip up sir.
    ……………..
    The movie is suggested viewing from time to time as it has elegant psychology cloaked in narrative.

    • John Ritson
      Posted Oct 26, 2013 at 2:15 AM | Permalink

      Geoff, how many times have I told you guys that I don’t want no horsin’ around on the airplane?

  19. Brian H
    Posted Oct 22, 2013 at 4:04 AM | Permalink

    Replication-challenge seems to have lost its savour. Wherewith shall it be salted?

  20. Jeff Norman
    Posted Oct 22, 2013 at 8:00 AM | Permalink

    “But the editor in chief, he’s honorable,” 

    This quote caught my eye.

    I can see the BS aspect of Nick’s complaint. How you feel about an event is completely subjective. Two different people experiencing the same event in the same context can have two completely different responses. Though I suppose one’s response could be telecommunicated to the other if say their calibration period did not overlap (or something).

  21. Posted Oct 22, 2013 at 8:03 AM | Permalink

    “My opinion of the paper has always been that it was a metaphor, disguised as modeling,” said David Pincus, a psychologist at Chapman University who specializes in the application of chaos theory to psychology. [ my strong ]

    What is the mapping between the physical and the psychological for the exacting balance between energy input and energy utilization in chaos. This critical balance produces bounded excursions of the dependent variables.

    I think human behavior is generally an elliptic-in-time problem. We sometimes make decisions at the present time based on estimations of what might happen at future time. We sometimes consider these future occurrences to be very highly likely, or very highly unlikely. Elliptic-in-time problems are well-known to be ill-posed.

    However, at other times our actions are based solely on past information; a hyperbolic problem. A discrete explicit hyperbolic problem is stable if the time extent of the past information meets certain limitations.

    Human behavior can never be a parabolic problem because we can never know all boundary conditions at all times.

  22. pesadia
    Posted Oct 22, 2013 at 8:32 AM | Permalink

    Seems like hockey sticks come in all shapes, sizes and guises

  23. Matt Skaggs
    Posted Oct 22, 2013 at 9:07 AM | Permalink

    Not a one among this esteemed group of skeptics seems to have caught the journalistic trick in the subject article. An effectively structured hit piece – in this case aimed directly at the entire field of positive psychology – starts with a general description of a field of study, then seamlessly segues into a description of a particulary embarrassing occurrence. The two are conflated to the extent that the reader is left with the impression that the entire field of study has no more merit than reading tarot cards. Our host has on many occasions warned readers that the statistical antics of Michael Mann do not mean that the entire field of dendrochronology is a circus. The field of positive psychology does have valid tools for posing and answering questions about the human condition, and some thorough and cautious practitioners. This is not to deny that there are field-wide issues that slow progress, such as disinterest in negative results.

    • j ferguson
      Posted Oct 22, 2013 at 9:49 AM | Permalink

      Matt,
      Is it possible that you missed the point. Brown’s concern was raised by no-one else objecting to this publication. And yes, I would tar the profession for at the minimum not paying attention.

    • miker613
      Posted Oct 22, 2013 at 9:57 AM | Permalink

      You note the problem. But what’s the solution? How’s a outsider like me supposed to know that anyone in the field is capable of doing real solid reproducible work? I don’t have the time or the inclination to plow through the research papers; I have to trust that the experts know what they’re doing. Or not.

      I really don’t see any way out of suggesting, politely but firmly, that if there are any competent people in positive psychology, it is their job not mine to smoke out and remove really ridiculous faux-science from their field.

      In climate science: It seems to me that hardly anyone on the AGW-believers’ side really grasps the impression that Climategate made on many of us: Huh – politicians, not scientists. That kind of impression takes a long time to go away, and it was completely self-inflicted, nothing to do with Oil Money. Mann and Schmidt did it to themselves. People like Ross McKitrick and Judith Curry help. Yes, help, fools out there – I now feel that there is some competent oversight. I at least will hear when new stuff is garbage, even if people who read SkS never will. They help. People who heap abuse on them don’t.

      • j ferguson
        Posted Oct 22, 2013 at 10:02 AM | Permalink

        Rob Wilson’s recent talk to the students reported at Bishop Hill’s is a good solution. Honest observations of a practitioner. Just as Judith Curry’s.

    • Skiphil
      Posted Oct 22, 2013 at 10:18 AM | Permalink

      Sure, one example may not discredit an entire field…. Except that this case involves two of the pre-eminent figures to date in the “positive psychology” movement (Frederickson and Seligman), plus the 350 or so articles which cited the Frederickson work without understanding. There may be much that can be salvaged from work in “positive psychology” to date, who knows, but it will require serious review and re-analysis.

      Also, there are many offshoots and spin-offs across more “applied” work in leadership, organizational development, and so on which need to be re-assessed.

      For instance, some years ago I picked up a book which had been required in a friend’s graduate level course in “leadership” for an MBA program. It is “Leadership and the New Science” by a Margaret Wheatley. I found it quite unreadable, just a melange of pseudo-scientistic analogies and metaphors posed as having a rigorous scientific basis.

      There is always danger from mis-placed physics envy in social science, and this Wheatley book seems to be an egregious example of pretended rigor for people who don’t know better. It is reportedly assigned in many university courses and might be another suitable object of some “auditor” debunking!

  24. hunter
    Posted Oct 22, 2013 at 10:23 AM | Permalink

    So Lewandowsky is not a fluke for this profession.
    The Nick Brown article is amazing.
    Too bad that AGW is even more insular than psychology.

  25. b4llzofsteel
    Posted Oct 22, 2013 at 10:37 AM | Permalink

    Retraction watch on this subject:

    http://retractionwatch.wordpress.com/2013/09/19/fredrickson-losada-positivity-ratio-paper-partially-withdrawn/

    • Skiphil
      Posted Oct 22, 2013 at 11:27 AM | Permalink

      Comment from Sokal at the Retraction Watch link is well worth quoting here as responsive to points raised in this thread (thinking about climate science comparisons):

      Sokal told us he thinks the case raises other issues:

      “Last but not least, there is a huge open question, which concerns not Fredrickson and Losada but the entire psychology community, and particularly those people working in “positive psychology”. How could such a loony paper have passed muster with the reviewers at the most prestigious American journal of psychology, netted 350 scholarly citations, and been repeatedly hyped by the “father of positive psychology” (and past president of the APA), without anyone calling it into question before a first-term part-time Masters’ student in Applied Positive Psychology at the University of East London came along and expressed his doubts? Where were all the leaders in the field of positive psychology? The leaders in the application of nonlinear-dynamics models to psychology? Was everyone really so credulous? Or were some people less credulous but politely silent, for reasons of internal politics?”

    • Skiphil
      Posted Oct 22, 2013 at 11:28 AM | Permalink

      Comment from Sokal at the Retraction Watch link is well worth quoting here as responsive to points raised in this thread (thinking about climate science comparisons):

      Sokal told us he thinks the case raises other issues:

      “Last but not least, there is a huge open question, which concerns not Fredrickson and Losada but the entire psychology community, and particularly those people working in “positive psychology”. How could such a loony paper have passed muster with the reviewers at the most prestigious American journal of psychology, netted 350 scholarly citations, and been repeatedly hyped by the “father of positive psychology” (and past president of the APA), without anyone calling it into question before a first-term part-time Masters’ student in Applied Positive Psychology at the University of East London came along and expressed his doubts? Where were all the leaders in the field of positive psychology? The leaders in the application of nonlinear-dynamics models to psychology? Was everyone really so credulous? Or were some people less credulous but politely silent, for reasons of internal politics?”

  26. paullinsay
    Posted Oct 22, 2013 at 11:12 AM | Permalink

    I found the whole article quite amusing, especially how late the psychologists are to the chaos fad. Peak chaos was about twenty years ago with all sorts of fields claiming new understanding based on chaos theory, of course none of them exhibiting any comprehension of non-linear dynamics. The application I liked the best was the businessmen who claimed that it enabled them to deliver cement to construction sites faster than their competitors in Mexico City. If you’ve ever experienced traffic in Mexico City that’s quite a claim.

    Psychology is never going to be a science because people aren’t atoms. Two different people will react to the identical situation differently and the same person may respond differently to the same situation but at different times. You’re probably better off with Aesop’s Fables, Biblical Proverbs, and Poor Richard’s Almanac for advice.

    • Skiphil
      Posted Oct 22, 2013 at 11:43 AM | Permalink

      Re: “chaos fad”

      Still going strong in many courses and workshops which have relied upon works such as the Margaret Wheatley, “Leadership and the New Science” over the past two decades:

      [emphasis added]

      “Amazon.com Review
      When Margaret J. Wheatley’s Leadership and the New Science was initially published in 1992, it outlined an unquestionably unique but extremely challenging view of change, leadership, and the structure of groups. Many readers immediately embraced its cutting-edge perspective, but others just could not understand how the complicated scientific tenets it described could be used to reshape institutions. Now Wheatley, an organizational specialist who has since coauthored A Simpler Way, updates the original by including additional material (such as an epilogue addressing her personal experiences during the past decade) and reconstructing some of her more challenging concepts. The result is a much clearer work that first explores the implications of quantum physics on organizational practice, then investigates ways that biology and chemistry affect living systems, and finally focuses on chaos theory, the creation of a new order, and the manner that scientific principles affect leadership. “Our old ways of relating to each other don’t support us any longer,” she writes. “It is up to us to journey forth in search of new practices and new ideas that will enable us to create lives and organizations worthy of human habitation.””

      • None
        Posted Oct 22, 2013 at 2:53 PM | Permalink

        “The result is a much clearer work that first explores the implications of quantum physics on organizational practice

        I could at first think of no means through which the discreet energy levels of individual subatomic particles could affect organizational practice. Perhaps they also just assume teleconnection ?

  27. j ferguson
    Posted Oct 22, 2013 at 12:13 PM | Permalink

    Wheatley:

    The result is a much clearer work that first explores the implications of quantum physics on organizational practice, then investigates ways that biology and chemistry affect living systems, and finally focuses on chaos theory, the creation of a new order, and the manner that scientific principles affect leadership.

    How much of a grasp of this subject does one actually need to conclude that anything she might have to say in this area is highly likely to be nonsense?

  28. Stacey
    Posted Oct 22, 2013 at 12:16 PM | Permalink

    Bulls**t baffles brains

  29. Posted Oct 22, 2013 at 3:37 PM | Permalink

    “To look for enduring stabilities across space and time,” Harris Friedman says, “they’re hard to come by.”

    “The essence of the criticism of the critical positivity ratio is that it takes quantitative reasoning to its absurd extreme,” he says, “that because we can talk about things in numerical terms, that that makes it scientific.”

    Turning tree rings into numbers, then taking varves and ice cores and corals and stalactites and doing the same thing and then merging them all – you end up with a lot of numbers. Granted science IS quantification, since the days 1050 years ago when reductionism and positivism took over. And it appears to be true that the “soft” sciences DO have “physics envy” in their earnest desire to be taken seriously. They early on had to decide to somehow put numbers to moods and feelings and states of mind.

    Something like this was bound to happen.

    But it isn’t very far from Deepak Chopra (especially) with self-help claptrap bullsh*t about tying quantum mechanics to New Age concepts. Not far at all. It just happened to be bullsh*t enough – by bringing in math that the editors were not willing to admit was over their heads.

    That is exactly what the climatologists have done with the politicians at IPCC – made it impossible for them to admit, “I don’t understand this part – can you explain the math and how it applies?” Instead the pols went the other route – and took the over-their-head math and added ANOTHER layer of bull to it, twisting even the dubious math into policy directions.

    Editors, politicians, wannabe scientist/hangers-on – they all allow themselves to be hoaxed by numbers of really dubious relevance and applicability.

    And Professor Mann has just the overbearing personality to rub their noses in it if they don’t “get it.” Just like he bullied the hell out of Keith Briffa in order to truncate the tree ring data at 1960 to hide the decline.

    People don’t want the world to know something is over their heads, so they will let it pass and hope for the best.

    It’s actually cowardice. The cowardice of being phonies – no one wants to admit they are ignorant in some aspect of their field. The original editors should have run the maths by mathematicians in the first place.

    Just as Mann should have run his work past statisticians who actually KNEW how to deal with such data.

    The public deserves better. Science is supposed to have gatekeepers. Those gatekeepers are the journals. When something gets through the gatekeepers, it is supposed to be vetted – peer-reviewed” they call it – and that is the point of trust in the system. It gets by, it’s good. It doesn’t, it is she*t.

    The pols take it the same way the public does: They trust that the science had the gatekeepers’ anointing. But if they have agendas, do they let that trust become gullibility?

    And that is what this is all about: Gullibility.

    Climatology or positive psychology or Deepak Chopra’s phony New Age guru posturing – throwing scientific bullsh*t at people, and most people think, “Well, he seems to be an authority on this, so it must be true.”

    So, what we end up with is Michael Mann being a phony New Age guru, because no one was willing to admit to being skeptical about his Deepak Chopra math. Except that when any skepticism came up about Mann’s work, Mann bullied the tar out of them, bullied them into submission.

    At least Deepak Chopra does it with a smile.

    • jeez
      Posted Oct 22, 2013 at 3:54 PM | Permalink

      Re: Steve Garcia (Oct 22 15:37),

      You made me go there.

      • Geoff Sherrington
        Posted Oct 23, 2013 at 12:47 AM | Permalink

        Well chosen clip, jeez. Thank you, jeez, Nick Brown, Steve & Stephen.
        Unfortunately, I have no novel suggestions for combatting the poor science that we meet so often.
        Long have I wondered aloud why authors’ colleagues and others skilled in a particular art can sit idly by, knowing that such poor quality material is being reviewed and published as if it was the epitome of excellence.

  30. Craig Loehle
    Posted Oct 22, 2013 at 5:43 PM | Permalink

    Back when chaos theory was newish, I published a few papers on it in the field of Ecology but based on actual dynamic models, not “concepts”. On that basis I was invited to work with some people on a cusp catastrophe model of something or other. After arguing for several rounds that the model they developed was floating in air disconnected from anything, I withdrew my co-authorship. The paper was published. It wasn’t bulls**t wrong, just incoherent. For me, it was not worth the publication to be an author on a paper that didn’t make sense.

  31. Posted Oct 22, 2013 at 8:55 PM | Permalink

    Craig -

    So much for the quality control claimed by the peer review process.

    The weak spot is – well, the weak spot is the peers doing the review, it seems. In more ways than one.

  32. Steven Mosher
    Posted Oct 22, 2013 at 11:42 PM | Permalink

    • Posted Oct 23, 2013 at 5:35 PM | Permalink

      I found it funny they called out Tony Robbins in the write-up. He has probably the best TED talk ever given.

      I think the larger point being, any attempt to parameterize soft notions in psych/soc is always less compelling than you think. But there is “non-math” approach to understanding humans and desires, which may be in fact more effective.

      • Steven Mosher
        Posted Oct 24, 2013 at 2:47 PM | Permalink

        yup.

        the issue has to do with unbridled “positivism” the notion that without numbers you cant say anything. hence the drive to reduce everything to numbers.

        also there are interesting things like the nocebo effect.

  33. EdeF
    Posted Oct 23, 2013 at 12:16 AM | Permalink

    I don’t often use Math with Psychology, but when I do, I prefer to use Schrodinger’s equation. XX

    • Geoff Sherrington
      Posted Oct 23, 2013 at 12:56 AM | Permalink

      EdeF
      When I mix math with psychology, I think of this -

    • AJ
      Posted Oct 23, 2013 at 6:04 AM | Permalink

      You must be the world’s most interesting mathematical psychologist :)

  34. James Smyth
    Posted Oct 23, 2013 at 1:41 AM | Permalink

    Wow. I slogged through the Brown paper, all the way to this:


    Unfortunately, there is one final, yet crucial, flaw lurking here: the values of
    sigma, b, and (especially) i plugged into Equation 6 are totally arbitrary, at least within wide limits; so the predicted critical positivity ratio is totally arbitrary as well. Choose different values of the parameters , b, i and one gets a completely different prediction for (P/N)crit. Recall that Saltzman (1962) chose sigma = 10 for illustrative purposes and purely for convenience; then Lorenz (1963) and Losada (1999) followed him. Were humans to have eight fingers on each hand instead of five, Saltzman, and in turn presumably Lorenz and Losada, might well have chosen sigma = 16 instead of sigma = 10 — which (with b = 8/3) produces a very similar Lorenz attractor, except that the borderline of chaos is now rcrit = 1040/37 = 28.108, and the predicted critical positivity ratio (with i = 16) is (P/N)crit = 1233/296 = 4.1655405. Yet other values of sigma, b, i would yield still different predictions for (P/N)crit. Thus, even if one were to accept for the sake of argument that every single claim made in Losada (1999) and Losada and Heaphy (2004) is correct, and even if one were to further accept that the Lorenz equations provide a valid and universal way of modeling human emotions, then the ideal minimum positivity ratio that Fredrickson and Losada (2005) claimed to have derived from Losada’s “empirically validated” nonlinear-dynamics model would still be nothing more than an artifact of the arbitrary choice of an illustratively convenient value made by a geophysicist in Hartford in 1962.

    • James Smyth
      Posted Oct 23, 2013 at 1:45 AM | Permalink

      Whoops, missed a link to the actual paper.

    • William Larson
      Posted Oct 23, 2013 at 7:03 PM | Permalink

      Yeah, and here’s something that I HOPE would have tipped me off had I been in Brown’s shoes in that class: This ratio (2.9013) in the article linked is reported to FIVE significant figures! What?? Hello!!? Can you even get any physically measured quality in physics to five sig figs (the density of a substance? the resistance of a resistor?), much less in psychology? Psychology?? And no one smelled this rat? And even in the Brown et al. paper: 1040/37 has only two sig figs and so cannot be reported as “28.108″, etc. I am willing to place a side bet that Fredrickson does not even know the definition of “significant figures”. What percentage of psychologists do know the definition? (I am pretty sure that Groucho Marx would have a definition, actually, but not from the perspective of a psychologist.)

      • Brian H
        Posted Oct 24, 2013 at 12:33 AM | Permalink

        Re: William Larson (Oct 23 19:03),

        And even fewer know that in any calculation, the result must be reported with the same count of SFs that is the least used in any factor or additive element.

      • William Newman
        Posted Oct 26, 2013 at 9:04 AM | Permalink

        “Can you even get any physically measured quality in physics to five sig figs (the density of a substance? the resistance of a resistor?), much less in psychology?”

        For what it’s worth, yes, there are many quantities in physics which can be measured more precisely than that. See e.g. the first 11 tables in “CODATA recommended values of the fundamental physical constants: 2010″ http://physics.nist.gov/cuu/Constants/codata.pdf . Note also that for some specialties this has been true for quite a long time: consider the famous Roemer/Huygens estimate ca. 1680 AD of the speed of light based on anomalies of a few minutes (over the course of many years) in the observations of the orbit of Jupiter’s moons.

        (In psychology? Probably not so much.:-)

  35. Posted Oct 23, 2013 at 11:07 AM | Permalink

    In his article ‘Nick Brown Smelled Bull’ (Oct 17 2013 on Website ‘Narratively’), author Vinnie Rotondaro said,

    “By the late winter of 2012, Friedman, Sokal and Brown were all in touch via email and working together towards a draft of what would become “The Complex Dynamics of Wishful Thinking.”
    The three men brought different skills to the plate. Brown was the outsider, the instigator, who, knowing no better, dared to question the theory in the first place. Friedman provided psychological expertise and played a diplomatic role, helping guide the paper towards publication. Sokal was the finisher, the infamous debunker with the know-how needed to dismantle the theory in hard, mathematic language.”

    - – - – - – - –

    Mosher, a great find and thank you for suggesting that it be posted here.

    What Rotondaro describes was an effective team concept to be encouraged in auditing climate research papers. The team consisted of: the critical questioning new apprentice; the diplomatic old skeptical master; & the independent sharpshooter cum mathematician.

    A team like that is led by the passionate apprentice. It says a lot about human collaborative efforts. : )

    The post was inspiring.

    John

    • Steven Mosher
      Posted Oct 24, 2013 at 10:16 AM | Permalink

      thanks. Steve’s framing commentary says it all.
      kinda speaks for itself doesnt it

  36. anonymist
    Posted Oct 24, 2013 at 4:28 AM | Permalink

    Before anyone gets carried away throwing laurels at Sokal’s feet, you should check what he’s said about the climate debate. In the preface (on pages xv-xvi) of his Beyond the Hoax he endorses the Republican-war-on-science/merchants-of-doubt explanation for the existence of continued controversy on the issue; the blurb trumpets that idea too. On the other hand, on pp. 97 and 101 he talks about “our lack of knowledge about other subjects, eg. the global climate” and says that “many of the central political issues of the coming decades — from health care to global warming to Third World development — depend in part on subtle and hotly debated questions of scientific fact”, not really in line with the overwhelming-consensus/science-is-settled thesis — but that’s a reprint of an essay written in 1996, unlike the preface from 2010. (Disclaimer: I don’t have access the while book yet.)

    • anonymist
      Posted Oct 24, 2013 at 4:30 AM | Permalink

      (That should read “have access to the whole book yet” of course.)

  37. Posted Oct 24, 2013 at 7:54 AM | Permalink

    WOW! I’m in awe. Calculating the ‘positivity ratio’ to FIVE significant digits? TAKE THAT you physicists! What an incredible data set they must have! I wonder where its been archived?

    W^3

  38. C. Baxter
    Posted Oct 24, 2013 at 9:13 AM | Permalink

    We all know that peer review has its “issues” – and always will. Usually, it does not matter to the wider world – only to the guys whose paper(s) get the can. We dust ourselves down and either fight the reviewers and win or fight them and loose. If the latter, then we usually move on. The trouble really occurs when politicians and the media latch on to something published by peer review and use it to further an agenda. Personally, I would ban all politicians from entering the Library. Well, I would do if I were convinced they could read.

  39. David L. Hagen
    Posted Oct 24, 2013 at 4:49 PM | Permalink

    See no evil! Tone down the language!
    Strong negative evidence may be needed!

    “Ironically,” (Gaustello) wrote, “I did send American Psychologist a comment on some of the foregoing points, which they chose not to publish because ‘there wasn’t enough interest in the article.’ In retrospect, however, I see how I could have been more clearly negative and less supportive of any positive features of the original article.”

    The dichotomy of “positive psychology”!

  40. JimmyBen
    Posted Oct 24, 2013 at 5:57 PM | Permalink

    I don’t know about these tricky positivity ratios.

    That great scientist, Walt Disney, had it summed up years ago:-

    When you wish upon a star
    Makes no difference who you are
    Anything your heart desires
    Will come to you

  41. HR
    Posted Oct 24, 2013 at 6:11 PM | Permalink

    If you’re looking for more debunking in the psychology field try this one.

    http://www.psychologytoday.com/blog/fulfillment-any-age/201301/the-secrets-behind-psychology-s-most-famous-experiment

    I heard Gina Perry talking onthe radio recently and very convincing she was. I’m ideologically mistrustful of the conclusions of the Milgram Experiment so it’s possible I find all this convincing because I want it to be true!

  42. Howard
    Posted Oct 24, 2013 at 11:57 PM | Permalink

    It’s worse than we thought…

  43. observa
    Posted Oct 25, 2013 at 6:08 AM | Permalink

    I was gasping throughout at the parallels of this ABC Catalyst programme ‘Heart of the Matter’ part1 between the Cholestorol Club and the Climatology Club as you can well imagine.

    http://www.abc.net.au/catalyst/stories/3876219.htm

    I’m looking forward to part2 when the manufacturers of Australias most prescribed stain drugs get asked the hard questions. Watch the Heart Foundation guy Manning up and trying not to smirk.

    The sublime irony of it all is completely lost on the left/green numpties at Aunty(our BBC equivalent) who have been busy this last week parrotting Gore, Figueres and having love-ins with McKibben as they all connect CAGW with the Sydney bushfires.

    Thinking about the programme later and how the rise and rise of the Cholestorol Club occurred from a cherry-picked hockey stick graph and how it sustained itself for so long took my breath away but you could see how the rise of modern heart surgery began to puzzle the cardiology community. What they were seeing on the operating table was not what their learned elders had taught them at uni and some began to look back at the basis of 40yrs of correlation without causation. Ouch!

  44. observa
    Posted Oct 25, 2013 at 6:09 AM | Permalink

    oops.. statin drugs

  45. observa
    Posted Oct 25, 2013 at 6:25 AM | Permalink

    Here’s a quick rundown of statin prescription, bearing in mind there’s only 23mill of us-

    http://www.smh.com.au/national/health/statins-offer-quick-fix-for-heart-for-a-price-20130921-2u6na.html

  46. Matt Skaggs
    Posted Oct 25, 2013 at 10:16 AM | Permalink

    Well this thread certainly turned into an ugly little experiment. Readers were presented two information sources, the article and the paper. The paper is no doubt (I did not read it) a very squishy bit of science, mostly nonsense. The article is a journalistic persuasion piece beginning with the equivalent of “it was a dark and stormy night,” with plenty of character development, misdirection and a helping of nonsense. Both were intended to persuade. But to a certain type of crowd, there was a vast difference between the two. The paper pushed all the wrong buttons for those with the preconceived notion that fields like psychology have no useful tools in the toolbox. In contrast, the article pushed all the right buttons, even leading with the significant figure bait, and full of red meat for self-avowed rational empiricists. In this thread we see nothing but thumbs down for the paper, which is as it should be, but we also see nothing but thumbs up for the article. In short, we see that even a group of skeptics who have self-selected for a blog based on sophisticated critical thinking do a very poor job of remaining objective when presented with an argument that plays to their biases.
    Don’t believe me? Answer the question below without rereading the article. The article contains this statement:

    “’Is it really true that no one saw through this,’ [Sokal] asks, ‘in an article that was cited 350 times, in a field which touts itself as being so scientific?’”

    “No one saw through this” is:
    1. True.
    2. Not quite right, there was one guy.
    3. Totally wrong, lots of folks criticized the paper.

    • Steven Mosher
      Posted Oct 25, 2013 at 10:44 AM | Permalink

      “The article contains this statement:”

      you meant : The article contains this question.

      The answer is one of the reasons why I forwarded the article to steve.
      As we all know from the climategate emails there were other critics of Mann.
      They were not so public.

      I’ve made the same point with respect to the Piltdown man. Early on the hoax was questioned. But no one paid attention. In fact early on some suggested it was a fluke. To counter this suggested a “confirming” second example was produced. Finally, it continued to be cited after it was discovered as a hoax. There is a lesson there about the sociology of science.

      My main purpose in forwarding this to steve was that I saw parallels between
      this, steve’s story and the piltdown episode.

    • Howard
      Posted Oct 25, 2013 at 10:49 AM | Permalink

      You need to look in the mirror Matt Skaggs. You claim:”nothing but thumbs up for the article” Many people did not comment on the article. In any event, you are merely pointing out that even at climate audit where there is a patina of objectivity, many suffer from motivated reasoning and confirmation bias. Your post identified you in this same category with a complete lack of self awareness.

      What is disturbing to me is that the lack of critical examination of studies and the overwhelming desire for “positive” results in peer review literature apparently pervades many if not all specialties in science.

      This is a human problem and the sooner you and everyone else accepts the philosophy of Pogo, the better off we will all be. Until then, we continue in the endless loop of finger pointing between each territorial circle-jerk.

    • Nicholas
      Posted Oct 25, 2013 at 6:03 PM | Permalink

      Matt, I think you’re reading a bit too literally.

      When I read the sentence you quote, it seems pretty obvious (to me anyway) that Sokal isn’t literally saying that *nobody* *ever* saw through this. Firstly, the implication he’s making is that nobody saw through it at the time it was published. Obviously, folks started to question it later. I don’t think that’s in dispute is it?

      But I think it’s more subtle than that. What I interpret that sentence as meaning (and perhaps he could have worded it better, but it is a quote after all and possibly an off-the-cuff remark) was to say “Is it really true that nobody at the time saw through this *and had both the ability and desire to prevent it from being published*”.

      I think a lot of people reading his quote will have a similar interpretation to mine. Did you not read that into it or are you just being obtuse? Admittedly, you have to think about what he actually means rather than what the sentence means if you interpret it completely literally but when I’m reading a quote that’s generally what I’m doing.

      Anyway, I think that’s kind of the point of both this episode and the Hockey Stick papers. In both cases, there’s a good chance that not only did multiple people see through the nonsense but that at least some of the were in a position to stop the papers being published or at least make a public objection. In both cases they decided not to. Whether it’s because it was easier to ignore it than do anything about it (probably), would have damaged their careers if they had made a stink, despite them being correct (almost certainly) or as part of an I-look-away-when-you-publish-rubbish-and-you-do-the-same-to-me culture, who knows. Although ClimateGate does give us some insight into what really went on behind closed doors.

    • Hartley Gardner
      Posted Oct 26, 2013 at 8:09 PM | Permalink

      Matt, I believe you are trying to say that it would be false to say that “no one noticed”, since the article clearly states that others had issues with the original paper. What I believe Sokal was trying to convey, however, goes a bit deeper, in that he is trying to ask why no one with the clout/prestige/power in that field of study noticed it was based on error/imagination and took steps to have it withdrawn (or at least modified), thus allowing the paper to exist essentially unchallenged for a long, long time. The parallels to other scientific fields should be clear..:)

  47. Matt Skaggs
    Posted Oct 25, 2013 at 3:06 PM | Permalink

    Thanks for the correction Steven. Even with obsessive proofreading, I always screw something up. There is some fascinating stuff in positive psychology using simple Game Theory models such as the Prisoner’s Dilemma. Personality is even more complicated than climate, but as with all relatively new areas of study, the scientists willing to take baby steps with simple tools can make big discoveries.
    I also find the comparison of Mann’s work with Piltdown Man fascinating. Once Piltdown Man was debunked, it was very difficult for anyone to believe that the perpetrator could be so ignorant as to naively assemble parts from different animals into a complete skeleton . Therefore there was not a substantial debate (AFAIK) as to whether Piltdown was fraud or incompetence…it had to be fraud. But if you stitch together a bunch of murky data sets of questionable relevance, apply some murky math and stonewall when asked for details, there will always be some doubt as to what you knew, and when you knew it.

    • Jeff Alberts
      Posted Oct 26, 2013 at 12:18 AM | Permalink

      But if you stitch together a bunch of murky data sets of questionable relevance, apply some murky math and stonewall when asked for details, there will always be some doubt as to what you knew, and when you knew it.

      Maybe, to people without brains.

  48. David L. Hagen
    Posted Oct 25, 2013 at 4:24 PM | Permalink

    A related issue are graphs – which may be
    difficult to read due to incompetence
    and/or
    are designed to “hide the decline” etc.

  49. Brandon Shollenberger
    Posted Oct 25, 2013 at 5:39 PM | Permalink

    Of course, it’s harder to expect people to believe simple incompetence when you calculate verification scores that show your key results are unsupported, hide those scores while publishing others then deny having calculated any at all.

    We can see via their code Mann et al knew their r2 scores were zero for the earlier portions of their reconstructions. In Michael Mann’s book, he acknowledged they knew (after the fact) their MBH98 results were dependent on a small amount of bristlecones. Despite (or because of) this knowledge, they went to lengths to hide these facts.

  50. observa
    Posted Oct 26, 2013 at 8:19 AM | Permalink

    My field of study was economics and in the early 80s we were busy feeding punch-cards into a Dec10 monster ready to mathematically model whole national economies, presumably in order to forecast and solve the world’s economic problems. Yes, well the best laid plans of mice and men as they say.

    It made as much sense as trying to model the human condition(Humanology anyone?) or global climate (Globology?) and it’s high time some common sense prevailed at the hubris and naivety of even thinking we can box up with a pretty pink ribbon such complex and chaotic big picture stuff. When men stop believing in God to explain the inexplicable they’ll obviously believe in anything, not least their own importance in the big scheme of things.

  51. observa
    Posted Oct 26, 2013 at 10:02 AM | Permalink

    JoNova teases out the obvious parallels and similarities-

    http://joannenova.com.au/2013/10/catalyst-says-consensus-wrong-on-cholesterol-but-unquestionable-on-climate/

  52. observa
    Posted Nov 1, 2013 at 8:05 PM | Permalink

    And you surmised correctly there’d be an appeal to authority, the science is settled, remember the precautionary principle, take your statins and let’s have no more of this nonsense Aunty-

    http://www.abc.net.au/pm/content/2013/s3878646.htm

Follow

Get every new post delivered to your Inbox.

Join 3,381 other followers

%d bloggers like this: