Lewandowsky Censors Discussion of Fake Data

The key issue for Lewandowsky et al 2012 is its use of fake data, a problem squarely addressed by Tom Curtis of Skeptical Science and discussed at CA here.

I raised this issue in a comment at Lewandowsky’s blog here as follows:

Rather than answer the question, Lewandowsky, the author of a paper entitled “NASA faked the moon landing|Therefore (Climate) Science is a Hoax: An Anatomy of the Motivated Rejection of Science”, deleted the question:

The invoked policy of the blog states:

No profanity or inflammatory tone. Again, constructive discussion is difficult when overheated rhetoric or profanity is flying around

My comment did not contain “profanity or inflammatory tone”. It was a direct question about Lewandowsky’s use of fake data.

He should answer the question, not avoid.

UPDATE: I followed up as follows:

Comment didnt last long.


  1. Cosmic Ray
    Posted Sep 10, 2012 at 4:11 PM | Permalink | Reply

    We should video these submissioms and when they get deleted, post them (before and after with links ) on YouTube
    so everybody can see how ‘open’ these people of truth are.

    • Steve McIntyre
      Posted Sep 10, 2012 at 4:20 PM | Permalink | Reply

      the snip tool in Windows is handy for imaging.

  2. hunter
    Posted Sep 10, 2012 at 4:16 PM | Permalink | Reply

    Telling the truth about an obvious, unartful con is inflammatory, to the con.

  3. Posted Sep 10, 2012 at 4:17 PM | Permalink | Reply

    Accusing someone of using fake data isn’t inflammatory? Interesting. Why not just come right out with it and accuse him of fraud?


    Steve: I;m not saying anything that Ton Curtis of SKeptical Science hasnt said. The responses were faked by readers of Deltoid, Tamino and Skeptical Science. I have no reason to believe that Lewandowsky was himself involved in construction of the fake responses, though his methodology was shoddy. Tom Curtis of Skeptical Science has raised identical issues and seriously questioned the ethics of Lewandowsky permitting the paper to go forward and urged him to withdraw it. Lewandowsky still has time to withdraw the paper but not much.

    • Hector M.
      Posted Sep 10, 2012 at 5:25 PM | Permalink | Reply

      Steve was not accusing Lewandonsky of deliberately faking data. He was referring to the possibility that some “fake responses” may have been included in the dataset instead of being excluded. A previous post in this same blog contains Steve’s detailed arguments to suspect that some fake responses exist (from non-skeptics posing as extremely skeptic cum conspiracy theory people) had made their way into the dataset. This hypothesis may be true or false, but the question is far from inflammatory, The issue of fake responses in online surveys is frequently discussed in the relevant methodological literature, as are also some remedies for preventing or minimizing them.

    • Ed Snack
      Posted Sep 10, 2012 at 9:19 PM | Permalink | Reply

      Straight out of the troll handbook, geez you guys are so good at misdirection and lying one would think you were specially trained it.

      Learn some comprehension skills and try reading the post before mouthing off, the accusation is that some of the responses to the survey were “faked”. Unless you wish to accuse Lewandowsky of actually making up the responses himself, is that your drift ? If so, I apologise but in that case I’m not sure what you’re accusing Steve of.

      • Posted Sep 10, 2012 at 9:52 PM | Permalink | Reply

        Well, actually the main point I was making was snipped by Steve which is fair enough. After all, this is his blog. The comments in the rest of this post verified the point i was making anyway. As for lying, I didn’t lie. Steve accused Lewandowsky of using faked data….sorry faked responses (is there a difference?) and the other part, that you can see was a question. Perhaps it’s not I that needs reading and comprehension skills?

        • Posted Sep 10, 2012 at 10:29 PM | Permalink

          Re: uknowispeaksense (Sep 10 21:52), Let’s assume that Lewandowsky is an honest person — i.e. — never lies. Let us also assume that he would never knowingly used “faked data”.

          If Steve McIntyre was able to show with elementary graphing and statistical analysis that the Lewandowsky analysis was wrong — could it be that only incompetence is left as an explanation for the Lewandowsky analysis of the survey?

          Or you could attempt to show that the McIntyre analysis is incorrect. That to me would be the only purposeful use of your time if you have a point to make. So we await your results.

        • Posted Sep 11, 2012 at 2:56 AM | Permalink

          @ WillR. Again, my main point was snipped. That’s fine. I think Steve knows I think he has achieved what I believe he actually set out to achieve. I say that because he is the one who ironically moderated my comment. For me, the first issue was about whether the comment was inflammatory or not. I think it was. Others beg to differ which is their right and that’s fine. As for Lewandowsky’s methodology, you can do all the assuming because while I am a scientist, I am not familiar with this kind of socialogy study and since I don’t suffer from the Dunning Kruger effect, won’t pretend to be an expert on it. -snip=

        • Posted Sep 11, 2012 at 8:37 AM | Permalink

          inaccuracy=inflammatory? Whatever you say. Congrats, by the way.

        • Skiphil
          Posted Sep 11, 2012 at 5:43 PM | Permalink

          good points, Barry Woods. I have made use of that website form to urge the editors of Psychological Science to have the paper revised and resubmitted for peer review (one hopes a more rigorous one this time). The tendentious title alone, unsupported by the contents of the paper (even as they stand now), should disqualify it as a credible scientific paper.

        • R A Kennedy
          Posted Sep 12, 2012 at 5:52 AM | Permalink

          The Journal Psychological Science is read by many professional and academic psychologists. It has a good reputation, reflected in a relatively high impact score.

          The journal has a retraction policy -

          It probably doesn’t have a “withdrawal” policy (few journals do), but the last thing any editor would want is to end up with a retraction (that goes for the publishers as well). In particular negotiating a retraction can be a fraught process when there are multiple authors.

          Regarding this paper, putting myself in the editor’s shoes, I would be worried. At the very least I would be having serious second thoughts about that title.

        • Posted Sep 12, 2012 at 7:41 AM | Permalink

          NASA faked the moon landing / Therefore (Climate) Science is a Hoax:
          An Anatomy of the Motivated Rejection of Science

          based on 3 response out of 1100, from very partisan anti-sceptic blog readers which appear to be scammed responses anyway.
          there are even comments in the survey article, about how ‘fun’ the conspiracy theories were.

          this alone should make the journal consider whether the journal’s prestige is beingused, and should prompt it to be sent back for a title rewrite/review.. and extra checking.

          (urls to surveys found here, 1st comment, it is also claimed that Skeptical Science hosted it, but very little evidence)

          The journal is probably oblivious to the online debate (or just hearing ‘one-side’)..
          If you let them know your concerns prior to publication, they may double check before publication.

          The journal has its own reputation to protect (and don’t necessarily care about any particular article, or activist professors reputations, it is a big flagship journal)


          Otherwise, once published it will be cited like Doran and Anderegg for ever (how ever many responses it receives.) and we will see quotes like ‘peer reviewed journal says all sceptics are conspiracy theorists’, also for ever.

          Perhaps ask the journal if any peer reviewer was expert in online survey methodology? I bet nobody asked for the data.

          The journal doesn’t seem to be that interested in climate change or sceptics.


          So no skin off their nose (or loss of face for the journal) if they ask for an extra peer reviewer for methodology, interpretation of 3 moon landing results, and the title.

          Why this journal, 2 years on from data collection (a red flag?) with perhaps a recommended (small) pool of peer reviewers to choose from, (this field is quite small)

          Another concern perhaps for the journal is perhaps the pre-press release (GUARDIAN) and press release (Telegraph) activity for this paper, prior to publication. Not good enough just to be in a flagship psychology journal, the professor seems to want to spread a message, using the journals prestige.

          I wonder if ANY of the reviews asked to see the raw data, OR were made aware of who the referring 8 blogs were – ALL hardcore antisceptic blogs, yet the Professor describes them very differently in the paper (ie diverse)

          No point contacting Hanich, or the ethic committee, or even writing criticisms of the paper on sceptical blogs, about concerns about this paper, (hanich is not even a co-author) but it is worth bringing to the journal editors attention, to be careful (as Lewandowsky is a co-author of John Cooks, debunking sceptics book), he clearly is not a neutral scientific observer in this paper) perhaps his own motivated reasoning and ideology, is unconsciously blinding him, and his peer reviewers to this.

          this is all, of course just my own opinion


        • R A Kennedy
          Posted Sep 12, 2012 at 11:24 AM | Permalink

          I agree with much of what you say, but journals don’t work that way. Perhaps they should, but they don’t. Unless the author agrees that the weaknesses are so severe that the paper should be withdrawn (and that seems a bit improbable with this chap), it will almost certainly find its way into the literature. But titles provide the route back to the paper and will be indexed. My point is, this title is not doing the journal any favours and an editor alerted to the issue might think it prudent to ask the authors to change it. The justification would be that it does not accurately reflect the content of the paper – I believe that is part of an editor’s duties and the editor has rights in that domain.

          The paper is quite extraordinarily weak (and I belive I am competent to make that judgement). But the only way of dealing with the issue now would be to submit a paper (or a “letter” – I don’t know whether Psychological Science accepts letters) pointing out the weaknesses and get that through the same peer review process. But somebody has to write it and frankly, why give the chap the publicity? He would no doubt enjoy writing his rebuttal since he clearly has a gift for the put-down. Rest assured he is not well known in psychology, although he may have a reputation in the green movement. So it is embarrassing for psychology down under (in the doldrums at the moment), and extremely embarrassing for the author, were he to know it. And it’s not doing the discipline of psychology any good to be used as a weapon by green activists. But if he wants to tough it out, he can. Although he might reflect on the possibility that his paper could become a “classic” in terms of how not to do things.

        • Posted Sep 12, 2012 at 2:40 PM | Permalink

          Re: R A Kennedy (Sep 12 11:24),

          Lewandowsky’s new response, and his promise of more, shows he clearly see nothing wrong and ain’t gonna change

          If I read it correctly its a great case of misdirection … the fake or suspect responses, especially the critical several regarding “Moon Landing” are not important according to him since, when taken in whole, the finding does not change for that category if you remove the two responses. While ignoring the huge affect those two fraudulent responses on the TITLE claim about skeptics and the moon landing.

        • Bob K.
          Posted Sep 11, 2012 at 12:35 PM | Permalink

          “Unknowingly speak sense” might be the only way that he will.

  4. Posted Sep 10, 2012 at 4:27 PM | Permalink | Reply

    As I mentioned on another thread, Lewandowsky deleted one of my comments entirely–it was up for a whole day and now it’s gone as if it never existed.

    • Steve McIntyre
      Posted Sep 10, 2012 at 4:36 PM | Permalink | Reply

      Re: thomaswfuller2 (Sep 10 16:27), better take snipping-tool pictures.

    • Skiphil
      Posted Sep 10, 2012 at 9:24 PM | Permalink | Reply

      Interesting…. SkS and Lewandowksy really are joined at the hip? Not a “conspiracy” theory, a matter of establishing evident facts. We know of much history of collaboration between SkS/John Cook and Stephan Lewandowksy, from the materials in the secret forum to the public “Debunking Handbook.” We know that Cook and Nuccitelli are blog authors at Lewandowsky’s place.

      Commentator Shub over at Bishop Hill points to some identical distinctive language used by mods at the Lewandowsky blog and at Skeptical Science to repudiate a poster’s complaint about blog moderation. Unless mods for Lewandowsky are into copying language from mods at other blogs it seems quite likely that SkS has taken over at least some of the moderating duties at Lewandowsky’s blog (as some had already speculated, and not surprising given the John Cook and Dana Nuccitelli are listed as blog authors at ShapingTomorrowsWorld):

      Shub at BH on identical language from mods at Lewandowsky blog and at SkS

      [emphasis added]

      at Lewandowsky’s “ShapingTomorrowsWorld”:

      Moderator Response: As an FYI, compliance with the Comments Policy of this site is non-negotiable; moderation policies are not open for discussion. If you find yourself incapable of abiding by these common set of rules that everyone else observes, then a change of venues is in the offing.

      at Skeptical Science:

      “…Finally, please understand that moderation policies are not open for discussion. If you find yourself incapable of abiding by these common set of rules that everyone else observes, then a change of venues is in the offing.”

      SkS is notorious for radical unwarranted snipping and deleting, changing post history, and generally being regarded as unreliable by those who are not slavish True Believers. Seems that we are seeing quite a similar approach at Lewandowsky’s ShapingTomorrowsWorld.

  5. sue
    Posted Sep 10, 2012 at 4:27 PM | Permalink | Reply

    His new post: http://shapingtomorrowsworld.org/lewandowskyGof4.html
    No mention of Junkscience posting the survey, but says SPPI was contacted.

  6. Anthony Watts
    Posted Sep 10, 2012 at 4:31 PM | Permalink | Reply

    Steve, now you’ve found his “tell”.

  7. Posted Sep 10, 2012 at 4:34 PM | Permalink | Reply

    Hi Steve.

    More censorship:

    Lewandowsky’s first article: Nasa and the Blogosphere

    I made the first comment, and I quoted Tom Curtis, and linked to Tom Curtis’c comment at Sceptical Science..
    This has now been censored:

    Tom Curtis: “Given the low number of “skeptical” respondents overall; these two scammed responses significantly affect the results regarding conspiracy theory ideation. Indeed, given the dubious interpretation of weakly agreed responses (see previous post), this paper has no data worth interpreting with regard to conspiracy theory ideation.

    It is my strong opinion that the paper should be have its publication delayed while undergoing a substantial rewrite. The rewrite should indicate explicitly why the responses regarding conspiracy theory ideation are in fact worthless, and concentrate solely on the result regarding free market beliefs (which has a strong enough a response to be salvageable). If this is not possible, it should simply be withdrawn.”



    This quote of Tom Curtis lasted at http://www.shapingtomorrowsworld.org for 6 days.

    This quote has now been removed saying accusations of fraud are not allowed, and too much extensive cutting and pasting.. (4 sentences!)
    they also removed the link to Skeptical Science, less any of their readers see it..

    I’ll send you before and after screen captures.
    Censored page: (i’m comment 1#)

    Tom did say (at Skeptical Sicence) he was under a lot of pressure, because of that quote…

    If any passing Austrlian citizens are reading this… ShapingTomorrowsworld.org is a PUBLICALLY funded blog with Grants from UWA

    Prof Stephan Lewandowsky (School of Psychology, University of Western Australia)
    Prof Steven Smith (ARC Center of Excellence in Plant Energy Biology, University of Western Australia)

    maybe someone should contact the editorial board (or Jo Nova ;-) )

    • Posted Sep 10, 2012 at 4:43 PM | Permalink | Reply

      This was also removed from comment 1#

      “Secondly, Tom Curtis at the ‘consensus blog’ – Skeptical Science’ one of the so called ‘pro-science’ blogs surveyed had this to say about the paper and its data. Perhaps a response from the authors is due”

      • Posted Sep 10, 2012 at 4:47 PM | Permalink | Reply

        I have had another comment removed completley (as if it was not there) I kept a copy of that as well

  8. Anthony Watts
    Posted Sep 10, 2012 at 4:35 PM | Permalink | Reply

    Steve, this comment you made today made it through, but he can’t allow the word “fake”:

    stevemcintyre at 07:16 AM on 11 September, 2012
    As I’ve mentioned at Climate Audit, the substantive issue with your paper is the reliance on (-Snip-) data in the survey.

    This is a substantive issue that I raise without “profanity or inflammatory tone.”

    An answer is overdue.

    Moderator Response: Please refrain from accusations of dishonesty (snipped) in this venue. This pertains to all parties.

    here’s the original:

  9. Posted Sep 10, 2012 at 4:40 PM | Permalink | Reply

    Hehe. I wonder how Lew’s discussion with his university’s ethics committee went.

  10. Robert E
    Posted Sep 10, 2012 at 4:46 PM | Permalink | Reply

    Steve, the link in the first sentence is missing.

  11. kakatoa
    Posted Sep 10, 2012 at 4:51 PM | Permalink | Reply

    I just finished reading “Over The Edge” by Jonathan Kellerman (1987). His thoughts on “Reaction formation Freud had termed it: the embrace of noble deeds in order to mask festering impulses.” come to mind when I review the process that was used to obtain the inputs for the paper in question.

  12. Steve McIntyre
    Posted Sep 10, 2012 at 5:09 PM | Permalink | Reply

    Lewandowsky has deleted another comment and issued a revocation warning.

    • HaroldW
      Posted Sep 10, 2012 at 5:27 PM | Permalink | Reply

      Perhaps you need to clarify that “fake” refers to *responders* feigning poll responses, rather than Lewandowsky et al. faking poll responses. L. seems to have a thin skin for an Aussie.

    • Bob Moss
      Posted Sep 10, 2012 at 8:04 PM | Permalink | Reply

      Perhaps Lewandowsky would approve “gleicked data”?

      He has in the past.

    • philjourdan
      Posted Sep 11, 2012 at 9:04 AM | Permalink | Reply

      You are exposing the vampires to sunlight. They will not tolerate it.

    • Shevva
      Posted Sep 12, 2012 at 5:21 AM | Permalink | Reply


      i)she got on the plane with a fake passport
      ii)a fake Cockney accent
      iii)a fake doctor

      And as you identify in your second comment left there ‘Fake Reponses’
      or to paraphrase:-
      iv)some of the responses to the survey where fake

  13. sue
    Posted Sep 10, 2012 at 5:13 PM | Permalink | Reply

    Steve, I can’t believe he just did that. His explanation contradicts what you had written.

  14. Posted Sep 10, 2012 at 5:18 PM | Permalink | Reply

    I believe that the concern here is over the term applied to palpably bogus data used to achieve desired results. ‘Fake’ could be construed as being somewhat pejorative and vulgar.

    Perhaps ‘Lewandowsky’ would be more diplomatic. For example, ‘…as compared to your use of ‘Lewandowsky’ responses.’ Presumably this synonym would not contravene posting protocol?

  15. William Larson
    Posted Sep 10, 2012 at 5:38 PM | Permalink | Reply

    I am confused: In the head post, Mr. McIntyre, you show your comment as posted at 7:41 a.m., then state that the comment didn’t last long. But the ensuing screen shot (showing your comment deleted) is comment no. 6 and was posted at 7:38, three minutes BEFORE the one (no. 7) that “didn’t last long”. Again, I am confused. Please enlighten.

  16. AndyL
    Posted Sep 10, 2012 at 5:40 PM | Permalink | Reply

    The word ‘fake’ seems to be as mis-understood by the recipient in this context as the term ‘upside-down’ in another.

    As you are posting in his ‘house’, why not use another term which might be acceptable there, such as ‘gamed responses’.

  17. Gerald Machnee
    Posted Sep 10, 2012 at 5:40 PM | Permalink | Reply

    He is not the only one to censor comments that are disagreeable to them, even when not inflammatory.

  18. Posted Sep 10, 2012 at 5:56 PM | Permalink | Reply

    I made another attempt … tried to be completely straightforward – doubt it’ll help:

    Mr. Moderator … before you ‘snip’ based simply on a word – I encourage you to actually read Mr. McIntyre’s detailed review of the data. The reference to “fake” data is not an ad-hominem attack, it is well documented in his review.

    This was also first pointed out by Tom Curtis at Skeptical science – certainly not a usual critic of someone like Mr. Lewandowsky – who says pretty much the same thing:

    Tom Curtis of Skeptical Science observed that at least “10 of the respondents have a significant probability of being produced by people attempting to scam the survey” and that the “paper has no data worth interpreting with regard to conspiracy theory ideation”.

    Steve McIntyre notes, as does Curtis, that the data in the paper includes these apparently “scam” or fraudulent responses.

    Whether you use the word “fake”, “scam” or fraudulent – they are all accurate descriptions of responses that are pretty clearly not valid survey responses, but rather attempts to manipulate and game the data. Attempts that are known to have been openly discussed in several of the pro-AGW sites. They should not have been included in the data, especially with so small a data set.

    And as Tom Curtis notes, without these clearly manipulative responses the survey as it stands has insufficient, essentially no, data to support its conclusions.

    You can snip all day but it won’t change the fact these are legitimate, fair criticisms that ethics and professional standards dictate Mr. Lewandowsky should be responding to.

    Mr. Lewandowsky, and various cohorts have been and are using the conclusions of this paper in a series of public articles and talks.

    Yet those conclusions have been shown to be unsupported, or at minimum highly questionable, by the data. Refusing to address legitimate, well documented, concerns while continuing to disseminate the conclusions is in my opinion nether ethical or professional.

    • Posted Sep 10, 2012 at 6:06 PM | Permalink | Reply

      Re: A.Scott (Sep 10 17:56),

      Didn’t take long:

      Moderator Response: As an FYI, compliance with the Comments Policy of this site is non-negotiable; moderation policies are not open for discussion. If you find yourself incapable of abiding by these common set of rules that everyone else observes, then a change of venues is in the offing.

      • Posted Sep 10, 2012 at 6:37 PM | Permalink | Reply

        Re: A.Scott (Sep 10 18:06),

        Made one more try – don’t know how one could be more straightforward and factual:

        Perhaps this will be more acceptable – please feel free to snip as you feel necessary:

        Mr. McIntyre’s question comes from his detailed review of the data. A read of his review shows it is not an ad-hominem attack, but rather a factually supported observation based on detailed analysis of the survey data. His conclusions and findings are well documented in his review.

        This was also first pointed out by Tom Curtis at Skeptical science – certainly not a usual critic – whose review of the data finds the same issue that the data shows a clear attempt to manipulate the survey data.

        Steve McIntyre notes, as does Curtis, that the data in the paper includes these manipulated responses. And that when this manipulated data is properly excluded, there is no longer factual support for the paper’s conclusions.

        Additionally, a number of sites have documented open discussion of this attempt to manipulate the survey results at several pro-AGW sites.

        One “skeptic” and one “pro-AGW” reviewer have come to same conclusions based on a detailed examination of the data. These appear to be legitimate, well documented, fair questions.

        As the paper’s conclusions are being widely disseminated publicly, if in fact there are legitimate issues with the data, asking for these issues to be addressed by the authors seems reasonable and appropriate.

        • Laurie Childs
          Posted Sep 10, 2012 at 7:37 PM | Permalink

          Shh! You mustn’t mention Steve’s “review”. You can mention Curtis’s, but you can’t mention Steve’s. Guess why? Cos it’s not peer-reviewed of course.

  19. MarkB
    Posted Sep 10, 2012 at 5:57 PM | Permalink | Reply

    ‘Brazen it out’ is a tried and true strategy among the ideologically inspired.

    I just plugged the word ‘sociopath’ in to Google, and found some interesting reading in the first hit. I am not a doctor, nor do I play one on TV.

  20. Hector M.
    Posted Sep 10, 2012 at 5:58 PM | Permalink | Reply

    I posted a very mild and respectful comment in L.’s blog, to the effect that Steve was apparently not accusing him of “faking data”, but rather arguing that (possibly inadvertently) some “fake responses” may have not been excluded from the dataset, and asking for a comment on that issue.
    My comment survived for about half an hour before mysteriously disappearing, leaving (alas) no trace of its brief existence. The issue itself remains unaddressed.

    • Skiphil
      Posted Sep 10, 2012 at 6:10 PM | Permalink | Reply

      The “disappearing” of your comment suggests that Lewandowsky et al are blindly afraid of any open and rational discussion. One cannot be sure who is doing all of the snipping and deleting there but since there are only two principals listed and the other is a biologist who has not shown any interest in SL’s nonsense, it’s likely that Lewandowsky himself is frantically manning his blog ramparts.

      • Laurie Childs
        Posted Sep 10, 2012 at 6:35 PM | Permalink | Reply

        I commented somewhere yesterday that the sheer number of comments (unprecedented for that site)was overwhelming the moderator(s) there. John Cook himself posts articles to that site and I surmised it was only a matter of time before SkS sent in the cavalry to help. Judging by the way comments below Lewandowsky’s previous posts on this matter are now being revisited and snipped/disappeared, it’s my guess that moderator privileges have now been granted to one or two of the SkS mods. The style is uncannily similar. The message “comments are now audited” which keeps on being posted, leads me to assume that someone is now moderating around the clock.

    • Cpt. Beefheart
      Posted Sep 10, 2012 at 6:15 PM | Permalink | Reply

      I was able to get a snap of your comment during its short life, Hector.

      • Brent Buckner
        Posted Sep 10, 2012 at 6:33 PM | Permalink | Reply

        Steve, you might consider updating your post with Cpt. Beefheart’s link to Hector M.’s comment. It is a great illustration of What Must Not Be Discussed over there.

        Steve: done.

      • Jeff Alberts
        Posted Sep 10, 2012 at 8:53 PM | Permalink | Reply

        Wow, could you have picked a place to post the image that didn’t pop up sex ads? Imageshack maybe?

        • Cpt. Beefheart
          Posted Sep 10, 2012 at 9:27 PM | Permalink

          Hmm, I’m not getting any pop-ups. I even checked a ‘family safe’ box before uploading. If it is a problem for others, steve can delete.

        • Mark T
          Posted Sep 11, 2012 at 9:49 PM | Permalink

          Targeted marketing, Jeff… ahem. ;)


      • Jan
        Posted Sep 11, 2012 at 5:57 PM | Permalink | Reply

        Yes, the wind energy is just the first step. Close that window and jasmine opens in the background.

      • Cpt. Beefheart
        Posted Sep 11, 2012 at 6:00 PM | Permalink | Reply

        Ugh, sorry folks. That’s what I get for being in a rush and picking an image host from About.com (imagevenue is ranked #2 on their list). I hosted a photo at (hopefully) a better place. Steve or Roman, please delete the old link, thanks.


        • Skip Smith
          Posted Sep 12, 2012 at 2:11 AM | Permalink

          Your post has been completely deleted.

      • dougieh
        Posted Sep 11, 2012 at 7:47 PM | Permalink | Reply

        no problem for me neither
        keep the spotlight on kid :-)

  21. daved46
    Posted Sep 10, 2012 at 6:05 PM | Permalink | Reply

    This pertains to all parties.

    So if people who we have no idea who are, appear to have faked, gamed, whatever the data, and this is pointed out, then this is a problem in what way? This fellow has a severe problem in communicating with people who don’t agree with him. Indeed, this seems to be endemic in the AGW world.

    • daved46
      Posted Sep 10, 2012 at 6:29 PM | Permalink | Reply

      Re: daved46 (Sep 10 18:05),

      Hmmm… I reread my reply and I realize that actually L. is reacting consistently. Since the apparent charge is that the presence of a few “bad apples” in the skeptic barrel makes all skeptics bad, if it appears there are fetid oranges in the anti-skeptic basket, this would require assuming all anti-skeptics are fetid (and he would thus be within his rights to consider the existence of fetid, though unidentified, anti-skeptics to reflect on his own ideas. The concept is logically flawed, but it is consistently applied.

  22. Timothy Sorenson
    Posted Sep 10, 2012 at 6:06 PM | Permalink | Reply

    They are under the gun and feeling pressured. I am sure that there exist well-defined terms in the social sciences for answers provided on surveys that are clearly disingenuous or intentionally designed to deceive the surveyors. The use of ‘fake data’ can EASILY lead others to believe that the surveyors faked the data rather than the respondents. I can see that they would not want to leave such a statements up to miss lead others. I can see someone picking up that term and a headline reading ‘Aussie researchers use faked data!” The implication and connotation is upon the researcher.

    Using a less ambiguous wording like: would be far nicer and I suspect more palatable as well as more precise and less prone to misinterpretation.

    • TerryS
      Posted Sep 10, 2012 at 6:47 PM | Permalink | Reply

      The problem with Lewandowsky’s response is that he is now accusing Steve of unjustly saying he personally faked data and Steve never did this.
      He had other options such as a moderator type comment clarifying any ambiguity he perceived in Steve’s comment. Alternatively he could have responded to Steve’s question.

    • Posted Sep 10, 2012 at 10:35 PM | Permalink | Reply

      I can see that they would not want to leave such a statements up to miss lead others. I can see someone picking up that term and a headline reading ‘Aussie researchers use faked data!” The implication and connotation is upon the researcher.

      I think that if Lewandowsky had made the slightest effort to respond to the legitimate questions that had been asked of him instead of diverting (by pretending that emails from Hanich were from him), you might have a valid point … although it does suggest that his primary blog audience may be deficient in critical thinking skills.

      But, IMHO, one who chooses to deliberately and misleadingly call a paper, “NASA faked the moon landing, Therefore (Climate) Science is a Hoax: An Anatomy of the Motivated Rejection of Science” when, in fact, such a link is clearly not supported by his data should get down off his high-horse and take a long look in the mirror.

      It would certainly serve him far better than his childish and churlish exercises in further avoidance by uh … redefining “conspiracy”, “inflammatory”, “accusations of dishonesty/impropriety” etc.

      In fact, had Lewandowsky actually taken the time to review Steve’s post outlining the problems with his methodology, it would have been perfectly obvious to him that it was (as Steve had explained) the responses that were obviously faked.

      Alas, it seems that accuracy and clarity are not part of Lewandowsky’s “communication” toolbox. Alternatively, or perhaps in addition, he may just be profoundly challenged in the reading comprehension department.

    • Jonas N
      Posted Sep 13, 2012 at 6:24 PM | Permalink | Reply

      Timothy S

      These are all reasonable points and explanations why one wouldn’t want to leave have a discussion about ‘faked data’ or ‘faked answers’ on one’s personal blog. Or for a media outlet would pick up on that.

      But you also need to remember that the whole Lew-Endeavour was exactly about that:

      For the media and wider (unknowing, gullible) circles to pick up a grossly misleading (essentially false) premise (the title) and smearing message under the disguise of it been proper science and reinforce such beliefs among the already faithful.

      And that there is and was a dire desire for having such predjudice ‘confirmed’ (ie reinforced) is easily established by just reading the comments (and postings) at those allegdedly ‘pro-science’ blogs.

      Just let me point out, that these latter points are equally good reasons for why one rather wouldn’t leave a discussion about the ‘faked data’ up there …

  23. tlitb1
    Posted Sep 10, 2012 at 6:14 PM | Permalink | Reply

    I would say adamantly putting to L “your use of faked responses” left him feeling no room for maneuver and would be quite un-nerving to him right now. I think he would respond to a more light approach?

  24. Posted Sep 10, 2012 at 6:25 PM | Permalink | Reply

    A Scott

    That’s too funny.

    You are being ‘moderated’ by the invisible hand of Skepticalscience. In all likelihood.

    This is standard boilerplate moderation text from one of Skeptioalscience’s moderators. See the similarity?

    Please note that posting comments here at SkS is a privilege, not a right. This privilege can and will be rescinded if the posting individual continues to treat adherence to the Comments Policy as optional, rather than the mandatory condition of participating in this online forum.

    Moderating this site is a tiresome chore, particularly when commentators repeatedly submit offensive or off-topic posts. We really appreciate people’s cooperation in abiding by the Comments Policy, which is largely responsible for the quality of this site.

    Finally, please understand that moderation policies are not open for discussion. If you find yourself incapable of abiding by these common set of rules that everyone else observes, then a change of venues is in the offing.

    Please take the time to review the policy and ensure future comments are in full compliance with it as no further warnings will be issued.

    The site has numerous variations of the same Orwell-speak text, all over the website. The censorship is an end, and not just a means.

    • DaveA
      Posted Sep 10, 2012 at 10:45 PM | Permalink | Reply

      Yep. Top is SkepticalScience, bottom is ShapingTomorrowsWorld.

    • Bob Koss
      Posted Sep 11, 2012 at 1:18 AM | Permalink | Reply

      Shub has a good eye picking up the moderation similarities. I checked registrar data and John Cook of skepticalscience is the admin. and tech. for Lewandowsky’s site. Cook is also the only email.

      Here are a few lines from Network Solutions.
      Registrant ID:DIT-1986046
      Registrant Name:Stephan Lewandowsky
      Registrant Organization:UWA
      Registrant Email:john@skepticalscience.com
      Admin ID:DIT-1986048
      Admin Name:John Cook
      Admin Organization:Sevloid Art
      Admin Email:john@skepticalscience.com
      Tech ID:DIT-1986047
      Tech Name:John Cook
      Tech Organization:Sevloid Art

  25. Posted Sep 10, 2012 at 6:33 PM | Permalink | Reply

    Steve’s title Lewandowsky Censors Discussion of Fake Data is fair. So is ‘fake data’ in comments on the man’s blog – the clearest description of the most important issue with the data.

    The self-appointed climate establishment is happy to call anyone who does not agree with all of its political program deniers, an abuse of language that, for all we know, from history, could be a prelude to far worse persecution. It now insists on a form of politeness which obscures the main issue with their latest blatant attempt to smear us.

    It’s hard to know what to do with such people. Pray that governments take away their influence for ever. That has to be the best bet.

  26. Posted Sep 10, 2012 at 6:34 PM | Permalink | Reply

    I wonder how long it will take before L. recognizes the folly of his hyper-defensive ways and resorts to yet another post in which – in the fashion of true “revisionist” scholarship (cf his attempt to claim that emails from his assistant were from him) – he posts excerpts from the excised/edited comments in order to “prove” that he didn’t do that which the evidence clearly shows he did!

    Amazing. Simply amazing.

  27. Posted Sep 10, 2012 at 6:56 PM | Permalink | Reply

    Reblogged this on Climate Ponderings.

  28. thefordprefect
    Posted Sep 10, 2012 at 7:01 PM | Permalink | Reply

    What a 2 faced post this is. There are just so many people claiming that censorship is rife on WUWT Talkshop, and CA – me included. And here you all are complaining that posts are being snipped by someone else.

    Your accolytes then continue trying to goad the site to get another rejected post they can complain about.

    I doubt this post will get through moderation so I may post on another blog.


    • TerryMN
      Posted Sep 10, 2012 at 8:31 PM | Permalink | Reply

      TFP – My question remains why the question with the least amount of either Agree or Strongly Agree answers (“faked the moon landing so…”) became the title of the paper. Do you think that the survey showed this? Thanks.

    • charles the moderator
      Posted Sep 10, 2012 at 9:41 PM | Permalink | Reply

      Re: thefordprefect (Sep 10 19:01), I call BS. Nothing like this has ever happened on WUWT. Occasionally a moderator oversteps their bounds or a taunting commenter is banned, but nothing like this full court press avoidance of uncomfortable questions. Nothing like this. And I have personally let dozens if not hundreds of your comments through back when I was moderating.

      • Posted Sep 11, 2012 at 6:27 AM | Permalink | Reply

        I was banned. My IP address was banned. he now bans anonymous ip addresses. All to stop posts which in all honesty do not deserve the bin. He also stifles posting by threatening to expose the posters posting during work time etc.

        As McIntyre snips posts long after they get posted (thus destroying the time line and leaving orphaned comment) and is following on the lines of Watts wer could discuss this on http://climateandstuff.wordpress.com/

        I am very surprised that the former post got through – the first for nearly a year.

      • Louise
        Posted Sep 11, 2012 at 7:31 AM | Permalink | Reply

        charles the moderator – WUWT also allows favoured posters to act as moderators under different pseudonyms. How’s that for ethical?

        Smokey, an exceptionally aggresive commenter on WUWT is also moderator dbs on that same site.

        I wonder how many other sockpuppets there are there that are on Anthony’s ‘side’ of the fence that are used to boost skeptic numbers?

        • a dood
          Posted Sep 12, 2012 at 2:48 PM | Permalink

          Hah, nice misdirection attempt! It seems the standard at Watt’s site for moderators to use their initials rather than post their full names when adding moderator’s comments. I wouldn’t call that sockpuppetry – just an attempt at being unobtrusive in the comments.

          Besides, I don’t think Anthony is hurting for genuine supporters. When one of his moderators passed a few weeks ago (Robert Phelan, aka REP) there was an outpouring of support. Almost $20,000 raised to help with funeral expenses. All from small contributions. Even Nick Stokes chipped in :)

        • Posted Sep 12, 2012 at 5:53 PM | Permalink

          Re: Louise (Sep 11 07:31),


  29. Bebben
    Posted Sep 10, 2012 at 7:06 PM | Permalink | Reply

    I would like to read the comments on Lewandowsky’s newest post but for some reason I’m unable to view them.

    Meanwhile, I try to read a bit “word for word” as Steve suggested, and the first thing I see is this:

    “Shortly thereafter, the first of the 5 bloggers, Mr McIntyre, found his misplaced email.”

    It’s “his” misplaced email. Of course, not “my” “misplaced” (misplaced how/where/why?) email – because he didn’t sent any, nor was there any email carrying his name. So therefore, he cannot say “my” email – which would have been the most natural thing to say (“Hi steve, didn’t you see my email?”).

    So it’s the pea and the thimble game again, but the professor do not seem too apt at it when even poor me can spot it.:)

    • Skiphil
      Posted Sep 10, 2012 at 8:00 PM | Permalink | Reply

      unreal — Lewandowsky runs an egregiously bad survey and now wants to blame bloggers for not remembering and locating one or two spammy emails out of thousands from two years ago??

      “Dear Mr. Lewandowsky, we write in regard to a spam email we sent you some two years ago…… what?? You’ve ‘misplaced’ it?? You did not realize it had a link which would enable you to obtain 30% of the gold fortune of the late great Minister of Mines in Nigeria? For shame, Mr. Lewandowsky, for shame!”

  30. Nullius in Verba
    Posted Sep 10, 2012 at 7:25 PM | Permalink | Reply

    Is accusing an entire group of being conspiracy theorists classified as ‘inflammatory’ or an ‘accusation of dishonesty/impropriety’ do you think? Is accusing someone of making such accusations an accusation of impropriety?

  31. Posted Sep 10, 2012 at 7:50 PM | Permalink | Reply

    I can’t help wondering if this is the real objective of the study. What the reaction is to what has happened… It really can’t be the paper–the survey it’s based on is crap and the paper goes out of its way to hide the weaknesses.

    Is it possible Lewandowsky is documenting all of this for a real paper? I’m only being semi-facetious here. That survey is so bad that there must be a real reason for all this.

    • charles the moderator
      Posted Sep 10, 2012 at 9:55 PM | Permalink | Reply

      Re: thomaswfuller2 (Sep 10 19:50), Never assume malice when stupidity will explain it.

    • Posted Sep 11, 2012 at 12:48 AM | Permalink | Reply

      That would bea great a twist ending.

      Some of the best “natural” behavior in a controlled study can be seen through having the subject report to a “reception area” where he has to allegedly wait for the current subject to finish the multiple choice test.

      Unknown to the subject, the events that are about to transpire while he “waits” is the acual test. The multiple choice, then, just provide demographic data.

    • Jeff Norman
      Posted Sep 11, 2012 at 11:43 AM | Permalink | Reply

      I suggested that this was a set up for a real experiment at Lucia’s days ago.

    • Posted Sep 11, 2012 at 7:51 PM | Permalink | Reply

      Is it possible Lewandowsky is documenting all of this for a real paper? I’m only being semi-facetious here. That survey is so bad that there must be a real reason for all this.

      Tom, I see that you actually posted this theory on his blog and that (as of today 04:05 PM PDT, at least) none of the snippers or snipers have touched it. Nor has anyone responded to your comment!

      It might explain why two years have lapsed twixt “research” and “publication” – not to mention the conveniently planted “revisionisms” in his posts on this.

      Nonetheless, if this is his intent, it strikes me that it would be highly unethical … certainly far more so than any disclosure of the names of the five “skeptic or sceptic-leaning” blogs that “were approached” but did not “[post] the link”.

      Accuracy and clarity are certainly not characteristics that spring to mind when one hears the name Lewandowsky, I agree. But your “theory” certainly fits with his Sept. 3 verbal excursion back to the future; he began this post by noting:

      I recently published a paper on the motivated rejection of science that is forthcoming in Psychological Science.

      How can one have “recently published” something that “is forthcoming”?! And why does he not specify the issue of the journal in which his opus will be found?

      Sloppy thinking? Careless writing? Or deliberately chosen “fog” and “cloud cover”?!

      If such a paper ever does materialize as you have theorized, I will be very surprised if he includes any of the many valid criticisms of this (as yet unpublished) faulty work. Given his past practice, I suspect he’s more likely to highlight the responses of his “defenders” (and/or those who’ve quoted him in their discussion of this travesty)!

      Hilary (thanking her lucky stars that she had the benefit of studying Psych and English back in the days of pre-post modernist education)

      P.S. While trying to find his paper in Psychological Science I came across one that L. seems to have missed: “False-Positive Psychology – Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant”. Although I believe it has been mentioned in one of the many threads on this issue, it’s not behind a paywall and is worth a read http://pss.sagepub.com/content/22/11/1359.full.pdf+html

      • Posted Sep 11, 2012 at 7:55 PM | Permalink | Reply

        Sorry, I should have noted “my bold” in the quote of L’s excursion back to the future.

  32. Posted Sep 10, 2012 at 8:14 PM | Permalink | Reply

    Where was this efficiency when collecting and interpreting the survey data? Seems to have it backwards, it is supposed to be diligent on the data collection, open minded on the discussion.

  33. Tom C
    Posted Sep 10, 2012 at 8:40 PM | Permalink | Reply

    What a fool “moderator” is to snip clearly legit comments from a guy who runs a widely read blog. Apparently he didn’t realize the truth would soon get out.

    • Posted Sep 11, 2012 at 6:46 AM | Permalink | Reply

      The skepticalscience moderation policy specializes in insulting prominent bloggers by butchering their comments just to make the point that there is a ‘policy’ in place they have violated. You can see an example of the tactic applied to scientist Roger Pielke Sr here. It is a shibboleth, signature move.

  34. Skiphil
    Posted Sep 10, 2012 at 9:25 PM | Permalink | Reply

    hmmm my comment may be lost in moderation due to a link… well there is a useful comment over at Bishop Hill from Shub, noticing identical language from mods at SkS and at Lewandowsky’s blog.

  35. Ed Snack
    Posted Sep 10, 2012 at 9:28 PM | Permalink | Reply

    Just to be quite c.lear, all this fuss is over an Internet survey ? I mean, where has Lewandowsky been the past 20 odd years, under a rock ? It’s like he’s never read or heard anything about self selecting surveys and the reliability thereof.

    Surely, this just has to be some sort of elaborate joke ?

    • Posted Sep 11, 2012 at 12:04 AM | Permalink | Reply

      Possibly Lewandowsky and coworker(s) are attempting to demonstrate the ‘Familiarity Backfire’ effect, which is explained here:


      The basic premise of familiarity backfire theory (FBT) is that myth refutation must be followed by myth insertion…here by pointing out that ‘skeptic’ responses were bogus (myth refutation), Steve has created a painful ‘myth gap’ (see diagram in article linked above, showing the location of the gap). The myth gap, in turn, leads to comment deletion (etc.). It is all pretty basic cognitive science really.

  36. R A Kennedy
    Posted Sep 11, 2012 at 12:56 AM | Permalink | Reply

    It is worth noting that Lewandowsky et al were aware the survey was being gamed. They discuss the exclusion of cases where the ages particiants gave were implausible (less than 10 or greater than 95). This category was lumped in with other exclusions so it’s impossible to know how many there were. But if it was more than a trivial number the outcome was a serious red flag for the whole enterprise.

    For those who may not know, Psychological Science is a distingished journal with good standards. I would hope that the editor at least asks for the title (which is tendentious) to be changed.

  37. Steven Mosher
    Posted Sep 11, 2012 at 2:35 AM | Permalink | Reply

    for christ sakes use the the phrase “suspect reponses” and dont turn this into yet another fight about moderation.

    HIS ENTIRE GAME is to goad everybody into fights about tangents.

    • bernie1815
      Posted Sep 11, 2012 at 3:40 AM | Permalink | Reply

      I agree. While accurate, “fake data” does seem to be unnecessarily incendiary and gives Lewandowsky cover for essentially ignoring Steve’s valid points.

      • Kingb
        Posted Sep 11, 2012 at 3:37 PM | Permalink | Reply

        Must agree as well. “Fake” has strong ethical implications and while it might correctly reflect the data’s pedigree since it is impossible to prove categorically that the data is “fake” alternative language is warranted. In my field we would refer to the data as being “questionable”, “potentially invalid” or “unreliable”. Steven’s suggestion “suspect responses” also gets the idea across without any nastiness and would allow us to get back to the issues with the methodology which is where the debate should concentrate.

      • Posted Sep 11, 2012 at 9:36 PM | Permalink | Reply

        I also agree. From what little I’ve read of his blog, he seems to enjoy trolling for a reaction, and displays little (if any) desire to engage critics of his paper in a meaningful way. I don’t think this guy is worth wasting any more time over … he’s having too much fun playing the partisan.

    • Posted Sep 11, 2012 at 4:28 AM | Permalink | Reply

      It hasn’t turned into another fight about moderation because the game Lewandowsky has been playing since 2010 is bigger. He’s using moderation to try and shore up the belief among the activiscenti around him that McIntyre is an unstable extremist. People only have to come over here to learn different. Those that are not willing to do that are probably a lost cause. This for me in the original thread is fair comment:

      Lewandowsky, like Gleick, probably fancies himself a hero of the Cause. But ironically, Lewandowsky’s paper will stand only as a landmark of junk science – fake results from faked responses.

      I say stick with the original terminology.

    • Willis Eschenbach
      Posted Sep 11, 2012 at 3:09 PM | Permalink | Reply

      Steven Mosher
      Posted Sep 11, 2012 at 2:35 AM

      for christ sakes use the the phrase “suspect reponses” and dont turn this into yet another fight about moderation.

      HIS ENTIRE GAME is to goad everybody into fights about tangents.

      Steven, your point might hold water if you had actually, you know, tested it … hows about you write a comment using your preferred term (“suspect responses” in lieu of “fake responses”) and see what happens before you start proposing solutions that may not work?

      If you think changing the words to “suspect responses” will make a difference, then it is your responsibility to go and see if it actually does make a difference before accusing other folks of doing things wrong and insisting that FOR CHRIST SAKE people use the (totally untested) method you obviously prefer …


      • Tony Mach
        Posted Sep 12, 2012 at 2:10 PM | Permalink | Reply

        Willis, now they will read anything as an euphemism for “faked” – the experiment is tainted.

        A little more clarity and a little more courtesy in the first place (e.g. something like ” … it appears to me that a small number of fraudulent participants have contaminated the survey’s results … “) would have done wonder, IMHO. It would have been much harder to use Steve’s words against him.

        I personally think their behavior does not warrant a courteous response, but we should always try to take the high road and don’t give them any ammunition they’ll use against us.

        • Willis Eschenbach
          Posted Sep 12, 2012 at 5:51 PM | Permalink

          Tony Mach
          Posted Sep 12, 2012 at 2:10 PM

          Willis, now they will read anything as an euphemism for “faked” – the experiment is tainted.

          What do you think we are playing here, “Mother May I”? They are making up excuses to ignore and censor ideas that they don’t like, and you want to sit here and make excuses for their making excuses, make excuses for their censorship?

          How about you let them make up their own excuses for their unscientific actions? Do you really think they need help cooking up reasons to ignore and censor people who are commenting on the trenchant stupidity of their study?

          The truth is, some of the responses were in fact obviously and clearly fake. False. Phony. Made up. Forged. Counterfeit. Fraudulent. Bogus. Fictitious. Suspect. Feigned. Spurious. Deceptive. Pick the word of your choice, the word won’t be pretty, because the reality is not pretty. Lewandowsky got suckered, he was conned in an absurdly simplistic fashion, now he’s censoring comment on the subject, and there’s no easy way to phrase that.

          So you and Lewandowsky might as well get over it and stop with the excuses. There’s no nice, politically correct way to tell Lewandowsky that he’s too dumb to recognize a faked response, too naive to realize that he needs to do just that, and too foolish to notice that censorship will not help his case. That’s big-time, major-league, no-holds-barred stupidity, even Pangloss couldn’t sugar-coat that.


        • Posted Sep 12, 2012 at 11:01 PM | Permalink

          While enjoying that, Willis, and fully agreeing with your defence of the terms Steve has used, I’d add that we can’t know for sure that Lewandowsky is that stupid. There is at least a finite probability that he knows the truth but doesn’t care to admit it. And that he has thus dissembled since at least August 2010. We just found out something similar about three senior people in south Yorkshire since April 1989, a fact that caused needless suffering to the relatives of victims of one of our worst football disasters. Never rely on that old aphorism that what you can explain by stupidity is automatically explained thereby. Sometimes it’s something far worse.

      • Tony Mach
        Posted Sep 12, 2012 at 2:11 PM | Permalink | Reply

        Or even better: “… significantly contaminated the survey’s results … “

    • Posted Sep 11, 2012 at 4:31 PM | Permalink | Reply

      Re: Steven Mosher (Sep 11 02:35),

      Steven … I tried that (see above)used “manipulated responses” … snipped

    • Robert
      Posted Sep 11, 2012 at 11:44 PM | Permalink | Reply

      Agree with Mosher – using the term “Fake” just gives an out. One frankly i’d probably take myself – it does imply things that would certainly irk me if I were on the other side. It’d be nice to for once have a blog discussion back and forth without perpetual snark from both sides

  38. Geoff Sherrington
    Posted Sep 11, 2012 at 3:18 AM | Permalink | Reply

    “So I knew that giving full and open disclosure generates trust in the minds of the passengers. Everyone feels vulnerable in these situations; they just want to be given the truth and be treated with respect.”
    This passage is from p.299 of “QF32″, the book by Australian Qantas airline captain Richard de Crespigny. He had just spent several hours gaining control of, then safely landing, a massively damaged Airbus A380. Yet, for nearly an hour more, he debriefed the bulk of the passengers in the safety of Singapore airport terminal. He several times repeated his private telephone number and invited calls in the time ahead if passengers thought he could fix or help fix enduring problems.
    Not one of the 469 people on board had suffered any physical injury.
    Some Australians are capable of extraordinary achievements and we love them for it. The 35 words quoted above gain enormous respect.
    The discomfort felt by a retired senior Australian scientist like me should not arise in this matter of the University of West Australia. It would not arise if the spirit those 35 words was applied. A person with the rank of Professor should lead and teach, as the Captain did. We Australians need to see acts that increase respect for climate and social science.
    Professor Lewandowsky, please contrast your conduct with that of Richard de Crespigny. You are being assesed by wise people, as was he at the time. He enhanced his reputation enormously by his conduct. The path is simple, the example is there.

    • Nicholas
      Posted Sep 11, 2012 at 4:41 AM | Permalink | Reply

      First Gergis and now this.. it’s a national embarassment that brings shame upon us. I feel especially bad since I bet there are thousands of great scientists who make contributions and are not afraid to discuss and debate but this is how the world has to see us?

  39. pdxrod
    Posted Sep 11, 2012 at 4:56 AM | Permalink | Reply

    “As well as being fraudulent, Gleick’s forgery intended to dehumanize his target, making the target seem less fully human than members of the in group”.

    Right. But that’s as true of the term ‘holocaust denier’ as it is of warm-mongers’ attitudes to climate skeptics. But you won’t find climate skeptics saying that!

    Steve demolishes Lewandowsky’s methods. But suppose a VALID survey found a correlation between AGW skepticism and 9/11 conspiracy theories. Or holocaust denial. It would prove nothing. It’s more important to criticize the implication of smear-by-theory-correlation as to criticize a particularly incompetent example of it.

  40. Posted Sep 11, 2012 at 7:39 AM | Permalink | Reply

    The expanding “Lewandowsky incident” has now reached to the Blaze, where Anthony and Lucia were mentioned. Has anyone yet noticed that the extremists on one side only, regularly censor accurate critique?

    Real Climate
    Deep Climate

    The list is endless actually. I bet I can list 20 censor heavy blogs without difficulty and they are all politically and environmentally oriented in the same direction. What they don’t seem to understand is that Steve has as much readership as he likes to call attention to whatever stupid behaviors they choose to exhibit. So Lewandowsky’s little blog snips a comment and it makes prime time as a CA post. Well done Dr. L. Great way to hide the problem.

    The main-stream media does read climate blogs too. Even if they are politically and environmentally on the same side, they recognize there is a risk in associating with someone exhibiting irrational behavior.

  41. Matt Skaggs
    Posted Sep 11, 2012 at 8:51 AM | Permalink | Reply

    “Is it possible Lewandowsky is documenting all of this for a real paper? I’m only being semi-facetious here. That survey is so bad that there must be a real reason for all this.”

    I thought about that as well. The discovery that Climate Depot was solicited makes the simpler explanation a bit stronger: Lewandowsky attempted a very simple-minded PR stunt to conflate the readers of Climate Audit with the readers of Climate Depot, which are actually at opposite ends of a spectrum. The irony is that in terms of aggregate ability to discern logic structures amidst complexity, the crowd at Realclimate would fit squarely in the middle.

  42. Craig Loehle
    Posted Sep 11, 2012 at 2:02 PM | Permalink | Reply

    attempted post vanished…
    It is ironic that lew sees conspiracy theorists, because what I see is incredible gullibility: believing in 20 or 100 ft sea level rise, believing it is ok to flip Tiljander sediments, believing wind power won’t destabilize the grid, believing that an ice sheet the size of Antarctica can “slide” into the ocean, and believing like Hansen that it is a viable solution to simply shut down all coal plants.
    And now we have believing that denizens of cagw activist sites would not try to spoof a survey. Why would any sceptics even be hanging out at those sites to post a response?

  43. Espen
    Posted Sep 11, 2012 at 2:14 PM | Permalink | Reply

    Regarding the censored post in your update: How dishonest to label “does not imply dishonesty” as an “accusation of dishonesty”! Or maybe the moderator simply doesn’t understand the most basic logic?

  44. John from CA
    Posted Sep 11, 2012 at 3:26 PM | Permalink | Reply

    Lewandowsky was warned his survey was no good
    By Andrew Bolt


  45. Posted Sep 11, 2012 at 4:38 PM | Permalink | Reply

    So much for ‘ethics’

    Prof Lewandowsky’s website/blog (‘ownwed’ by John Cook, Who-Is) is tweeting about outing the 4 bloggers that did not post his survey

    New STW post: Amnesiagate. The 4 bloggers who didn’t post that survey outed: http://bit.ly/PXh5nl

    Give that Lewandowsky claims he had provided search keys like – via interview on Desmogblog!

    “One stated “Thanks. I will take a look” and another asked “Can you tell me a bit more about the study and the research design?”
    Perhaps an inbox search for these phrases might help some bloggers to move on from their latest conspiracy theory.”

    Desmogblog (http://s.tt/1mwBG)

    Just shows how odd he is being…

    He even implies his own ‘conspiracy’ about why the 4 denied contact..

    “2. Why would the people who were contacted publically fail to acknowledge this fact?
    Several hypotheses could be entertained but I prefer to settle for the simplest explanation.

    It’s called “human error.” It simply means the 4 bloggers couldn’t find the email, didn’t know what to search for, or their inboxes were corrupted by a move into another building, to name but a few possibilities.

    The only fly in the ointment in that hypothesis is that I provided search keys and exact dates and times of some correspondence.”

    Now if he had supplied the search key of the subject of the email sent, or even the NAME of the sender – Hanich.. perhaps it would have been the tiniest bit more helpful.. Given that sceptics were looking for the name Lewandowsky, or even the co-authors (Hanich not one of them)
    The man is clearly playing odd games.

    Lewandowsky stated:
    “It will be noted that all 4 have publically stated during the last few days/weeks that they were not contacted.”

    Robert Pielke Jnr said this when Jo Nova emailed him to ask if ‘Lewandowsky’ had contected him:
    “Hi Joanne- Never heard of the guy, and a search of my email finds no contact from him. Hope this helps”

    As Robert doesn’t consider himself a ‘sceptic’ I’m sure he was surprised to discover someone caled Hanich, had sent him an email, he only searched again, because Mcintyre had found it.

    Lucia states: “Roger’ is not stating publicly that he was not contacted. It’s responding to Joanne’s question and truthfully reporting that he had never hear of the guy and reporting that a search of the email find no contact from him. And guess what: Roger was never contacted by Lewandowsky and no search for Lewandowsky would have permitted Roger to find the email. Roger was contacted by Hanich. The evidence shows Roger was entirely truthful, and went above and beyond the call of duty in answering Joanne’s questions.

    I do hope concerns about the methodology and the paper are being fed back to the journal.

    If I were the journal, the behaviour of Prof Lewandosky would also be of a concern. I will probably send them an email about this needlessly atagonistic behaviour by Prof Lewandowsky

    The feedback page for the journal is here:

    It is entirely possible that the journal is entirely oblivious, about concerns about this paper methodology, it’s title, and the interpretation of data, with clealry ‘supicious responses’ from blog readers that are antagonistic to ‘sceptics’

    Perhaps they could ask the peer reviewer if they downloaded the data and looked at it carefully, or add an extra peer reviewer (with experience in data collection online surveys) to clarify the methodology. The journal has its own reputation to protect, and could stall publication for a bit, just to check Tom Curtis’ concerns…

    If the journal is only told about ‘one side’ of the vigorous online debate, if they are even aware of this debate, will they check?

    • GPhill
      Posted Sep 12, 2012 at 3:05 AM | Permalink | Reply

      Quickly scanning the various preambles to the initial questionnaires, which obviously provide a lack of appropriate detail for participants; the implied threat to “name” contacts, and the totally unacceptable and demeaning language of Lewandosky, I would direct readers to the following information

      “If you have any questions or comments about this research you may address them to the experimenter, Charles Hanich, at hanich@cyllene.uwa.edu.au.

      The Human Research Ethics Committee at the University of Western Australia requires that all participants are informed that, if they have any complaint regarding the manner in which a research project is conducted, it may be given to the researcher or, alternatively to the Secretary, Human Research Ethics Committee, Registrar’s Office, University of Western Australia, 35 Stirling Highway, Crawley, WA 6009 (telephone number +61 8 6488-3703).”

      Hanich is currently listed as Laboratory Manager for Lewandosky’s department at UniWA. If the researchers involved (Hanich/Lewandosky) refuse to acknowledge concerns associated with the project, as is the case here, but moreover indulge in disparaging comments towards people making those same concerns, I would have no hesitation in demanding that the University Ethics Committee act immediately, as this is the key body that oversees and approves all research studies. Ethics Committee’s today are (and have to be) very cognisant of protecting the rights of study participants and the integrity of a University research processes.

  46. Rufus
    Posted Sep 11, 2012 at 5:51 PM | Permalink | Reply

    for people interested in the general topic of inaccuracies in studies and approaches to solving these problems, I highly recommend this recent podcast.


    “Nosek argues that these incentives create a subconscious bias toward making research decisions in favor of novel results that may not be true, particularly in empirical and experimental work in the social sciences. In the second half of the conversation, Nosek details some practical innovations occurring in the field of psychology, to replicate established results and to publicize unpublished results that are not sufficiently exciting to merit publication but that nevertheless advance understanding and knowledge. These include the Open Science Framework and PsychFileDrawer.”

  47. R.S.Brown
    Posted Sep 11, 2012 at 9:15 PM | Permalink | Reply


    No one I’ve ever read that’s done serious opinion survey work would used “loaded” questions
    as seen in the Lewandowsy survey(s). “Loaded” questions wirh a four point forced-choice
    answer set (strongly agree; agree; disagree; strongly disagree) but with no “undecided”,
    “uncertain”, “no opinion” or even “duh!” middle-ground answer available to the respondent
    are totally unprofessional.

    Rather than be force to choose a response that the respondent considered “untrue” to
    his/her own opinions, they leave that answer blank and move on to the next question.

    I don’t see where or how Dr. Lewandowsky accounted for “blank” answers in his survey(s).

    In this instance, when the test designer uses a loaded question and forces someone who’s
    never heard of a particular concept or conspiracy to agree/disagree +/- he’s automatically
    creating an answer the anonymous respondent wouldn’t have otherwise made.

    “Spurious” responses often result in polluting the data and, without being fully accounted
    for, end up fostering a “skewed” statistical analysis.

    Anyone who’s ever taken an U.S university undergrad Psych class in testing, or a Political
    Science class in opinion polling or even worked with serious product test surveys has a handle
    on these basic testing principles.

    What in the name of science are they teaching in the land down under ?

    • Skiphil
      Posted Sep 11, 2012 at 10:20 PM | Permalink | Reply

      Good points, a lot of the questions are junk (loaded etc.) and the “forced” choice with no neutral or “don’t know” means a lot of the more reasonable respondents would not complete the survey. He does say in the paper they discarded surveys with any blank responses, and that means (I think) they winnowed the acceptable surveys down to a higher proportion of the most opinionated and more extreme respondents.

      When I started to take the WUWT version (which did allow a neutral answer) I stopped pretty quickly and exited from it because I said “these questions are junk!”

      Lewandowsky says in the paper that a total of 232 surveys were not used in the study pool (which then offered 1145 “complete records for analysis”). In addition to “incomplete” surveys I wonder if there were many people like me who never went far through the survey and clicked away from it — would I have been counted as an “incomplete” survey or would I not even be recorded because I never got far enough to click the “submit” button? I suspect the latter, but either way a very substantial number of non-extreme, non-conspiracist people would have been much more likely to reject the survey’s terms and click away in disgust.

      In addition to Lewandowsky’s biased quest to concoct data with extreme responses to bad questions he seems to have ended up with a really lousy paper in its own analytical terms, as Steve has detailed and as the inept title of the article shows. He wanted an outrageous talking point for John Cook and friends and produced it to order. What a mess.

      • Skiphil
        Posted Sep 11, 2012 at 10:28 PM | Permalink | Reply

        note: the figure of 232 (71 + 161) rejected surveys includes those rejected for a variety of reasons, not only the “incomplete” surveys:

        “Following standard recommendations (Gosling, Vazire, Srivastava, & John, 2004),
        duplicate responses from any IP number were eliminated (N = 71). An additional 161
        responses were eliminated because the respondent’s age was implausible ( 95) or
        values for the consensus items were outside the 0 – 100 range, or because responses were
        incomplete. This left 1145 complete records for analysis.”
        “NASA faked the moon landing, Therefore (Climate) Science is a Hoax:
        An Anatomy of the Motivated Rejection of Science” page 8 (manuscript version)
        Lewandowsky et al (2012)

        I wonder if many who put an implausible age were protesting at the ridiculous nature of the survey? Else why take the trouble to complete the survey and then so obviously try to spike the result?

  48. DM of Western Australia
    Posted Sep 11, 2012 at 9:28 PM | Permalink | Reply

    If I were one of those climate change denying conspiracy theorists I might suspect that Lewandowsky concocted this whole affair to distract Steve McIntyre from the real business of auditing climate science.

    In my humble opinion both Lewandowsky and his “research” are not worthy subjects – except possibly as a minor source of internet humour; certainly not the volume of analysis and discussion he has generated up to now.

    I look forward to seeing normal service resume sometime soon on Climate Audit.

  49. Posted Sep 12, 2012 at 3:42 AM | Permalink | Reply

    Barry Woods above touched on an issue which I think has quite wide implications.

    Lewandowsky’s “Shaping Tomorrow’s World” site, where all this argument is unfolding, is run with funds from UWA and another university listing Lewandowsky and a UWA plant science professor as “Principals” – and a number of academic,business and political luminaries as the Advisory Board.

    Its high minded “mission statement” includes the words:-

    From climate change to peak oil and food security, our societies are confronted with many serious challenges that, if left unresolved, will threaten the well-being of present and future generations, and the natural world. This website is dedicated to discussion of those challenges and potential solutions based on rigorous scientific evidence and objective scholarly analysis.
    Our goal is to provide a platform for re-examining some of the assumptions we make about our technological, social and economic systems.

    However, running WHOIS on site reveals:-

    Registrant Stephan Lewandowsky

    Registrant organisation UWA

    Registrant email contact John Cook @ Skeptical Science

    Admin contact John Cook @ Skeptical Science

    Tech contact John Cook @ Skeptical Science

    In other words this website, set up by universities using public funds with the ostensible purpose of contributing to academic debate, has been handed over in its entirety to well known climate change activist John Cook.

    We have a similar situation in the UK where Adam Corner, the “climate psychologist” who launched Lewandowsky’s paper in the Guardian, over a month before it’s official press release, has used public money from his university (Cardiff) and another (Nottingham) to run a climate activist site “Talking Climate” in conjunction with longtime green activist George Marshall.

    Apart form the obvious dishonesty involved in diverting public funds, intended to education, into blatant activism – these covert “joint ventures” between political activists and previously respectable academic institutions makes it impossible to draw a line between objective science and political opinion.

    Something which is being exhibited in spades at Lewandowsky’s blog right now.

    • Skiphil
      Posted Sep 12, 2012 at 10:11 AM | Permalink | Reply

      Indeed, and FOI requests are needed to try to determine how the project for the activist blog ShapingTomrrowsWorld came to be Proposed and funded. It is clearly propaganda offshoot of John Cook’s SkepticalScience. How exactly is this STW an academic project??

      Lewandowsky’s blog articles certainly do not show an appropiate academic and scientific approach.

      • Steve McIntyre
        Posted Sep 12, 2012 at 10:27 AM | Permalink | Reply

        According to the SkS private forum, Cook and Lewandowsky are very close. One of the forum participants, Tom Dayton, described his background as follows:

        Then my PhD in experimental psychology from the University of Oklahoma in Norman, Oklahoma, where I briefly crossed paths with Steve Lewandowsky, John Cook’s current academic advisor and coauthor while he was a visiting professor

  50. tlitb1
    Posted Sep 12, 2012 at 6:08 AM | Permalink | Reply

    People might want to see Lewandowsky’s latest, discussing the data handling in more detail


    He has a co-author Klaus Oberauer now, so maybe that explains the less snarky feel to this latest? ;)

  51. Posted Sep 12, 2012 at 6:54 AM | Permalink | Reply

    With all due respect, I think Steve McIntyre’s speculations about skydragons are a red herring. There is just no way of knowing whether the respondents known as skeptics are real people or the denier bots that Lewandowsky’s collaborator and co-author John Cook threatened to launch on the blogosphere in private emails to his co-authors at Skeptical Science.

    I have had a lengthy and fruitless exchange of emails with Cook over the question of whether the survey was indeed publicised at his blog Skeptical Science, as Lewandowsky has claimed. All I have managed to learn is that he was asked to publcise it and assented to the request, 28 August. There is no sign of the post on the site or on Wayback, and Cook thinks he must have deleted the post once the survey period was over.

    There is much discussion of Lewandowsky in Cook’s emails to his coauthors from 10 August 2010 onwards, but nothing about the posting at the end of August. On 3 Oct 2010 Cook informs his colleagues: “..then I got involved with Steve Lewandowsky and some of his cognitive colleagues who is very interested in the phenomena of science blogging and they’re planning to do some research into the subject that I’m going to help them with”. 6 October 2010 he tells them: “I’ve been having some intriguing conversations with Steve Lewandowsky who’s throwing cognitive experiment ideas at me to see what’s technically possible. Having a significantly sized group of people classified as skeptic or proAGW makes all sorts of interesting experiments possible.”

    It makes no sense that he should be informing his fellow authors that he’s going to do something he’s just done two months ago. I therefore conclude that the survey was not publicised on Skeptical Science, and the eight blogs who are said in the paper to have provided respondents therefore shrink to six. How Lewandowsky managed to get 1300+ respondents from Tamino’s, Deltoid and Scott Mandia’s is a mystery he should be asked to explain.

    • Bob Koss
      Posted Sep 12, 2012 at 8:18 AM | Permalink | Reply

      It’s easy to get 1000 survey responses. Simply allow repeated visits from the same IP address.

    • Posted Sep 13, 2012 at 3:49 AM | Permalink | Reply

      Re: geoffchambers (Sep 12 06:54),

      Steve’s post about distributing the survey on campus perhaps might be an answer your question …

  52. Posted Sep 12, 2012 at 8:35 AM | Permalink | Reply

    You might be interested in the discussion of this post on the James Randi Conspiracy Forum.

    Here’s a sample quote,
    “I can understand his problem with the methodology, but the idea that conspiracy theorists are bat crazy and believe all sorts of dumb things? This is proof that the study is a “scam”? Nonsense. Anyone who has spent time on this forum knows this to be the case. McIntyre is clearly disturbed by the company he is keeping and is trying to ‘deny’ that such an association exists.”

  53. Posted Sep 12, 2012 at 5:26 PM | Permalink | Reply

    I’ve been watching Newsnight on BBC 2 about today’s report on the Hillsborough Disaster in which 96 football fans died in a crush in a Sheffield stadium in 1989. Three people, including a senior police officer and a Tory MP, are now shown to have plotted the cover-up which smeared the Liverpool fans. ‘The extent of the organised conspiracy is outrageous’ as a current Liverpool MP has just said. Nobody is calling those who campaigned for the truth for 23 years paranoid nutcases, not today anyhow. But all those who have, including a father who lost two daughters, notable in his fairness towards the many policemen who helped, admit that they have been shocked by the extent of the conspiracy. The Sun newspaper has said it is deeply sorry for having been taken in by fake (sometimes called fabricated) reports from the police in the days after the tragedy. Another outraged commentator said on air to the South Yorkshire police chief “Why on earth was there not a single whistleblower among all the policemen who knew that what was being said was lies?”

    This has happened only a day after the memorials for 9-11. The truth must be handled with the utmost care when the untimely deaths of many are involved. And the debate about AGW touches the lives of billions, through its impact on the price of energy and food. There’s may not be as much to laugh about in this area as is often assumed.

  54. Eugene WR Gallun
    Posted Sep 12, 2012 at 10:17 PM | Permalink | Reply

    Taking five minutes of time and off the top of my head — how about comparing these two sets of questions.

    1) The anti-global warming movement is fueled by the maga-millions of Big Oil.
    2) Heartland’s total budget of seven million dollars is many orders of magintude greater than the collective total budgets of all institutions or orgainizations promoting global warming.
    3) The Koch brothers spend tens of millions of dollars a year funding fake skeptic science.
    4) Freedom Of Infromation requests are really nothing more than attempts to impede the vital work of climate scientists.

    1) If Obama is re-elected the Tea Party will attempt a militant takeover of the United States.
    2) Voter I.D. is a right-wing conspiracy to depress voter turnout.
    3) There is a media conspiracy to lie about the fact that Bain Capital took thriving companies and looted their assests and pensions leaving the fired workers destitute.
    4) The Repbulican Party wants to put black people back in chains.

    Somehow I think that very few warmists would disagree with any of the above statements. And if forced into a public answer I don’t believe our vice-president would either.

    We all know where the real conspiracy loons hand out.

    Eugene WR Gallun

  55. Posted Sep 14, 2012 at 2:00 PM | Permalink | Reply

    Typical bunch of tactics, many of them trying to put the monkey on the questionner’s back:
    - claim language is inflammatory/mean/etc
    - stonewall (like saying moderation rules are not open to change, when the issue was a specific application of the rules)
    - sometimes gets funny as the wriggler accuses opponents of doing what he does (in Lewandowsky’ case accusing skeptics of false stories like NASA faking moon landing).
    - moving the pea around (claiming skeptics didn’t find what was inferred as his email when it actually came from a colleague – Barry Woods calls it “odd games”, but at very best I call it scatterbrained, someone who can’t think, can’t check what he and colleagues were doing, more likely he is trying to confuse people by making correct but misleading statements (connections left out, a favourite trick of con artists).
    Only worth the digital ink to add to the evidence of incompetence and bad behaviour of yet another climate alarmist.

  56. BFJ
    Posted Sep 25, 2012 at 12:41 AM | Permalink | Reply

    There is no stronger evidence of dishonesty, than a refusal to hear accusations of it.

  57. Kevin
    Posted Aug 5, 2013 at 3:52 PM | Permalink | Reply

    It is fun to dig. Viewing the source data from view source: http://www.shapingtomorrowsworld.org/lewandowskyGof4.html, you find an addthis id of xa-4bbd967265e13691 and an AdSense id of pub-6821292826962982 and a Google analytics ID of UA-548140. Google those ids and you find some interesting websites associated with/registered to John Cook. Does make you wonder about the human psychology for sure.

    Will keep digging for that official timestamp. The fun is only beginning…

    • bernie1815
      Posted Aug 5, 2013 at 4:35 PM | Permalink | Reply

      Should this be on the more current thread?
      Can you say more about what you found? I hate going to odd sites.

    • DGH
      Posted Aug 5, 2013 at 7:36 PM | Permalink | Reply


      This info would be more useful on the Lew Backdating post. I’ll be interested in what you find because I looked at this angle pretty carefully with admittedly limited expertise.

      Based on Steve’s post and additional info in the thread we know the latest date/time when Dr. Lew posted. It would be interesting if any of the hidden info in the page could frame the earliest date/time that posting might have been made.

9 Trackbacks

  1. By The Daily Lew | Watts Up With That? on Sep 10, 2012 at 4:29 PM

    [...] Lewandowsky Censors Discussion of Fake Data « Climate Audit Rather than answer the question, Lewandowsky, the author of a paper entitled “NASA faked the moon [...]

  2. [...] Climate Audit on moderation of comments over at Lewandowsky's blog [...]

  3. By The Lewandowsky Incident « the Air Vent on Sep 11, 2012 at 8:01 AM

    [...] seems that his attention is now focused on the basic errors in his methodology, which apparently could have been picked up by anyone with even the most [...]

  4. [...] the discussion over at Climate Audit has gotten beyond silly as the flying monkeys, whipped into an orgy of conspiratorial thinking and [...]

  5. [...] Lewandowsky Censors Discussion of Fake Data [...]

  6. [...] seems that his attention is now focused on the basic errors in his methodology, which apparently could have been picked up by anyone with even the most [...]

  7. [...] We have our troublemakers on WUWT as well, and I’ve banned a few, and I understand this is sometimes neccessary, but this sort of intolerant behavior when it comes to debating facts in evidence has surfaced again recently with Steve McIntyre’s straightforward questions to Dr. Lewandowsky. See Lewandowsky Censors Discussion of Fake Data: [...]

  8. [...] also turns out, that Stephan Lewandowsky is John Cooks academic advisor: The AGW Attack Team comes out at a moments call to try and lie their way out of spreading AGW hoax [...]

  9. By Lewandowsky Timeline | Geoffchambers's Blog on Mar 24, 2013 at 7:36 PM

    [...] http://climateaudit.org/2012/09/10/lewandowsky-censors-discussion-of-fake-data/ [...]

Post a Comment

Required fields are marked *



Get every new post delivered to your Inbox.

Join 2,882 other followers

%d bloggers like this: