Daniel Kahneman, a Nobel laureate, recently wrote a scathing letter about experiments by social psychologists purporting to link social priming to associative memory (h/t reader Fred S). Kahneman described a “storm of doubts” about social priming results, inability to replicate claims and characterized the field as a “train wreck looming” and a “mess”. Kahneman’s letter was covered yesterday in Nature here.
A prominent academic in the field, one who has published both on priming and associative memory, is also well-known to Climate Audit readers.
Kahneman, undoubtedly stung by recent scandals in social psychology, wrote a remarkably hard-hitting letter:
I write this letter to a collection of people who were described to me (mostly by John Bargh) as students of social priming. There were names on the list that I could not match to an email. Please pass it on to anyone else you think might be relevant.
As all of you know, of course, questions have been raised about the robustness of priming results. The storm of doubts is fed by several sources, including the recent exposure of fraudulent researchers, general concerns with replicability that affect many disciplines, multiple reported failures to replicate salient results in the priming literature, and the growing belief in the existence of a pervasive file drawer problem that undermines two methodological pillars of your field: the preference for conceptual over literal replication and the use of meta-analysis. Objective observers will point out that the problem could well be more severe in your field than in other branches of experimental psychology, because every priming study involves the invention of a new experimental situation.
For all these reasons, right or wrong, your field is now the poster child for doubts about the integrity of psychological research. Your problem is not with the few people who have actively challenged the validity of some priming results. It is with the much larger population of colleagues who in the past accepted your surprising results as facts when they were published. These people have now attached a question mark to the field, and it is your responsibility to remove it.
I am not a member of your community, and all I have personally at stake is that I recently wrote a book that emphasizes priming research as a new approach to the study of associative memory – the core of what dual-system theorists call System 1. Count me as a general believer. I also believe in a point that John Bargh made in his response to Cleeremans, that priming effects are subtle and that their design requires high-level skills. I am skeptical about replications by investigators new to priming research, who may not be attuned to the subtlety of the conditions under which priming effects are observed, or to the ease with which these effects can be undermined.
My reason for writing this letter is that I see a train wreck looming. I expect the first victims to be young people on the job market. Being associated with a controversial and suspicious field will put them at a severe disadvantage in the competition for positions. Because of the high visibility of the issue, you may already expect the coming crop of graduates to encounter problems. Another reason for writing is that I am old enough to remember two fields that went into a prolonged eclipse after similar outsider attacks on the replicability of findings: subliminal perception and dissonance reduction.
I believe that you should collectively do something about this mess. To deal effectively with the doubts you should acknowledge their existence and confront them straight on, because a posture of defiant denial is self-defeating. Specifically, I believe that you should have an association, with a board that might include prominent social psychologists from other field. The first mission of the board would be to organize an effort to examine the replicability of priming results, following a protocol that avoids the questions that have been raised and guarantees credibility among colleagues outside the field.
Cited in the Nature article as an example of priming literature was a 1998 article by Djikerhuis and van Knippenberg, who postulated people who were primed by thinking about professors were at least temporarily smarter than people primed by thinking about secretaries (or at least better at Trivial Pursuits questions.) Apparently, spending more time meditating about professors led to even more smartness.
Stephen Lewandowsky, familiar to CA readers, is apparently among the leaders in the field of associative memory (“associative memory” being one of the concepts central to the Kahneman letter.) In 1993, Costa and Garcia-Marques (1993) described Lewandowky’s theory of associative memory as the “most prominent” in the literature:
The Theory of Distributed Associative Memory (TODAM, Lewandowsky & Li, 1994; Lewandowsky & Murdoch, 1989; Murdoch, 1993, 1995, 1997) based on the notion of associative chaining seems to be the most prominent theoretical proposal in the literature.
In a 1992 review here, Abhijit Sanyal stated:
Lewandowsky, Kirsner and Bainbridge (1989) have suggested a sense-activation view to reflect implicit memory for new associations…
For instance, past research found that simple perceptual manipulations can induce false memories, such as through immediate repetition priming (Jacoby & Whitehouse, 1989), immediate semantic priming (Lewandowsky, 1986), enhanced visual clarity of a test word (Whittlesea, Jacoby, & Girard, 1990), or fragment completion prior to testing (e.g., Luo, 1993; Watkins & Peynircioglu, 1990). In the current research, we investigate the claim that changes in recognition memory with immediate priming are due to changes in perceptual fluency
Lewandowsky had specifically published on priming. For example, in 1986, he published “Priming in recognition memory for categorized lists” (Journal of Experimental Psychology: Learning, Memory, and Cognition, 12, 562-574).
In 1989, Lewandowsky, Kirsner and Bainbridge published “Context Effects in Implicit Memory: A Sense-Specific Account”, a section of a book edited by Lewandowsky and two co-editors entitled “Implicit Memory: Theoretical Issues”.
In 1993, Lewandowsky and two coauthors published “Context effects in repetition priming are sense effects” (Memory & Cognition, 21, 619-626).
Lewandowsky has continued to publish academic articles on the topic of associative memory. (See his publications here).
I do not know whether Lewandowsky was included in the list of recipients of the Kahneman letter or whether Lewandowsky’s results were among those which worried Kahneman.
However, the replicability of the results of Lewandowsky et al 2012 in a properly supervised survey of actual skeptics seems very dubious to me. Kahneman expressed serious concerns about the experimental set-up of important social psychology experiments on priming – concerns that apply even more forcefully to the Lewandowsky survey where no effort was made to ensure data integrity and which was carried out at anti-skeptic sites, sites that, in some cases, had been primed by prior knowledge of Lewandowsky’s conspiracy theory.
The Nature article viewed “skepticism” about social psychology articles as entirely justified, noting that some of the names involved in the prominent recent frauds in social psychology were also connected with “priming” research:
This scepticism has been fed by failed attempts to replicate classic priming studies, increasing concerns about replicability in psychology more broadly (see ‘Bad Copy’), and the exposure of fraudulent social psychologists such as Diederik Stapel, Dirk Smeesters and Lawrence Sanna, who used priming techniques in their work.
Gone, I guess, are the halcyon days when psychology journals could publish articles on astrology without inconvenient criticism e.g. Fuzeau-Braesch (1992) “An empirical study of an astrological hypothesis in a twin population”, published in an Elsevier academic psychology journal, Personality and Individual Differences. At Lewandowsky’s blog, commenter John Mashey sharply criticized the publication of a second article by Fuzeau-Braesch in a lower-tier science journal, a criticism with which I agree. However, neither Mashey nor Lewandowsky took exception to the earlier Fuzeau-Braesch astrology article in a psychology journal, a class of journal for which they apparently have lower expectations.