A Bayes factor analysis of Extrasensory Perception (ESP) claims

Introduction:
Daryl Bem a social psychologist at Cornell University has claimed that people can feel or sense important events in the future that could not otherwise be anticipated.  In Bem’s experiments he claimed that people can feel future reward and punishment events and were able to anticipate a random choice at a rate above chance.  Bem, in his experiment, used a conventional approach where the same basic phenomena was targeted from slightly different angles. Bem used null-hypothesis significance testing and p-values as evidence to reach his judgements.  The authors of this article, Rouder and Morey, employed a Bayes factor analysis to assess the evidence in the Bem experiments.   

Summary:
The authors’ of the article attempted to employ a Bayes factor measurement to evaluate Bem’s evidence.  A Bayes factor measurement is a method of model selection based on Bayes posterior odds and is used as an alternative to frequency hypothesis testing (which is what Bem used).  The posterior odds in a Bayes analysis is the probability ratio of data given hypothesis.  The formula for the posterior odds is usually given as:

Where:

• Pr(M|D) is the posterior odds, which describe the analyst’s degree of belief in the hypothesis after observing the data.  Usually described as the Probability of M given D.
• Pr(D|M) is a likelihood and represents the probability that some data is produced under the assumption of the model.

The basic assumption in Bayes factor analysis is that prior (Pr(D|M)) and posterior information (Pr(M|D)) are combined into a ratio that provides evidence in favor of one model versus another.   

The general form given for Bayes factor is:


The interpretation of K is given as:

Rouder and Morey’s analysis found that people can “feel” the future with neutral and erotic stimuli to be slight, with Bayes factor scores (K value) of 3.23 and 1.57 respectively.  There was, however, some evidence for Bem’s hypothesis that people can “feel” the future with emotionally valenced nonerotic stimuli. A Bayes factor of 40 was recorded for this type of stimuli.  The authors think the K value of 40 is noteworthy, but, they believe it is still a magnitude lower than what is required to overcome appropriate skepticism of ESP.

The summary of Rouder and Morey’s findings are below:

Conclusion:
Based on the number of articles read to create this summary, Bayes factors appear to be flexible and allow for the comparison of multiple hypotheses simultaneously.  Statisticians claim the Bayes factor is intuitive, however, the factors are difficult to calculate. Also, Bayes factor analysis is unlikely to be undertaken by someone with only a cursory understanding of statistics.   

Source:
Rouder, J., & Morey, R. (2011). A bayes factor meta-analysis of bem's esp claim. Psychonomic Bulletin & Review, 18(4), 682-689. Retrieved from http://drsmorey.org/bibtex/upload/Rouder:Morey:2011a.pdf