What do parapsychology and data science have in common?

I have been interested in parapsychology ever since picking up a copy of the excellent Eysenck and Sargent book – Explaining the Unexplained many years ago. It’s a bit dated now but still a great introduction for anyone interested in learning more about the topic.

Just like with data science there is sometimes confusion about what parapsychology is. Perhaps it’s easier to start with what it’s not. It’s not astrology, ghost busting, monster hunting, fortune telling or investigating UFO sightings though these are things that often come to mind when one thinks of the paranormal mainly because of the influence of television shows on the ‘paranormal’.

Parapsychology can be defined as the scientific study of experiences which, if they are as they appear to be, are in principle outside the realm of human capabilities as presently conceived by conventional scientists (Irwin & Watt, 2006). Note that this definition does not make any claims about the paranormal, it merely indicates that parapsychology is the study of experiences which seem to be paranormal.

Admittedly that definition is quite broad. It might be useful to consider the general subject matter of parapsychology. There are 3 main areas of study …
1) The psi hypothesis
2) Psychokinesis
3) The survival hypothesis

The first of these, psi, deals with telepathy (mind to mind communication), clairvoyance (extrasensory awareness of events) and precognition (extrasensory awareness of a future event). Psi is a Greek term denoting the apparent paranormal element of these processes. The second of these main areas of study, psychokinesis, concerns the direct influence of mind on the physical environment or an organism (as in Direct Mental Interaction with Living Systems). Psychokinesis (PK) is sometimes divided into macro PK (visible effects) and micro PK (statistical effects are apparent). The survival hypothesis is based on the idea that consciousness or some part thereof survives death.

Given its subject matter you might be wondering why I am writing about parapsychology on a data science blog but in fact the two are more related than may appear at first sight. This is because parapsychology and data science both touch on meta science and questions about how scientific research is conducted in general. Also currently much of the debate about evidence for psi concerns statistical matters. Before discussing statistical issues in parapsychology, lets consider some of the criticisms of this controversial area of research.

Fraud
Skeptics have argued that the possibility, no matter how remote, that fraud could have occurred should invalidate the results of parapsychology experiments. This criticism has more or less dropped by the wayside since it is not really a tenable position. Taken to it’s logical conclusion it would make scientific progress difficult since most research, parapsychological or otherwise, would be invalidated.

Publication Bias
Publication bias refers to the observed bias towards publishing significant results while ignoring non significant ones. It is an issue across all areas of scientific endeavour and something parapsychologists are very aware of. Given the normal significance threshold of .05 one can expect to see significant results around 5 times out of 100 purely by chance. Skeptics have argued that the published results represent this 5% and the remaining 95% of studies are discarded to languish in a file drawer somewhere. Since 1975 however the Parapsychological Assocation has adopted a policy of publishing null results. In fact they were the first scientific organisation to do so!

In addition, one of the pioneers of meta-analysis, Rosenthal showed how to calculate a fail-safe N that would indicate how many unpublished studies are needed to negate the significant ones that make it to publication. Storm and colleagues showed that for the Ganzfeld database there would need to be 2414 unpublished null studies to reduce the database to non significance. Some have argued that Rosenthals fail safe N may not be conservative enough since it assumes that the results of the group of unpublished studies are unbiased when in fact they may be directionally negative. Even correcting for this, and in this most conservative of scenarios the number of unpublished studies would need to be 384. Given the paucity of resources directed to studying parapsychology and the existing policy of publishing null results a file drawer problem sufficient to negate the significant results observed may be considered very unlikely.

Replicability
Skeptics have argued that psi is not a replicable phenomenon and therefore doesn’t exist. There are a number of problems with this criticism. Firstly not all experimental effects are replicable on demand. This is particularly true in experiments dealing with human capabilities. This doesn’t mean those effects don’t exist. Chris Carter draws an analogy with baseball.

Even the best hitters in baseball cannot produce home runs on demand nor can we predict with certainty when home runs will occur…Yet over the long run, a good baseball player should hit the ball a fairly consistent proportion of times and we should expect fairly consistent differences between players (Carter, 61-62)

Secondly when looking at replication in parapsychology one needs to consider study power. The power of a study is it’s ability to correctly reject the null hypothesis. It is dependent on a number of factors including sample size and the effect size of the phenomenon under investigation. Given the small effect sizes evident in parapsychology research and the small sample sizes (mainly due to underfunding) it is not surprising that many experiments fail to demonstrate a psi effect. In fact it has been shown that the proportion of replications which reach statistical significance is about what would be expected given the power of the average Ganzfeld experiment.

Thirdly by aggregating the results of studies, as in meta-analysis, one can show that over the long run there is a significant trend demonstrating a psi effect, far above that which would be expected by chance.

‘Real World’ Applications
Some skeptics argue that psi doesn’t exist since if it did casinos would go out of business, fortune tellers could predict lottery numbers, tell our futures etc. To me this shows a misunderstanding (wilful or otherwise) of what the results of laboratory experiments have shown. Some of the best evidence for psi comes from Ganzfeld experiments. What these studies show is that receivers can correctly identify a target about 32 times out of 100 when 25 times from 100 would be expected by chance. This may not sound impressive but the aggregated results over many different subjects and studies are highly statistically significant and indicate that something other than chance is operating. It also shows that psi abilities, in the general population at least, appear to be relatively weak.

By the way if you want to know how fortune tellers or professional psychics give readings without using any psi whatsoever then I highly recommend this book on cold reading. As for casinos it is difficult to imagine a less psi conducive environment!

Methodological Flaws
Some skeptics have indicated that positive results in parapsychology can be explained by methodological flaws such as sensory leakage, optional stopping, pilot to confirmation etc. Nonetheless recent Ganzfeld studies show a significant and positive correlation between ratings of study quality and effect size. This is the opposite of what would be expected if results were due to flaws!

In a recent study Bierman and colleagues used survey research to estimate the extent of what they call ‘questionable research practices ‘ (QRPs) in experimental psychology and then simulated a Ganzfeld meta analysis to assess how the results may have been influenced by QRPs. They concluded that the effect size noted in Ganzfeld meta analysis is very likely inflated by QRPs, however even after adjusting the results for this, they note that meta analyses still evidence a significant effect not explainable by the QRPs they simulated. This approach assumes a similar rate of QRPs in parapsychology as in other areas of experimental psychology something which, given the scrutiny parapsychological research is subject to, may not be warranted.

In 1995, Utts in her review of the evidence for psychic functioning concluded that…

Using the standards applied to any other area of science, it is concluded that psychic functioning has been well established. The statistical results of the studies examined are far beyond what is expected by chance.

Nothing has happened in the intervening years that warrants changing this conclusion. In fact it seems hypocritical of some skeptics to claim the evidence for psi is non existent or weak and at the same time accept experimental findings of similar evidential value in more mainstream research.

The key part of Utts conclusion is the bit about using the standards applied to any other area of science. Some skeptics now claim that parapsychology should be subject to different standards than other areas of scientific endeavour. If so, the questions remains…Why?- Is it just me or can anyone else sense goalposts being moved here 🙂

Some have also argued that findings from parapsychology illustrate why psychologists as a group need to change the way they analyse their data. It is to this debate about statistics and how data from scientific experiments is analysed that I will turn to in the second part of this article.

If interested in reading the primary sources for this article yourself then take a look at the bibliography for references.

One comment

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.