Friday, September 1, 2017

Why are we not evaluating multiple competing hypotheses in ecology and evolution?

By : Praveen Kumar Neelappa

This article suggests that there is the gap between theory and practice in the use of analysis of competing hypotheses (ACH). It identifies several intellectual and practical barriers that discourage the use of multiple hypotheses in the field of ecology and evolution. This article points out that scientist have a bias or a motivation to consider one hypothesis over other (Intellectual barriers) and there are practical limitations inherent to factorial design, the standard experimental design that allows researchers to evaluate several explanatory variables and their interactions in the same study, one variable at a time (Practical barrier).

Cognitive bias makes us think that we are making a logical, rational and effective decision while considering the alternative hypothesis, but our unconscious bias influences the experiment and its outcome. There is a tendency of scientist to put more weight on evidence that supports favored ideas more than other evidence that is available (Confirmation bias), seek for the pattern in their experiment (Pattern seeking bias) and be judged only by their internal consistency (Belief bias). There are several ways one might minimize the effect of cognitive bias in science so that one does not rely exclusively on one’s perceptions. It can be achieved by masking (kept) information about the experiment from the participant, to reduce or eliminate bias, until after a trial outcome is known (Blind bias), working with other scientists with different perspectives (Work with the enemy) and a null model which generates a pattern in the absence of any biological process, forcing the researcher to think about many different hypotheses, which could potentially minimize the negative impacts of cognitive biases in science.

Any study that has a simple, easy to understand explanation will be preferred over a study that employs complex and perhaps less-elegant ideas (Simplicity bias) to avoid practical barrier. Editors and reviewers tend to rely on prior knowledge when evaluating a manuscript, creating additional difficulties for researchers when publishing studies that confront well-established ideas. This tension between new and old ideas could reflect a conflict between new and old generations (Publication bias).

The article concludes by suggesting that ecological and evolutionary research is aimed at understanding patterns arising from nonlinear and stochastic interactions among a multitude of processes and agents at multiple spatial and temporal scales. If we wish to truly advance scientific progress despite this complexity, we must better commit to strong inference in our scientific inquiries by simultaneously evaluating multiple competing hypotheses.

Critique:

The use of ACH is widely promoted to enhance the effectiveness of the scientific investigation. This article points out some valid draw backs of using ACH in the different field of studies and discusses these drawbacks and solutions to them in detail. The article clearly illustrates various types of biases and the various stages where they can be encountered while carrying out experiments. It is imperative that the individual carrying out the experiment is objective in data collection and maintains an objective view at every stage of the experiment which will be the best way to counter any possibility of the final results being biased. Additionally, measures need to be put in place to reassess for bias along the way to ensure the results are void of any form of bias.

Citation:

Betini GS, Avgar T, Fryxell JM. 2017 Why are we not evaluating multiple competing hypotheses in ecology and evolution?.R. Soc. open sci. 4: 160756. http://dx.doi.org/10.1098/rsos.160756

8 comments:

  1. I really like how the article breaks down the various types of biases that an individual faces when conducting a study/analysis. The danger of cognitive bias can not be overlooked or ignored. The subtlety of cognitive bias is one reason the individual conducting the experiment should pay close attention when using ACH.

    I agree with Praveen on the need for objectivity.

    ReplyDelete
    Replies
    1. Thank you Oddi, the article do cover various bias and some suggestion meeting on how to minimize it.

      Delete
  2. "The article concludes by suggesting that ecological and evolutionary research is aimed at understanding patterns arising from nonlinear and stochastic interactions among a multitude of processes and agents at multiple spatial and temporal scales. If we wish to truly advance scientific progress despite this complexity, we must better commit to strong inference in our scientific inquiries by simultaneously evaluating multiple competing hypotheses."

    I have been saying this about evolution research forever. Especially the belief bias. There is literally no reason water is the determining factor for all life in the universe but that's all science knows. They need to ACH it haha

    ReplyDelete
    Replies
    1. I agree that water is not the only determining factor for life is universe. But for life, as we know it, it's one of the most determining factor. I believe ACH would not be a good way to determine that, bias would rip apart the scientific community on this. :)

      Delete
  3. I agree with your critique that it is imperative for the analyst to remain objective at every stage during the experiment. This article makes a good point discussing simplicity and pattern-seeking bias. Today we live in a world confronted by many complex problems and it is tempting for many analysts to overlook problems/issues within their research in an attempt to simplify an issue that in reality is not so simple. Many analysts seek to avoid practical barriers and often fall victim to pattern-seeking bias in an effort to produce a preferred, easy to understand study. Analysts must be careful to avoid these biases which may ultimately undermine the validity of their products.

    ReplyDelete
    Replies
    1. There are some suggestion given in the article on how to avoid bias. But as you pointed out " we live in a world confronted by many complex problems and it is tempting for many analysts to overlook problems/issues within their research in an attempt to simplify an issue" it is tough to be objective.

      Delete
  4. I found it interesting that of the 100 randomly selected studies regarding ecology and evolution, only 21 tested two hypotheses, and only eight tested more than two. I also appreciated how the article breaks down the barriers that may prevent researchers from utilizing multiple hypothesis testing. As I posted on Jared's article, I truly believe collaboration with individuals having different perspectives is a key combatant of cognitive bias.

    ReplyDelete
    Replies
    1. I was surprised to see such low numbers. Ecology and evolution doesn't allow interpretation to human imagination as in case of other social studies. So its really hard to have multiple hypothesis in this field. So such less number in this field of studies

      Delete