Bayesian Analysis Versus Null Hypothesis Significance Testing

Introduction
While Bayesian cognitive models have grown in popularity, Bayesian statistical data analysis still has its fair share of detractors. This Trends in Cognitive Sciences article consists of an elaboration on some advantages Bayesian analysis holds over traditional null-hypothesis significance testing (NHST).


Summary
One of the most significant flaws of NHST that the article pinpoints is the concept of and reliance upon "p-values." The p-value forms a semi-arbitrary line marking whether a set of data offers statistically significant results, but has no unique value for any given set of data. Results may be "significant" at a p-value meaning 90% of randomly-sampled sets would share these results, but not significant if only 89% of such sets shared the results. Important patterns and trends can be easily dismissed by barely missing the p-value mark. Bayesian analysis does not rely on p-values. NHST also summarizes a data set with a point-estimate value such as t or F, with similar problems of variability and arbitrary approximation of confidence intervals.

Compounding these problems are certain computational restraints - in calculating variance, p-values, and other values for NHST, the calculations become much more complex if data sets do not share the same number of data points or variance ranges.

Comparatively, the article goes on to discuss how Bayesian data analysis offers a descriptive model that is expressly not reliant on the arcane p-value or a certain significance test that only works for a certain data-set size. Bayesian inference is also extremely flexible and customizable, capable of comparing any number of variables across the parameters of a group without computational penalties.


Conclusion
As a model of data, Bayesian analysis is more robust and less restrictive overall than NHST. The article predicts that Bayesian methods for data anlysis will be the preferred method going forward in the 21st century due to the advantages listed in abbreviated summary above.


Source
Kruschke, John K. (2010). What to believe: Bayesian methods for data analysis. Trends in Cognitive Sciences. Volume 14, Issue 7. pp 293-300. Available online at http://www.sciencedirect.com/science/article/pii/S1364661310000926.