In this article Anthony O'Hagan analyzes the differences between Bayesian statistics and traditional frequency statistics. He does this by defining the theory of Bayesian statistics and then outlining its strengths and critiques. O'Hagan begins by defining Bayesian Statistcs, saying that Bayesian starts by creating a statistical model to link data to parameters. This Is the only step where Bayesian and frequency statistics are similar. Then Bayes formulates prior information about the parameters. Next it combines the two sources of information and finally it use the resulting posterior distribution to derive inferences about parameters. This process leads " to less pessimism when the data are unexpectedly bad and less
optimism when they are unexpectedly good".
The author then begins to critique Bayesian statistics. O'Hagan demonstrates that the reason Bayes is so controversial is that it is inherently subjective. By using prior information, Bayes becomes subjective because one researchers prior information may be different than someone else's and therefore replicability becomes a problem. O'Hagan then states that Bayes accounts for this by scaling and assessing the prior data for subjectivity. He also states that frequency statistics are hypocritical in there assumption that frequency statistics assume that they have. O'Hagan also states that frequency statistics are usually misinterpreted.
At the end of the article O'Hagan outlines the total benefits of the Bayesian approach to statistics. Which are that Bayes gives a more direct, intuitive
and meaningful statement of the probability that the hypothesis is true, that Bayesian methods can answer complex questions cleanly and exactly, that no relevant information is omitted in a Bayesian
analysis, and that Bayes can quantify uncertainties for decision makers.
This article really does a great job of explaining how Bayesian Statistics are done and the theory behind the method. It gives a number of resources for learning about the application of Bayes and some critiques and analysis of frequency statistics that are rarely brought up. However, I did not think that this article truly critiques Bayesian statistics. The author ever really accepts the weakness of Bayes that there is no real way to manipulate the prior information to make it uniform, and that in order to perform a good Bayesian analysis you need to have serious computational power. It seems to me that Bayes can be a great tool but until more research is done to refine it it will only ever be just another regression method to use.