Summary
There has been controversy over the claim that prediction
markets elicit ‘aggregate beliefs’, normally understood to mean the average of
beliefs for the population. There is no debate over whether these markets generate
good predictions, in the sense that they forecast outcomes well. Instead the
debate has been over the additional claim that the prices in these markets
recover the mean of aggregate beliefs. In other words, are the observed prices
in these markets good estimates or predictors of aggregate beliefs?
According to the article, prediction markets cannot
always be relied on to elicit any interesting statistic of aggregate beliefs. Formal
derivations of the bets placed in prediction markets can be viewed as demands
for state contingent commodities.
Prediction markets can be expected to do a good job
recovering the average of aggregate beliefs under certain circumstances:
unimodal distributions of beliefs, with no a priori reason to expect
heterogeneity on either side of the market. Indeed, this environment might characterize
many interesting settings, such as political elections or closed prediction
markets in which there is minimal sample selection into the market (on the
basis of beliefs, preferences and endowments). But the result is not general,
and it is easy to construct examples in which prediction markets do a
predictably poor job of recovering average beliefs.
Fountain,
John, and Glenn W. Harrison. 2011. "What do prediction markets
predict?." Applied Economics Letters 18, no. 3: 267-272. Business
Source Complete, EBSCOhost (accessed September 18, 2015).
Critique
While I would concur to a cursory extent that the effect
that prediction markets would assist with initial predictions, it is imperative
to account for the nature of constantly changing variables in this market. In other
words, though they would help to create initial models used to create plans,
there needs to be a degree of fluidity in the plan built in to account for
adaptations said market.
Posts
This article is confusing, extremely hard to comprehend, and I have no idea what the math behind their experiment means. However, I do have three questions which I hope will help me understand the premise of the author’s arguments.
ReplyDeleteFirst question, are they saying that the average beliefs derived from prediction markets results in the predicted forecast? Meaning the outcome of whatever the bet or forecast was, is a result of what the population believes in?
Second question, I copy and pasted this part because I don’t know how to reword the paragraph. “Instead the debate has been over the additional claim that the prices in these markets recover the mean of aggregate beliefs. In other words, are the observed prices in these markets good estimates or predictors of aggregate beliefs?” Can you explain what the authors mean, or provide additional literature on the mentioned debate?
Third question, based upon your critique, and from what I understood from the article, are the authors saying prediction markets do a poor job of forecasting when the results can’t be measured in a “yes or no” format?
I think Shadya hit the nail on the head in response to your first question. For the other two, I would have preferred more qualitative answers, or an explanation without all the statistical terminology (which is frequently too much for me to keep straight). That being said, I think the key point in the second question is the difference between "good estimates" and "predictors of aggregate beliefs" and I think that depends on who is buying and selling on the market. Markets composed of calibrated analysts will likely have prices reflecting good estimates, while markets composed of untrained individuals will likely have prices that simply reflect aggregate beliefs, regardless of the quality of estimate.
DeleteWith the third question, I think the main problem with Yes/No questions is that, depending on the difficulty of the question, the average of all estimates will trend towards 50/50, which really does not help a decision maker. However, with questions that have multiple possibilities, such as "who will win the Republican presidential nomination," the analytic estimates are distributed over several possibilities and can provide a much better picture of probabilities.
Thank you for commenting on this article. I understand your frustration and how the terminology used in the article can be confusing.
ReplyDeleteQuestion: “First question, are they saying that the average beliefs derived from prediction markets results in the predicted forecast? Meaning the outcome of whatever the bet or forecast was, is a result of what the population believes in?”
Response: What the author means is that there is no claim that these markets elicit individual beliefs or that they are free of individual irrationalities: see Forsythe et al. (1999) for a survey of field and laboratory evidence on this issue. The claim about aggregate beliefs rests on informal statements about the propensity of ‘marginal traders’ to get market prices right (in some sense).
Forsythe, R., Nelson, F., Neumann, G. R. and Wright, J. (1992) Anatomy of an experimental political stock market, American Economic Review, 82, 1142–61.
Based on the information above, I believe your assumption is correct to a certain extent. I assess the author is trying to say that the “population beliefs” or sentiment is factored into the process.
Second question and statement: “Instead the debate has been over the additional claim that the prices in these markets recover the mean of aggregate beliefs. In other words, are the observed prices in these markets good estimates or predictors of aggregate beliefs?” Can you explain what the authors mean, or provide additional literature on the mentioned debate?"
To answer this question, I think it’s important to define what is an aggregate belief or aggregate data. Aggregate data is data combined from several measurements. When data are aggregated, groups of observations are replaced with summary statistics based on those observations. In a data warehouse, the use of aggregate data dramatically reduces the time to query large sets of data. (For some researchers, the ‘summary of statistics’ can pose a problem when conducting a forecast.)
Consider the next thought on what aggregate beliefs are.
The University of Michigan’s Strategic Reasoning Group on aggregate belief: “We consider the problem of belief aggregation: given a group of individual agents with probabilistic beliefs over a set of uncertain events, formulate a sensible consensus or aggregate probability distribution over these events.” http://web.eecs.umich.edu/srg/?page_id=319
The author’s original debate question: Are the observed prices in these markets good estimates or predictors of aggregate beliefs?
The authors of the article argue that prediction markets cannot be relied on to always elicit any interesting statistic of aggregate beliefs. Formal derivations of the bets placed in prediction markets can be viewed as demands for state contingent commodities. Researchers provide derivations for two popular cases, log utility and Constant Relative Risk Aversion (CRRA) utility, connecting these derivations to familiar scoring rules. They then use these results to demonstrate how the properties of prediction markets depend critically on the assumed homogeneity of participants.
Conclusion of the results: Putting the results together, and in conjunction with those of the earlier literature, researchers conclude that prediction markets can be expected to do a good job recovering the average of aggregate beliefs under certain circumstances: unimodal distributions of beliefs, with no a priori reason to expect heterogeneity on either side of the market. Indeed, this environment might characterize many interesting settings, such as political elections or closed prediction markets in which there is minimal sample selection into the market (on the basis of beliefs, preferences and endowments). But the result is not general, and it is easy to construct examples in which prediction markets do a predictably poor job of recovering average beliefs.
Third question: “Are the authors saying prediction markets do a poor job of forecasting when the results can’t be measured in a “yes or no” format?”
Third question Answer: Based on the evidence, prediction markets do a good job of recovering the average of aggregate beliefs under certain circumstances, such as unimodal distributions of beliefs with no a priori reason (relating to or denoting reasoning or knowledge that proceeds from theoretical deduction rather than from observation or experience.)
ReplyDeleteMoreover, if there is a single mode in the distribution function, the distribution function is called "unimodal."
So, the aggregate beliefs must be measurable. A ‘yes or no’ question is measurable and something that you can average, but will it provide for the best prediction? According to the author, prediction markets do a predictably poor job of recovering average beliefs in where in which there is minimal sample selection into the market (on the basis of beliefs, preferences and endowments). I hope this helps you formulate an opinion.
Andrew, this is an interesting point of view. I agree with your point in that it depends who is buying and selling in the market as well as who is calibrating the "aggregate beliefs" or "summary of results." Having a 'yes or no' question does prove to be ineffective in certain circumstances. Although, having several possibilities can provide a much better picture of probabilities, I believe the 'summary' of the data can often leave out details that can cause a 'misfire' in the estimate of said prediction market.
ReplyDeleteIt was difficult to explain the article in a qualitative way due to the statistical data supporting the author's claims, but I think your explanation is a good representation of what the author is trying to say.
Shadya, thank you for the well thought out comments. I have read your responses, and the information you provided answered all my questions.
ReplyDelete