Friday, September 18, 2015

Prediction Market Accuracy in the Long Run

Prediction markets are designed and conducted for the primary purpose of aggregating information so that market prices forecast future events. These markets differ from typical, naturally occurring markets in their primary role as a forecasting tool instead of a resource allocation mechanism. In this paper the authors suggest that the prediction markets outperform polls for longer horizons by documenting the evidence from 1988 to 2004 elections. They compare unadjusted market predictions to 964 unadjusted polls over the five Presidential elections since 1988. What they found is intriguing: The market is closer to the eventual outcome 74% of the time. Further, the market significantly outperforms the polls in every election when forecasting more than 100 days in advance.  In their study they utilized the Iowa Electronic Markets (IEM) prices and raw poll dataset. We can compile the merits of predictions markets compared to polls based on the study findings:
  •      Polls can`t have the true random sample; whereas the prediction markets customers can be very heterogeneous.
  •       Production markets can forecast complex phenomena due to several reasons.

o   The market design forces traders to focus on the specific event of interest more than simple consideration of a fictitious election “if it were to be held today” (as polls ask respondents to consider)
o   Traders must act rationale since they put money on stake.
o   Markets aggregates dynamic information from a wide variety of sources, i.e. traders.
o   The markets provide an incentive to generate, gather and process information across information sources and in a variety of ways. (If you do good, you prosper.)
  •       For the five elections, the average absolute error in the market’s prediction of the major-party presidential vote share across the 5 days prior to the election was 1.20 percentage points, while opinion polls conducted during that same time had an average error of 1.62 percentage points.
  •       Unlike polls` random selection, the participants of prediction markets are self-selected.
  •       Unlike polls or expert panels in which participants are asked for their independent opinions, each trader in the market sees the net effect of the beliefs of all other traders, and the time series of changes in those beliefs and can alter his own perceptions accordingly.
  •       Unlike polls that ask each respondent how he or she would vote if the election were held today, the market asks traders to forecast how everyone will vote in the actual upcoming election. (We can suggest that the sentiments play role on polls; whereas the factuality in PMs)
  •       ``Convention bounce`` effects don`t appear in prediction markets.
  •       It gives continuous updates.
  •       Because they react dynamically to information, they can also be used as evaluation tools to assess the impact of decisions such as policy positions, candidate viability, campaign strategies, etc.

The study lays out the facts effectively. The authors compile the prediction markets` advantages and superiorities over the polls. However, the study does't mention about the weaknesses of the technique which can be the sign of confirmation bias. Moreover, polls attract many attentions currently. If the prediction markets have lots of merits over polls, I would expect the study to identify the reason why still polls make people interested in them. Nevertheless, the study explains very well why the prediction markets outperform polls through five consecutive years.
Berg, J. E., Nelson, F. D., & Rietz, T. A. (2008). Prediction market accuracy in the long run. International Journal of Forecasting, 24(2), 285-300.


  1. The authors mentioned that prediction market forecasting elections more than 100 days outperformed the polls. Do you think the technique has the same success percentage in different areas of research? Or is simply a political phenomenon when compared to polls?

    1. Since the polls and prediction markets strive to answer a question in a 'Yes or No' format, the authors were able to compare them. Otherwise it would be like comparing orange and apple. With this in mind, yes, this technique may show high percentage of successes in other fields too if their research questions are sought be answered as yes or no.

    2. Understood, and thank you for the response.

  2. The article implies that changing trends in the prediction market may affect and alter participants' own perceptions accordingly. Therefore, it has similar effects with the subsequent rounds of the Delphi method. It somehow forces participants to reach a consensus. I believe this feature makes both of these two methods robust and successful.

  3. I liked the articles examination of how the prediction market can be used as a political barometer of sorts in that politicians can get real time feedback on decisions made. I think it is also important that the market is cognizant that the election is not today and is more accurate, farther out, as a result of it. Respondents recognizing the challenges of a long campaign and taking into account a larger number of variables makes it an effective tool.

  4. I agree with Dan. Prediction markets have a different timeline than polls allowing for more deliberation and situations to develop further. I could not help but think of the presidential campaign and Donald Trump when I read this. It makes me wish the media used prediction markets instead of polls when reaching out to the people.