Prediction Markets (Rating: 4 out of 5 stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the  articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in September 2015 regarding Prediction Markets as an Analytic Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Prediction markets are designed and conducted for the primary purpose of aggregating information so that market prices forecast future events. These markets differ from typical, naturally occurring markets in their primary role as a forecasting tool instead of a resource allocation mechanism. For example, hedge fund prediction markets can allow people to stake bets, both real or imaginary, in predicting an outcome.

Strengths:

• This method allows the development of a consensus estimate/prediction.
• This method can give participants an incentive to provide accurate and truthful information.
• This method provides anonymity for its participants.
• This method accounts for more nuance than random opinion polling.
• This method allows a percentage bet to occur, thus giving the bet some value instead of only a straight yes/no. You can thus quantify something like “highly likely”.
• This method provides fairly accurate answers compared to other analytical techniques predicting the same event with a similar participant pool unless they are trying to answer a question in a format other than ‘Yes or No’ format.

Weaknesses:

• This method requires a high volume of trading.
• This method relies on how good the information is that is given.
• This method works well if the preconditions are set properly.
• This method is highly accurate only for very narrow questions. It lacks to answer DM’s broad requirements.
• This method works best with a large group of people.
• This method does leave room for bias and manipulation amongst participants.

How-To: (Note: there are many ways of conducting a prediction market, the exercise conducted is just one of many and was based on PredictIt.org’s method)

1. Create a market for the buying and selling of predictions on a specific yes/no question.
2. Over time, as participants buy shares of a particular outcome they become more highly valued and will result in less of a gain if the prediction is correct.
3. The predictions can be bought and sold amongst participants so that the market accurately reflects the current perception of the overall population.
4. As the market continues it will close in on the end result with greater accuracy than the best estimate of an individual.
5. The question must be resolved in a yes or no. The winnings are provided to those who estimated correctly while those who were incorrect lose what they staked.

Personal Application of Technique:

The prediction market exercise consisted of providing estimates on the 2003 NCAA Men’s Basketball tournament.  However, the year and outcomes were concealed from the class until the end of the exercise.

In conducting the exercise, we utilized Google Sheets to allow the analysts to submit their estimates and the percentage which they thought was likely.  They then were given the opportunity to examine the “market,” what other students had estimated, and alter their own estimate.  They submitted estimates on three games, the two semi-final games (Final Four), and the championship game.

After minor changes in the first round, analysts were only allowed to make estimates of 50% or greater, and if correct, would be compensated. We used M&Ms as betting currency and granted compensation with additional candies (Correct estimates: 50% = 0 additional candies; 60% = 1 additional candy; 70% = 2 additional candies; 80% = 3 additional candies; and 90% = 4 additional candies).  Incorrect estimates would forfeit the candies they had put at stake.

The exercise was limited through by a small sample size, an inability to trade/buy/sell other shares, and a simplistic overall scheme which did not reflect a real market.

When employing this version of prediction market, the question asked must be resolvable. In order to use the technique as a teaching tool or as an example as we didn, historically resolved information must be used (like the 2003 NCAA Men’s Basketball tournament). The participants must also not have any knowledge on the data, or means of researching the information.

For additional information:

Prediction Market Spreadsheet

For the exercise PowerPoint, please email the author.

Why Prediction Markets still have a future

"Why Prediction Markets still have a future"
By: John Authers
Financial Times  -  March 11th, 2013

Source: http://www.ft.com/cms/s/0/9e282b2a-8a48-11e2-9da4-00144feabdc0.html#axzz3mL3VZ02b

Summary:

There are numerous prediction markets today, however in 2013 the best prediction market according to this writer for the Financial Times is "Intrade". Unfortunately it closed in 2013 and the author wanted to examine whether prediction markets would continue to grow or whether it would decrease in popularity.

First he explains what prediction markets are. Quite simply, it is a system that allows people to stake bets, both real or imaginary, in predicting an outcome. Th example he gives is sports betting, which allows gamblers to stake bets on both the outcome, point spread, and other factors related to sports. It is also used in the financial world in predicting outcomes. At the time the Wall Street Journal even published the odds related to trading so investors could hedge their bets.

As expected, a common usage of prediction markets is predicting political results. The interesting tidbit of this is that political prediction markets date back to the turn of the century. In the early years of the 20th century, it is estimated that prediction market gambling on political results generated $37 million in bets. It should also be noted that these political prediction markets all outperformed the Gallup poll in predicting the political outcomes. The author does note that while these prediction markets are usually very accurate, they are only as reliable as the information that is available and fed. For example, Intrade wrongly predicted that the US would discover WMDs in Iraq, but as we know in hindsight that was due to faulty information. To put it simply, the author believes prediction markets are too useful to go away.

Critique: 

I think the author could have done a better job in explaining exactly what type of information is fed into a prediction market. The author in my opinion explains it scantly and makes prediction markets appear to be random betting like amateur gamblers at a horse race. A random gambler may take a chance on some stock that he or she feels will be profitable, a true prediction market expert will have reliable information that exceedingly increases the odds in their favor. I do find it interesting that prediction markets are more accurate than the Gallup Poll in politics, but that makes more sense in that the information fed into prediction markets is far more nuanced than random opinions by people, especially in light of our complicated political system. I do think in hindsight he is wrong partly, sports betting has taken off however I did not have any knowledge prior to this class about prediction markets, and in politics the Gallup Poll is still widely used and respected.

Predicting new product success with prediction markets in online communities

By: Kurt Matzler, Christopher Grabher, Jürgen Huber, and Johann Füller

Source: http://eds.b.ebscohost.com/eds/pdfviewer/pdfviewer?sid=fc8cd7f5-3da1-460d-802f-8d40d58bd464%40sessionmgr111&vid=3&hid=103

Summary:

The introduction of this article discusses the history of predicting new product success and compares the pros and cons of the traditionally-used methods to the use of prediction markets. It first describes how there are many methods that are extremely error prone and how it is believed that prediction markets could be a very viable alternative to those methods. This is due to the many cons of the traditional ways as well at the many pros of prediction markets. Some of the many issues with the current methods are that experts are difficult to identify, there are declining survey response rates, consumer resentment is growing, and cost associated with traditional market research are increasing. The pros to using prediction markets include the fact that they can be effective with small and non-representative pools of participants, can efficiently aggregate asymmetrically dispersed information, and have other benefits such as speed, adaptive interactivity, and task engagement.

According to this article, decades of research and studies have proven that the accuracy of prediction markets tends to be much higher than that of other research tools such as polls, questionnaires, and censuses. In addition to high accuracy, this method has a very wide range of applications and has been used to predicts everything from election outcomes in 1988, outcomes of sports competitions, statistical weather forecasts, and many different business events. This article also states that under the right circumstances, groups are often very intelligent (a.k.a. the concept of "wisdom of the crowd") which would support the statement that prediction markets have a high degree of accuracy.

This article also describes an experiment in which prediction markets were tested for accuracy when trying to predict the sale of new skis. In order to do this, 62 participants  used PIM Sports (which was set up like a Facebook application) to predict the sales of skis before the main 2010/2011 skiing season, and then the results were compared to the actual sales. Each person was given $40,000 ($10,000/market, four markets created in total) in virtual money to use in order buy and sell individual assets. The four markets in this scenario were race skis, technology products, the powder segment, and women's skis. In total, the application was open for 12 days. After the 12 days were over, this data was compared to the actual sales and ended up showing that prediction markets are mostly accurate. In this instance, race skis, technology products, and the powder segment had pretty high levels of accuracy while the accuracy for women's skis was fairly low in comparison. The article explained that this was probably due the the fact that liquidity (trade volume) is a key factor in the accuracy of prediction markets, and in this case there was much less liquidity in the women's skis market. Overall, I found this article to be a very interesting read.

Critique:

The main issue that I found with this study was that the PIM Sports application had a total 1,345 users, yet the experiment was only based on the 62 active users. While there was technically a great amount of diversity in total (people from over 50 countries visited the sight), that diversity could have been lacking in the sample of people that were actually used. Basically, even though the results matched those of many other studies, it is hard to say that they were accurate because there was a small sample of people who were used for the results, and because diversity (which is very important for this method) could have been lacking because of the low active participation. Also, like any other experiment there is always room for error so that could always have a possible impact on the accuracy of the results.

"What do prediction markets predict?"

Summary

There has been controversy over the claim that prediction markets elicit ‘aggregate beliefs’, normally understood to mean the average of beliefs for the population. There is no debate over whether these markets generate good predictions, in the sense that they forecast outcomes well. Instead the debate has been over the additional claim that the prices in these markets recover the mean of aggregate beliefs. In other words, are the observed prices in these markets good estimates or predictors of aggregate beliefs?

According to the article, prediction markets cannot always be relied on to elicit any interesting statistic of aggregate beliefs. Formal derivations of the bets placed in prediction markets can be viewed as demands for state contingent commodities.

Prediction markets can be expected to do a good job recovering the average of aggregate beliefs under certain circumstances: unimodal distributions of beliefs, with no a priori reason to expect heterogeneity on either side of the market. Indeed, this environment might characterize many interesting settings, such as political elections or closed prediction markets in which there is minimal sample selection into the market (on the basis of beliefs, preferences and endowments). But the result is not general, and it is easy to construct examples in which prediction markets do a predictably poor job of recovering average beliefs.

Fountain, John, and Glenn W. Harrison. 2011. "What do prediction markets predict?." Applied Economics Letters 18, no. 3: 267-272. Business Source Complete, EBSCOhost (accessed September 18, 2015).

Critique

While I would concur to a cursory extent that the effect that prediction markets would assist with initial predictions, it is imperative to account for the nature of constantly changing variables in this market. In other words, though they would help to create initial models used to create plans, there needs to be a degree of fluidity in the plan built in to account for adaptations said market.