Predicting new product success with prediction markets in online communities

By: Kurt Matzler, Christopher Grabher, Jürgen Huber, and Johann Füller

Source: http://eds.b.ebscohost.com/eds/pdfviewer/pdfviewer?sid=fc8cd7f5-3da1-460d-802f-8d40d58bd464%40sessionmgr111&vid=3&hid=103

Summary:

The introduction of this article discusses the history of predicting new product success and compares the pros and cons of the traditionally-used methods to the use of prediction markets. It first describes how there are many methods that are extremely error prone and how it is believed that prediction markets could be a very viable alternative to those methods. This is due to the many cons of the traditional ways as well at the many pros of prediction markets. Some of the many issues with the current methods are that experts are difficult to identify, there are declining survey response rates, consumer resentment is growing, and cost associated with traditional market research are increasing. The pros to using prediction markets include the fact that they can be effective with small and non-representative pools of participants, can efficiently aggregate asymmetrically dispersed information, and have other benefits such as speed, adaptive interactivity, and task engagement.

According to this article, decades of research and studies have proven that the accuracy of prediction markets tends to be much higher than that of other research tools such as polls, questionnaires, and censuses. In addition to high accuracy, this method has a very wide range of applications and has been used to predicts everything from election outcomes in 1988, outcomes of sports competitions, statistical weather forecasts, and many different business events. This article also states that under the right circumstances, groups are often very intelligent (a.k.a. the concept of "wisdom of the crowd") which would support the statement that prediction markets have a high degree of accuracy.

This article also describes an experiment in which prediction markets were tested for accuracy when trying to predict the sale of new skis. In order to do this, 62 participants  used PIM Sports (which was set up like a Facebook application) to predict the sales of skis before the main 2010/2011 skiing season, and then the results were compared to the actual sales. Each person was given $40,000 ($10,000/market, four markets created in total) in virtual money to use in order buy and sell individual assets. The four markets in this scenario were race skis, technology products, the powder segment, and women's skis. In total, the application was open for 12 days. After the 12 days were over, this data was compared to the actual sales and ended up showing that prediction markets are mostly accurate. In this instance, race skis, technology products, and the powder segment had pretty high levels of accuracy while the accuracy for women's skis was fairly low in comparison. The article explained that this was probably due the the fact that liquidity (trade volume) is a key factor in the accuracy of prediction markets, and in this case there was much less liquidity in the women's skis market. Overall, I found this article to be a very interesting read.

Critique:

The main issue that I found with this study was that the PIM Sports application had a total 1,345 users, yet the experiment was only based on the 62 active users. While there was technically a great amount of diversity in total (people from over 50 countries visited the sight), that diversity could have been lacking in the sample of people that were actually used. Basically, even though the results matched those of many other studies, it is hard to say that they were accurate because there was a small sample of people who were used for the results, and because diversity (which is very important for this method) could have been lacking because of the low active participation. Also, like any other experiment there is always room for error so that could always have a possible impact on the accuracy of the results.

"What do prediction markets predict?"

Summary

There has been controversy over the claim that prediction markets elicit ‘aggregate beliefs’, normally understood to mean the average of beliefs for the population. There is no debate over whether these markets generate good predictions, in the sense that they forecast outcomes well. Instead the debate has been over the additional claim that the prices in these markets recover the mean of aggregate beliefs. In other words, are the observed prices in these markets good estimates or predictors of aggregate beliefs?

According to the article, prediction markets cannot always be relied on to elicit any interesting statistic of aggregate beliefs. Formal derivations of the bets placed in prediction markets can be viewed as demands for state contingent commodities.

Prediction markets can be expected to do a good job recovering the average of aggregate beliefs under certain circumstances: unimodal distributions of beliefs, with no a priori reason to expect heterogeneity on either side of the market. Indeed, this environment might characterize many interesting settings, such as political elections or closed prediction markets in which there is minimal sample selection into the market (on the basis of beliefs, preferences and endowments). But the result is not general, and it is easy to construct examples in which prediction markets do a predictably poor job of recovering average beliefs.

Fountain, John, and Glenn W. Harrison. 2011. "What do prediction markets predict?." Applied Economics Letters 18, no. 3: 267-272. Business Source Complete, EBSCOhost (accessed September 18, 2015).

Critique

While I would concur to a cursory extent that the effect that prediction markets would assist with initial predictions, it is imperative to account for the nature of constantly changing variables in this market. In other words, though they would help to create initial models used to create plans, there needs to be a degree of fluidity in the plan built in to account for adaptations said market.

A Research Agenda for Prediction Markets

Patrick Buckley

Source: http://ceur-ws.org/Vol-1148/paper10.pdf 

Summary:

This article outlines the usage of prediction markets to enable collaborative intelligence by first discussing the general concept of prediction markets and then diving deeper into its methodology, and concludes with a research agenda that can address certain shortcomings. The purpose of prediction markets is to find and aggregate trader (participant) information, and use that information to “make predictions about specific future events”. This method is essentially created by participants buying into one of two outcomes. Prediction markets are different from financial markets in two ways: 

• Prediction markets allow participants to share information and trade contracts with each other
• “Its primary concern is the elicitation of information”

In other words, the shared information and forecast estimation are the main goals of prediction markets, not the financial gain or risk.

There are three broad divisions of operational prediction markets:

• Public prediction markets using real currency
• Public prediction markets using virtual currency
• Private prediction markets

Public prediction markets are open to the general public whereas private prediction markets have a sponsor who recruits participants from a specific group of people.

Strengths:

• Provide participants with incentive to share truthful information

• Provide an algorithm for automatic information communication and aggregation

• Can be conducted on a very large scale with up to hundreds or even thousands of participants

• Can operate over long periods of time

• Provides trader anonymity

Weaknesses:

• Might only attract individuals with certain personality traits (i.e. high risk tolerance individuals)

• Participants may manipulate the system by buying into an outcome that contradicts their truthful information because voting for that option may provide a better incentive

Critique:

Although the strengths are talked about in great detail, the weaknesses are barely mentioned, and there are only a few. Also there could be more detail about the public and private prediction markets and how they operate. This article’s conclusion is that there needs to be more collaborative research on prediction markets in regards to intelligence and decision making.

Prediction Market Accuracy in the Long Run

Summary

Prediction markets are designed and conducted for the primary purpose of aggregating information so that market prices forecast future events. These markets differ from typical, naturally occurring markets in their primary role as a forecasting tool instead of a resource allocation mechanism. In this paper the authors suggest that the prediction markets outperform polls for longer horizons by documenting the evidence from 1988 to 2004 elections. They compare unadjusted market predictions to 964 unadjusted polls over the five Presidential elections since 1988. What they found is intriguing: The market is closer to the eventual outcome 74% of the time. Further, the market significantly outperforms the polls in every election when forecasting more than 100 days in advance.  In their study they utilized the Iowa Electronic Markets (IEM) prices and raw poll dataset. We can compile the merits of predictions markets compared to polls based on the study findings:

•      Polls can`t have the true random sample; whereas the prediction markets customers can be very heterogeneous.
•       Production markets can forecast complex phenomena due to several reasons.

o   The market design forces traders to focus on the specific event of interest more than simple consideration of a fictitious election “if it were to be held today” (as polls ask respondents to consider)

o   Traders must act rationale since they put money on stake.

o   Markets aggregates dynamic information from a wide variety of sources, i.e. traders.

o   The markets provide an incentive to generate, gather and process information across information sources and in a variety of ways. (If you do good, you prosper.)

•       For the five elections, the average absolute error in the market’s prediction of the major-party presidential vote share across the 5 days prior to the election was 1.20 percentage points, while opinion polls conducted during that same time had an average error of 1.62 percentage points.
•       Unlike polls` random selection, the participants of prediction markets are self-selected.
•       Unlike polls or expert panels in which participants are asked for their independent opinions, each trader in the market sees the net effect of the beliefs of all other traders, and the time series of changes in those beliefs and can alter his own perceptions accordingly.
•       Unlike polls that ask each respondent how he or she would vote if the election were held today, the market asks traders to forecast how everyone will vote in the actual upcoming election. (We can suggest that the sentiments play role on polls; whereas the factuality in PMs)
•       ``Convention bounce`` effects don`t appear in prediction markets.
•       It gives continuous updates.
•       Because they react dynamically to information, they can also be used as evaluation tools to assess the impact of decisions such as policy positions, candidate viability, campaign strategies, etc.

Critique

The study lays out the facts effectively. The authors compile the prediction markets` advantages and superiorities over the polls. However, the study does't mention about the weaknesses of the technique which can be the sign of confirmation bias. Moreover, polls attract many attentions currently. If the prediction markets have lots of merits over polls, I would expect the study to identify the reason why still polls make people interested in them. Nevertheless, the study explains very well why the prediction markets outperform polls through five consecutive years.

Source:

Berg, J. E., Nelson, F. D., & Rietz, T. A. (2008). Prediction market accuracy in the long run. International Journal of Forecasting, 24(2), 285-300.