Monday, October 13, 2014

Summary of Findings: Prediction Markets (3.5 out of 5 stars)

Summary of Findings: Prediction Markets (3.5 out of 5 stars)


Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 5 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in October 2014  regarding Prediction Markets specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.


Description:
A prediction market is an analytic method in which participants buy and sell estimates based on the  probability they assess an event to have.  For example, a participant may pay 45 cents for a prediction stock of an event that they believe has a 45 percent chance of occurring.  There are also prediction markets that do not have the ‘stock market’ element.  The estimates, regardless of the type of prediction market used, are aggregated to create a more accurate estimate of a specific event.


Strengths:
1. Prediction markets have been used successfully across multiple fields (economics, finance, intelligence)
2. Can incorporate insight from experts across many different fields
3. Output can come out as a single point or a range of values
4. Similar structure to that of Nominal Group Technique (NGT)
Weaknesses:
1. Prediction markets require a large number of analysts to create the number of estimates needed
2. Constant traffic is required within a prediction market to create the volume of estimates needed
3. The integrity of the prediction market is susceptible to manipulators
4. The purpose of the prediction market must be to create to accurate estimates
5. Long term estimates are at risk of forecaster apathy
6. Some modest level of expertise is required to be a forecaster


Step by Step:  
  1. Define a question resolvable by prediction markets
  2. Design a prediction market to reduce uncertainty about future outcomes by aggregating individual estimates within a predetermined time frame
  3. Define rules of the prediction market constraining participant behavior and applicable payouts
  4. Open prediction market to participants
  5. Readjust prediction market as new information comes to light
  6. Close prediction market at predetermined date and declare winners
  7. Use output of prediction market to support decision-making or feed into another technique


Exercise:
Participants were provided a jar filled with a predetermined amount of tootsie rolls only known by the administrator to make an estimate on the amount of candy in the jar. The participants were not allowed to talk to other participants during the exercise and were not allowed to open the jar. Participants were allowed to use the computer for ten minutes in any way to help them form their estimates. After the ten minutes, the participants wrote down their one estimate on a post-it note and turned it faced down. The proctor came around and collected all of the estimates and displayed them on the board and took the average. The average of the participants was closer to the real estimate then individually.

What did we learn from the Prediction Market Exercise
After researching this topic, students were able to learn the methodology of prediction markets. Participants of this exercise were able to use various approaches to develop estimates. The average of the participants responses were closer to the actual number of tootsie rolls in the jars.  

3 comments:

  1. Have been a forecaster for about two years. It is my second job, at least that is how I see it. As far as the weaknesses mentioned in the post I would like to comment.

    1)The group I work within has three thousand forecasters, we are large.

    2) Traffic is constant, daily review for me is pretty much the normal. It takes huge amounts of time to access open source on a worldwide basis. I know I spend twenty plus hours a week, most of the time much more.

    3)There clearly are manipulators, one has to do your own work, they are easily observed and identified.

    4) We have been told we are doing a great job, the idea of group intelligence works. The numbers we see speak volumes. Yes , black swans happen, but the occurrence is limited.

    5) The best of us stay with it, people do get burned out, but the structure of the program spurs us on to continue.

    6) I have thirty years of experience in analysis, it definitely helps. The goal always has been to have a diverse group in all aspects to limit any bias. That approach seems to be working.

    ReplyDelete
  2. Replies
    1. Hi Joy, figured you would like to hear from a forecaster directly on this, hope you have great success in your studies, let me know if there is anything else I could provide as first-hand account, or maybe just answer some questions. Your call. Anyway, good luck!

      Delete