Affecting Policy by Manipulating Prediction Markets: Experimental Evidence

Summary
Can prediction markets that are successful at forecasting in the absence of manipulators be corrupted when manipulators actively undermine the prediction markets? An experimental design developed by Deck, Lin, and Porter (2011) provides evidence suggesting that well-funded manipulators who only care about the forecast and exclusively concern themselves with misleading market observers can disrupt a prediction market's ability to aggregate information and mislead those who make forecasts based upon market predictions, effectively eliminating the prediction market's ability to improve a forecaster's performance. However, evidence from the experiment also suggests that the detection of manipulators might be possible due to the trade volume increases and price variance decreases observed. Organizations aspiring to base policy on prediction markets need to condition what the threats of manipulation are for their prediction markets. The experiment models the confrontation between agents attempting to deviously manipulate prediction markets and decision-makers attempting to use prediction markets to guide a course of action. The research objective was to understand how manipulators influence forecasters, not how well the market aggregates information.

Prior research indicated that prediction markets are robust to manipulative attacks and resulting market outcomes improve forecasting accuracy regardless. The profit motive usually proves sufficient to seeing that attempts at manipulating prediction markets are unsuccessful. The authors use the analogy for the now defunct Policy Analysis Markets, if a wealthy terrorist's only motivation is to cause significant loss, his own financial profit or loss does not enter the decision process. This is a separate issue from whether or not prices in prediction markets or asset markets reflect all available information. The limitation of prior research on the manipulability of prediction markets is that the manipulators suffered the financial losses associated with manipulation. The authors assert this is true in any market, but in some cases the relative value of manipulating the market dominates the financial losses associated with attempting to do so. 

This experiment is different from previous market manipulation experiments. Where previous research concerned itself with so called "trade-based" manipulators in financial markets that move price with current trading in order to profit from later trades; this experiment has manipulators who control the event being forecasted but do not want decision-makers to uncover the outcome. For instance, a terrorist makes plans for an attack, but does not want security forces to discover the target. Decision-makers make investments to counter possible attacks. If decision-makers use prediction markets to assist them in detecting terrorist plans and making investments, manipulators would like to mislead decision-makers to make incorrect investments by manipulating market prices. In this experiment, manipulators were not paid in any way for their market earnings. Instead, they were paid solely based upon the average amount that Forecasters invested in the incorrect event. This gave manipulators strong incentive to mislead Forecasters. A critique of previous research on manipulation was that the incentives to mislead were too weak. This schema more accurately models the confrontation between manipulators and decision-makers attempting to use prediction market outcomes to set policy.

Another way the experiment differed from prior research was enabling the Forecasters to have a range of investment opportunities to measure the intensity of their confidence instead of prompting Forecasters to make binary predictions. The implication is that prior research cannot distinguish between a Forecaster who thinks the chance a particular event will occur is 51% and a forecaster that believes the likelihood the event will occur is 90%. 

Experiments conducted at the Economic Science Institute at Chapman University over the course of three days.

 When manipulators are absent, the research found that market prices correlate with the true state and forecasters successfully use price information to make predictions. However, when manipulators are present, the research found that the prediction markets failed to aggregate good information and forecasters consistently failed to predict events. Additionally, manipulator trading increased trade volume compared to markets without manipulators. An unintended finding was that manipulators earned positive profits in almost 70% of the periods in which they were active, which mitigates concerns over financial losses on the part of manipulators. 

The results suggest that manipulators can reduce the predictive power of prediction markets and create situations where Forecasters are unable to make good decisions by actively trading in the markets, which provides a means of identifying the likelihood of manipulator presence. At a statistically significant level, markets with active manipulators had greater trade volume and less variation in prices.

What market information should forecasters use to make a prediction? When manipulators are present only excess bids has predictive power.

 The research identified a case where manipulators cause forecasters to make predictions that are no better than random guessing and concludes that decision-makers should not indiscriminately rely upon prediction markets. An unintended finding was that even though the manipulators were solely motivated by misleading market prices, their strategies resulted in trading profits rather than trading losses, which deviates from prior research from the literature review. 
 

Critique

The prerequisite for successful prediction market manipulation identified in the research is sufficient liquidity to have a measurable impact on trade volume and excess bids. In this particular experiment, the manipulation treatment group had manipulators start with 4 times the amount of experimental currency units of regular traders, this represented one-third of the money in the market overall. How much money aspiring manipulators need relative to the market to be successful in getting forecasters to predict incorrect outcomes is a present gap in the academic literature. 

Source
Affecting Policy by Manipulating Prediction Markets: Experimental Evidence