Use of Prediction Markets to Forecast Infectious Disease Activity

Summary and critique by: Ian Abplanalp

Summary:

In this article researchers took the methodology of prediction markets also known as information markets, or future markets, and applied it to the medical field. The purposed study purposed that a group of medical experts from various field who were privy to different bodies of information could aggregate and accurately forecast what FLU strains would be prevalent in the upcoming year. 

Prediction markets work by having "traders" invest money in outcomes that they believe will come true. As time progresses and more traders invest in differing options, the buy in prices for probable outcomes shift to become more expensive and less expensive to invest in low probability outcomes. These market prices reflect the traders collective confidence that an outcome will happen. Due to the potential to earn money for a correct selection the participant is encouraged to invest in something that will yield a payoff. 

As the more people invest into the market that as time progresses that market will come closer and closer to mirroring what will actually occur in the future. Prediction markets have a very high forecasting accuracy. For example if eighty-five percent of investors suggest this will happen, then that has an eighty-five percent chance of happen. This mimics the principle that the more one flips a coin over time the closer it will be to fifty percent heads and fifty percent tails. 

Three things are highlighted that are key to making a prediction market valid as a methodology. The first is that a prediction market needs a diverse group of traders, to pull information from a large amount of sources. Sufficient amount of traders is also required as there must be enough to encourage market value changes but not so many that there is a large amount of error. The correct number of traders is still up for debate as research has yielded a wide array of results with different amount of traders and there has been no consensus of the issue. The third criteria a prediction market must have is that there must be incentives to trade within the market or it would essentially be a betting pool and not a market. 

When in application 62 medical professionals from various subfields in medicine participated in the market. The market introduced new contracts, which are the equivalent to shares, into the market every two weeks. As the market went on the predictions where more accurate throughout,  both with correctly forecasting the strain color, but also within one color variation (See Figure 1).  The researchers concluded that even though there was some wiggle room within the results that given the volatility of the FLU market it was an overall success. 

Critique:

Prediction markets have inherent strengths of having a malleable method throughout the course of an experiment to accurately represent what is likely to happen in the future. Prediction markets are versatile, as they can be applied to many different fields. They also trump surveys as they allow for informed decisions throughout the entire process by being allowing trade through the process, as opposed to a survey which is filled out once. This allows for confidence in an event, to rise and fall appropriately leading to an overall better forecasting agent. The downfall to prediction markets are they require a great deal of time and effort to set up as well as run. They also require people who care about an outcome to invest into a market with money. This investment raises an ethical question in some fields were it may seem inappropriate to be investing physical money such as medicine, or college sport outcomes. The last pitfall prediction markets have is that they can be swayed by manipulators who are able to purchase large portions of a market if said big buyer wanted to suppress an unpopular opinion to them.

Sources: Use of Prediction Markets to Forecast Infectious Disease Activity

  

Have Corporate Prediction Markets Had Their Heyday?

Summary and critique by Kevin Muvunyi

Summary

Thomas Wolfram in his article “Have Corporate Prediction Markets Had Their Heyday”, examines the reasons behind the relatively low adoption of prediction markets by large business entities as a forecasting tool, and then proceeds to provide possible avenues to revive this important prediction technique. To achieve the purpose of his research, the author reviewed existing literature on the subject and conducted interviews with 32 key business executives.

 In his attempt to understand the contradiction between the strong academic support for prediction markets and its slow uptake by businesses, Wolframm first examines the rationale behind the popularity of this methodology in the world of academia. According to the author, multiple research studies have proven that prediction markets are more efficient in decision making and forecasting, because they eliminate the problematic caused by bias and group pressure in traditional decision making settings by allowing the participants to make their decision anonymously. Furthermore, the technique incentivizes individuals through the promise of possible remuneration to bring forth new information, which can then be aggregated to predict the outcome of a future event, thus, making them more efficient in comparison to long-standing forecasting instruments like questionnaires, surveys, and polls according to the author. 

After examining the merits of the methodology, Wolframm then moves to provide possible explanations to the current withering status of prediction markets in the business world, primarily basing his assertions on interviews conducted with key corporate decision makers. The author summarizes the explanations into three key points as follows:

• Finding appropriate and knowledgeable experts (traders) is complicated; it does not help if participants are diverse but ignorant of the issue as it undermines their predictions

• Lack of trust from the top echelons of management: management trusts consulting groups more than own employees, and also believes that prediction markets disturb the concept of hierarchy in an organization due to the fact that their ideas have a chance of being rejected based on this model, thus, undermining their authority.

•  Businesses “going digital” in a different direction from prediction markets; as businesses turn digital prediction markets become obsolete, because everything becomes data driven, therefore, big data analysis and other similar techniques are more insightful and appropriate.

In the light of apparent insurmountable obstacles to the revival of prediction markets in corporate circles, Wolframm advocates that they are possible ways to remediate this issue. For example, the author suggests that to ensure knowledgeable traders, a convenient graphical user interface and communication exercises accompanying prediction market implementations would be more appropriate. Furthermore, Wolframm brings forth the notion of idea markets as an innovative approach to the forecasting tool. In contrast to traditional prediction market whereby participants are allowed to trade on the outcome of uncertain future events, idea markets could provide a platform for the generation and assessment of ideas through the trading of virtual stocks representing products and concepts.

Critique:

The author does a great job at analyzing the root cause of disaffection of prediction markets as a forecasting technique in the corporate world. He also equally provides sound alternatives to revive the technique. Nonetheless, the only problem with his article is that it fails to demonstrate in a tangible manner, the superiority of improved prediction markets in comparison to big data analytical tools in the new data driven world that we live in today.

source: http://eds.a.ebscohost.com.ezproxy.mercyhurst.edu/ehost/pdfviewer/pdfviewer?vid=10&sid=4bed5a56-fc91-4969-aac9-62fa90b8808f%40sessionmgr4008   

The Power of Prediction Markets

Prediction markets can be uncannily accurate — sometimes. Scientists have begun to understand why they work, and how they can fail.

Summary and Critique by Oddinigwe Onyemenem

Summary

In 2012, a group of international psychologists embarked on a project dubbed the “Reproducibility Project” in an effort to repeat dozens of psychology experiments to see which held up. One of the participants, Ann Dreber, who leads a team of behavioral economists at the Stockholm School of Economics, viewed it as an avenue to mix science with gambling and thought it would be fantastic to bet on the outcome. The team was specifically interested to see whether scientists could make good use of prediction markets: mini Wall Streets in which participants buy and sell ‘shares’ in a future event at a price that reflects their collective wisdom about the chance of the event happening. As a control, Dreber and her colleagues first asked a group of psychologists to estimate the odds of replication for each study on the project’s list. Then the researchers set up a prediction market for each study, and gave the same psychologists USD 100 apiece to invest. In 2015, the project had replicated fewer than half of the studies examined and Dreber found that her experts hadn’t done much better than chance with their individual predictions. But working collectively through the markets, they had correctly guessed the outcome 71% of the time. According to Mann, experiments of this nature depict the power of prediction markets to turn individuals’ guesses into forecasts of sometimes startling accuracy.

Mann points out that prediction markets are increasingly being used to make various kinds of forecasts such as the outcomes of sporting events and business decisions. Prediction markets advocates claim that it allows people to aggregate information without the biases that affect traditional forecasting methods such as polls or expert analysis. The application in science was shown by Dreber and her team by giving researchers a fast and low-cost way to identify potential problems with replicated studies. On the other hand, skeptics point out that prediction markets are far from being perfect. This is due to an incorrect notion that a great prediction is almost, always guaranteed when a market is set up. According to Eric Zitzewitz, an economist at Dartmouth College, it is an area of active research to determine the best designs for prediction markets and the limitations. Nevertheless, advocates of prediction markets argue that even imperfect forecasts can be beneficial. For instance, hearing there’s an 80 or 90% chance of rain can make an individual take an umbrella.

The prediction-market idea was revived by the spread of the Internet, which dramatically lowered the entry barriers for creating and participating in prediction markets. In 1988, the University of Iowa’s Tippie College of Business launched the not-for-profit Iowa Electronics Market (IEM) as a network- based teaching and research tool. Over the years, IEM has set up several markets to predict election outcomes which a 2008 study found that the its predictions across five presidential elections were more accurate than the polls 74% of the time. The success of the IEM helped to inspire the creation of dozens of other prediction markets.

The article also addresses the fact that prediction markets have also missed the mark by a long shot in several cases. In the Brexit case, the prediction market gave the odds of a stay vote as 85% on the day of the referendum, whereas the outcome was narrowly in favor to leave. Also, prediction markets were off the mark in predicting the outcome of the 2016 US presidential election, which elected Donald Trump instead of the highly-favored to win, Hillary Clinton. These sorts of instances have caused academics to probe prediction markets about why they work so well, their limits and reasons for failures? Mann points out that if prediction markets offer a way to update guesses considering new information, they will do as well or better than other forecasting methods. Prediction markets in general still need to deal with challenges such as how to limit manipulation and overcome biases.

Critique

As rightly stated in the article, prediction markets are used in a wide array of markets such as sports, politics, movie, business, technology, etc. While some fully embrace it, others remain highly skeptical due to the possibility of manipulation or infusion of biases.  Research into improving it still needs to be done to ensure, as best as possible, the integrity of the process. In the next decade, prediction markets can be a major driving force behind significantly improving decision-making as more tools are implemented.

Source:

https://www.nature.com/polopoly_fs/1.20820!/menu/main/topColumns/topLeftColumn/pdf/538308a.pdf

Summary of Findings: Wargaming (3.75 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in October 2017 regarding Wargaming as an Analytic Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Wargaming, when done correctly, can provide a range of reasonably accurate forecasts and , in these conditions, is a best considered an analytic method. When the simulation is not validated or has been simplified for the purposes of playability, wargaming is best thought of as a analytic modifier designed to teach the basic elements of a conflict without actually being used to predict outcomes.  It can and has been used in military, law enforcement, and business. It is used by the military to recreate historic battles or simulate future conflicts in order to prepare for all possible contingencies before an actual war begins. Similarly, teams within businesses play as their company and the competition too. Teams are made to reflect all the other competitors that affect a given industry to simulate a number of possible outcomes. This can take into account numerous aspect of business such as strategy, marketing, and finance.

Strengths:

• When executed correctly, Wargaming can simulate realistic conditions
• Its structure allows for replicability of the simulation
• Can create insight in possible avenues of attack/approach not previously considered
• Able to identify unforeseen obstacles in current strategy, procedures, or tactics
• Gives alternative perspectives for evaluating scenarios
• Provides accurate, holistic depiction of active, dynamic competition

Weaknesses:

• Requires extensive lead-time to prepare (game design, playtesting)
• Difficult to balance realism vs playability
• Realism is often difficult to achieve
• Susceptible to over-confidence or systematic bias
• Often complex, costly, and time-consuming
• Validity can be an issue; bad information can give false hope
• Can’t plan for unexpected variables/events

How-To:

1. Choose a situation or environment for which to simulate in a Wargame
2. Design the simulation to encapsulate all possible factors within the environment
3. Run the simulation until complete
4. Compare and analyze the results toward reality
5. Re-run the simulation to increase confidence level in results
6. Reevaluate factors and re-run as necessary

Application of Technique:

The class received handouts of grid paper that had obstacles courses outlined on them. The class was then divided in half so that each person had a partner to participate in the activity against. The class then was instructed on the rules of how they were to move through the obstacle course, with the objective to make it through the course as quickly as possible and ahead of their opponent. The rules were that each person was assigned an identifier (Shape on the grid as a game piece) that they were able to move either horizontally or vertically in a given turn. Each player was able to accelerate their piece by one block per turn so they could gain speed, the same principle was applied to deceleration. If a player hit an object or a barrier in the course the player's momentum would drop to zero. The same principle applied to if players pieces were to come in contact with each both players momentum would drop to zero.

For Further Information:

1. Wargaming - Wiki
2. Business War Games - Wiki
3. Pathologies of Wargaming

Why Wargaming Works

Summary:
by Matthew Haines

       Dr. Perla and Dr. McGrady outline what wargaming is and how it can be both successful and unsuccessful in its attempt to inform and instruct its players. They both agree that wargaming has a significant impact on its players’ decisions outside of the simulation and they propose a combination of reasons for this. They begin by comparing a games narrative to that of literature and the effect it has on a reader. That while reading prose the reader builds an imaginary space, that encompasses the work of fiction, but is perceived as real in the moment. This literary term is called the l’entre deux, or the “between place”. This phenomenon is what ties the emotional response a reader has towards problems that a fictional narrative is proposing. The author’s use President Clinton’s fear of the repercussions of biological terrorism.
Dr. Perla and Dr. McGrady then outline the neuroscience behind this idea. That when reading a fictional work that applies suspense and emotion to otherwise historical facts, a person must pause and remember what is factual and what is not. They site a study done on participants who were given a factual “cut and dry” recounting of President George Washington’s campaign, and a narrative that painted a scenario that the race was down to the wire. After the study participants were asked if George Washington became the first president of the United States, and the participants that read the “cut and dry” piece answered significantly quicker than those that read the other piece. The authors state that this place between is heightened even further in wargaming.
They state that wargaming does not only create a narrative designed to build emotion and suspense, but it is also influenced by the players’ actions. Therefore, wargaming is the closest place that a person can get to the “between space”, thus intensifying the effects it has on the players. The author’s state that these effects can be both good or bad depending on the design of the game. A well designed game can be used to
       
        help players learn how better to balance the equation between the cost of preparing for the
        uncertain future and the risk of not doing so; can help enlighten players about the fact that
        unexpected and unpredictable events, including embarrassing ones, do happen and that there are
        real consequences when they do.


However, a poorly designed game can under or over estimate the effects variables have on an outcome, and create a false sense of reality for the player. Wargaming is also hindered by its inability to account for unknown unknowns. This can make it extremely ineffective when dealing with problems that are outside the scope of a game designer’s cognitive biases.

Critique:

        What this article shows is that if wargaming can even be described as an analytical/forecasting tool it is a very poor and dangerous one. The authors were able to show that wargaming and simulation are one of the best ways to make the problems simulated a top priority for participants. This is just another way of creating a bias. To use these simulations, as described above, for intelligence purposes is a bad idea. More biases do not allow for best possible intelligence products. That said the article does highlight some good points for using wargaming as a teaching tool. If wargaming is used in conjunction with brainstorming processes, decision trees, and unique factors then it could be used in a more effective way.

Source:

http://eds.b.ebscohost.com/eds/pdfviewer/pdfviewer?vid=2&sid=0cb67813-c777-4daa-a98f-8b2af3c1a712%40sessionmgr101 

Wargaming: Training, Educational Tool for the Future

Colonel Thomas M. Lafleur examines the use of wargaming as a strategic  tool. One of the first questions he posed was the ability to transfer results of a war game conducted in earlier years to a later time period. Later in the study, he found elements of maneuvers in war games can be transferred to any strategy in later periods, but if the gaps between time periods is large enough, the maneuvers themselves cannot be due to changes in circumstances.

To ensure a complete transference of a war game to strategy, Colonel Lafleur presents three things to be done: a nuanced scenario must be in place. The nuances mentioned are a realistic scenario with leaders who know the scenario extremely well. Second, the participants in the war game must have specific and detailed knowledge of how to proceed in the war game as well as skill sets relating to the war game. Third, examine the qualities developed out of the scenario as possible solutions for future problems.

Critique:
Colonel Lafleur does a good job examining the ability to transfer strategies developed in war games to actual military strategies. He goes into detail on what is needed for an effective war game.

Citation: Lafleur, Thomas M. http://www.arcic.army.mil/App_Documents/UQ/Wargaming.pdf.

DYNAMIC COMPETITIVE SIMULATION: WARGAMING AS A STRATEGIC TOOL

Summary and Critique by: Jared Leets

Summary:

The authors, Treat, Thibault, and Asin, begin by explaining what wargaming is in business. In a wargame, teams of the managers from a company role play their own company, their top competitors, and the marketplace. Then a control team plays all other competitors that can affect the industry. This simulates possible real life business outcomes and can provide guidance for the company’s future strategies. Teams lay out objectives, decide on what strategies to take, and what and where to invest. Teams from the company, such as the market team, will review market reactions and decide where to move next. In addition, the finance team can provide feedback and tell teams, taking part in the wargame, where the profits and losses would occur if the company goes in a certain direction.

The authors describe ideal situations for using wargaming for competitive intelligence. The first situation occurs when there is a competitive dynamic in a certain industry. The second situation deals with the market reaction and its unpredictability due to constant change, emergence of new technology, or a change in market demands. Typically a deterministic model will not be able to predict that and will likely be useless. Another ideal situation would be when simulations are the only viable way of gaining insight, when not much information is available of the competitor, or when there exists an excessive amount of dimensions to the problem.

Next they describe how traditional planning does not work as well as wargaming. According to the authors, wargaming works because it confronts concerns about planning in isolation and dealing with discontinuities, and it is a holistic enterprise. Strategy can be easily dismantled, if one removes one element from the strategic plan it can fall apart and then all that is left are tactics which simply relate to the execution of the strategy. That is the main difference between wargaming and traditional planning is that wargaming demands a team to look at the entirety of the plan, while traditional planning has the team look at certains parts to the plan. Wargaming simulations are able to complete paradigm shifts and promote "out of the box thinking" due to the fact that it investigates the implications of changes in strategy with no real negative risks involved. Usually any typical strategic analysis will not work because it will interpret what happened in the past and use that to forecast the future. In this case the past never truly repeats itself. Scenario planning uses historical planning to predict future outcomes and analysis cannot predict how a competitor will react to changing conditions, for example with a new product being released. The authors state that scenarios tend to be guesses at the future which makes it relatively easy to become biased. 

They conclude by stating how wargaming is much more beneficial than traditional planning. Wargaming improves strategic capability thinking for managers in a company and the feedback from those who have participated claim that it forced them to keep strategic issues at the forefront. Those that partook in the wargaming simulation focused more on their competitors plans and how to counter them.

Critique:

Treat, Thibault, and Asin explain, in great detail, what wargaming is in the business world and how it can be effectively used. Their description of it and what the ideal situations are for when a company should use it were phenomenal and they went in depth comparing it to other traditional forms of planning. Not only did they explain why traditional planning does not stack up well against wargaming, but they provided quality examples to help comprehend it all. While they did an impressive job of telling why wargaming simulation is much better for businesses, they did not do enough to explain what the potential drawbacks of using wargaming might be. Overall the article argued its point well.

Source: Treat, J. E., Thibault, G. E., & Asin, A. (1996). Dynamic competitive simulation: wargaming as a strategic tool. Strategy, Management, Competition, (Second Quarter), 46-54. http://hershbine.net/wp-content/uploads/20-DYNAMIC-COMPETITIVE-SIMULATION-WARGAMING-AS-A-STRATEGIC-TOOL.pdf

 

Ex Ante Strategy Evaluation: The Case for Business Wargaming

By: Michael Pouch

Summary:

The purpose of Jan Schwartz research is to introduce wargaming as a useful tool for ex-ante strategy evaluation for the business world. He lists some important benefits of business wargaming as an ex-ante tool such as assessing a particular strategy, by simulating reality, the consequences with respect to the organizational environment (especially its competitors), and with respect to the future impact on the strategy and the future of that strategy.

The author begins to lay out several advantages of using wargaming in the business world to test a particular strategy. First, while using participants, wargaming provides not only the use analyzation of a market or a competitor but the simulation an industry, market, or competition over time. This so-called experience or simulation provides a preview of how a strategy might fare.

Second, Schwartz provides an argument that wargaming can be used to challenge the mental models of participants. Wargaming allows participants to think and act like their competitors, thereby forecasting what their competitors will likely do. In addition, war-gaming provides a simplified version of strategic planning that allows participants to test strategies but a form strategic planning.

Third, war-gaming can offer a means of fostering organizational learning. When the comparison between situations or moves within the business wargame or with mental models become communal the knowledge derived from a business wargame will be transferred when a similarly structured situation occurs outside the simulation.

Fourth and last, the intense competition that is involved within war-gaming, forces participants to rigorously examine the situation from several perspectives. The benefits that come along with the competition gives the participants the ability to convey a deeper understanding of the competitive situation and an awareness of the participant’s or a team’s strategy will play out, and especially how the industry will develop.

On the whole, a well-designed and conducted business wargame is important to test of strategy that sheds light on competitor’s motives and actions and ultimately see your strategies play out but also the likely results when implemented.  As shown in figure 1, wargaming provides many benefits were as other methods are limited to certain actions. 

 Figure 1: Characteristics of Business Wargaming

Critique:

Overall, the advantages that come with wargaming in business outweigh the limitations.  Although wargaming takes up a lot of resources, preparation, and participant demand, the results allow you to come out of the simulation with many external and internal perspectives in mind. Another limitation implies a human element where the simulation might not have the right participants or the participants willingness to engage in the exercise. By and large, it is an essential exercise that needs to be designed, prepared, and well-executed for a successful simulation.

Reference:

Schwarz, Jan Oliver. (2011). Ex Ante Strategy Evaluation: The case for Business Wargaming. Business Strategy Series. 12. 122-135.

The Do’s and Don’ts of Course of Action (COA) Wargaming

This article sets COA Wargaming in context, describes its aim and – briefly – the processes involved, and shows when it is best used. Most importantly, it suggests some do’s and don’ts that have been shown to significantly increase the benefits derived from COA Wargaming. 

According to the article, there is no formal definition or aim for COA wargaming. Although there have been several attempts to define it, those definitions are lacking in precision. It defines wargaming as a systematic method of analyzing a plan in a conscious attempt to visualize the ebb and flow of an operation or battle. A Wargame is a staff tool designed to visualize the battlefield and the possible interaction between opposing forces. By wargaming, commanders and staffs attempt to foresee the dynamics of action, reaction, and possible counteraction of battle.

COA wargaming is one instance in the professional wargaming tool-set. COA wargaming is primarily characterized by conflict or opposition to fully test the forming plan. Also, another important characteristic is that COA Wargaming will, almost certainly, be carried out under significant time pressure; this impacts how and when it is best conducted.

The paper continues to answer important questions like the aim of wargaming, characteristics, inputs, outputs, methods, participant, and when it should occur. It focuses on do’s and don’ts of the wargaming. 

The paper concludes that with well executed COA Wargaming can significantly enhance the decision-making process. If used, the do’s and don’ts suggested, it would go a long way in ensuring that the potential benefits offered by COA Wargaming are realized.

Critique: This article provides really good information regarding COA wargaming. Having a good understanding of COA wargaming is crucial in developing sound decision-making skills and good judgement. The ability to visualize possible scenarios in wargaming is a vital aspect to this process. It answers a lot of “what ifs” and helps for better preparedness – in learning to formulate a viable plan, acting, and reacting. It’s almost like a simulated foreshadowing of the battle and creates immense advantages that should not be overlooked. 

The application of wargaming concept in the ever-changing business world has huge benefits. It can help to anticipate competitors reactions and strategies in response to changes that may occur over the course of time.

Link: http://lbsconsultancy.co.uk/wp-content/uploads/2011/03/The-dos-and-donts-of-COA-Wargaming.pdf

Improving Operational Wargaming: It’s All Fun and Games Until Someone Loses a War

Summary and Critique by Claude Bingham

Summary 

As part of an initiative to reinvent wargaming in the Department of Defense, Lt. Col. Matthew E. Hanson wrote this monograph to explore the strengths and pitfalls of wargaming. Wargames have ten pathologies, or elements of a wargame that are subject to failure. They are: objectives, scenario, database, models, rules and procedures, infrastructure, participants (including players, controllers, and observers), analysis, culture and environment, and audiences.

Any one, or multiple, of these can fail to be designed correctly, balanced properly, or evaluated accurately. Wargames are often underspecified in their inclusion of existing doctrine into game design and inconvenient outcomes can be rejected or disregarded. The most important idea from this study is that military doctrine often diverges from wargaming and neglects to integrate lessons learned into living doctrine.

Game Element Analysis, or GEA and wargame failure modes can be used to develop wargames and evaluate wargame outcomes and doctrine. Instead of choosing the United States' plan for Midway, Lt. Col. Hanson chose Japan's because of evident failures to evaluate results of their wargame plan. Hanson explains wargaming as a "synthetic experience" to test decision-making and strategy in an environment with limited information. He also states that a good wargame not only proves or disproves the effectiveness of evaluated strategies, but also reveals flaws and gaps in those strategies. A bad wargame will fail to push a strategy far enough.

Japan's loss at the Battle of Midway was not only affected by poor wargaming, but also Japan's early naval successes during WWII. Planning suffered because of an unwillingness to challenge that momentum procedurally. Additionally, wargame outcomes that were seen as undesirable were completely disregarded. This confidence led to a wargame scenario that left Midway's attacking battle group underpowered and a verbal order to keep bombers in reserve was unheaded.


Critique 

This study did an excellent job showing both the value and pitfalls of a wargame scenario. It also used quotes from Japanese officers that speak to their realization of what went wrong. Like in most cases, wargames are affected greatly by human error, especially that which underestimates the ability of an adversary to fight back. I feel that this research was missing an important piece, however. It does not recommend absolute dedication to recording all procedural discussions during a wargame's development, Course-of-Action, or post-mortem. That appears to be a must. No stone can be left unturned.