The Problems with Monte Carlo Simulation

Summary and Critique by Keith Robinson Jr.

Summary

David Nawrocki, professor of Finance at Villanova University, examines the failings of Monte Carlo simulation in the financial realm. The author explains that Monte Carlo simulation is only useful in situations where data and analytic models are unavailable; we possess some knowledge about the population, however, sampling data is unavailable. Rubinstein (1981) explains Monte Carlo simulation is appropriate when:

• It is impossible or too expensive to obtain data
• The observed system is too complex
• The analytical solution is difficult to obtain
• It is impossible to validate the mathematical experiment

Further exploring the extant literature at the time, Nawrocki found no articles in support of using Monte Carlo simulation with financial return markets. Additional literature suggested that the while the methodology educates people on uncertainty and risk, it does not reduce uncertainty, instead increasing it because it is derived from assumptions which can lead to incorrect decisions. Implementation is not easy. Nawrocki provides a number of cases in which Monte Carlo simulation is unnecessary or fails. Because Monte Carlo simulation assumes all distributions are normal and correlations are zero, it does not accurately capture the interrelationships between multiple variables contained in historical data, therefore, it does not depict real-world complexities.  

He suggests that Monte Carlo simulation lacks an adequate benefit/cost ratio and provides no demonstrably better answers than other analytic techniques. The researcher argues that if a number is derived from unrealistic assumptions, then they possess no real value. Regarding policy implications, Nawrocki explains that the best policy adapts to uncertain conditions, rather than relying on the most likely course of action produced by the simulation. So, while useful for cases where data and analytic models are unavailable, Monte Carlo simulation requires more work and does not necessarily produce better answers than other analytic techniques.

Critique:

When analyzing Monte Carlo simulation's for use in intelligence, it very well could be a powerful tool for assessing risk, but not necessarily for reducing uncertainty.  Monte Carlo simulations can provide the most likely outcomes, but that does not necessarily reflect real-world scenario. Assuming an adversary/competitor will proceed down a specific path is comparable to mirror imaging bias. While an outcome may be the most likely, one cannot assume all actors are rational actors.

Source: Nawroki, D. (2001). The Problems with Monte Carlo Simulation. Journal Of Financial Planning, 14(11), 92-106.

Citations:

1. R. Y. Rubinstein, R. Y., (1981). Simulation and the Monte Carlo Method. New York: John Wiley and Sons.