Use of Monte Carlo Simulation in EPA Risk Assessments - Dr. Roy L. Smith

Summary and Critique by Evan Garfield

Summary
The author first notes that the EPA's current risk assessment methods express health risks as single numerical variables, or single-point estimates of risk. He argues that this technique provides little information about uncertainty and variability surrounding the risk estimate. These single-point estimates are accompanied by a qualitative discussion of uncertainty. The public tends to focus on the single-point estimate and to overlook the uncertainty, which may span several orders of magnitude. EPA risk managers, though aware of the uncertainty, objectively justify their decision to either accept or reduce the single-point risk. Accordingly, the author argues single-point assessment methods place the risk assessor in an inappropriate risk management role.

In response, he suggests developing "multiple descriptors" of risk using Monte Carlo simulations to provide more complete information to decision makers and the public. Monte Carlo simulations involve determining the impact of identified risks by running simulations to identify the range of possible outcomes for a number of scenarios. A random sampling is performed by using uncertain risk variable inputs to generate the range of outcomes with a confidence measure for each outcome. When Monte Carlo simulations are applied to risk assessment, risk appears as a frequency distribution graph rather than a single-point estimate.

The author continues to discuss some limitations of Monte Carlo Simulations with regards to EPA risk assessment listed below.

1. Available software cannot distinguish between variability and uncertainty.
2. Ignoring correlations among exposure variables can bias Monte Carlo calculations.
3. Exposure factors developed from short-term studies with large populations may not accurately represent long-term conditions in small populations
4. The tails of Monte Carlo risk distributions, which are of greatest regulatory interest, are very sensitive to the shape of the input distributions.

Despite these limitations, the author argues Monte Carlo simulations are superior to the qualitative procedures currently used to analyze uncertainty and variability, providing an example.

At a Superfund site, volatile organic compounds migrated to residential wells. The single-point estimate of lifetime cancer risk to exposed residents, based on ingestion of tap water and inhalation while showering, was 1.14e-3. Risk was then calculated 5000 times, with each calculation based on a different randomly-selected exposure scenario. The risk estimate fell between the 95th and 99th percentiles in this example. This frequency distribution provided more complete risk information than the single-point estimate.

Critique
This article provides support for integrating Monte Carlo Simulations into risk assessment. Despite some limitations, this risk assessment method remains superior to qualitative procedures currently used to analyze uncertainty and variability. For baseline assessments, uncertainty and variability surrounding single-point estimates should rely on multiple descriptors of risk. Nonetheless, it would be interesting to study the risk assessment effectiveness of of Monte Carlo simulations vs. traditional qualitative procedures used to analyze uncertainty and variability.

The author mentions that a limitation of Monte Carlo Simulations is that current software cannot distinguish between variability and uncertainty. It must be noted that this article was produced in 1994. Accordingly, this may no longer be a limitation as computer software has advanced substantially since then.


Source: https://www.epa.gov/risk/use-monte-carlo-simulation-risk-assessments