Comparing Uncertainty Visualizations for a Dynamic Decision-Making Task

Summary

This research compared various visual representations to express uncertainty. Additionally, this research compared graphical representations of uncertainty against numerical representations. Bisantz et al. hypothesized that graphical representations of uncertainty are superior to doing so numerically.

The study performed had 24 participants, aged 19-32, participate in a Missile Game. Bisantz separated the participants into two experimental groups: one with just graphical representation and one with graphical and numeric representation. During this exercise, participants were charged with identifying missile icons amongst bird and plane icons in order to eliminate the threat. Participants had between 5 and 20 seconds to label an icon as a missile or not. There were four different methods for displaying the icons:

1. Most likely solid: The icon of the outcome that is most likely to occur is displayed
2. Most likely transparent: The icon of the outcome that is most likely to occur is displayed but its uncertainty is displayed by how transparent it is.
3. Missile transparent: Only the missile icon shows with its uncertainty displayed by its transparency
4. Toggle: Participants can switch between the three methods

Each participant completed two trials using each of the four methods for a total of 8 attempts.

Figure 1. Overall score by graphical representation against numeric representation

The result of the study concluded that participants scored better with the inclusion of numeric representation. Of the three methods for displaying uncertainty, Most likely transparent resulted in the highest scores. The use of numeric representation also resulted in a shorter time duration for making decisions.

Figure 2: Distance from endpoint when decision was made

Critique

Bisantz designed to compare the three methods for graphically representing uncertainty; however, there was not an experimental group to compare between just graphic and numeric representation. A slight tweak to the experimental design would have provided insight into whether visualization is needed at all.

Source
Bisantz, A., Cao, D., Jenkins, M., Farry, M., Roth, E., Potter, S. & Pfautz, J. (2011). Comparing uncertainty visualizations for a dynamic decision-making task. Journal of Cognitive Engineering and Decision Making, 5(3).