B

Types of Uncertainty

There are two distinct types of uncertainty that analysts must address and communicate—uncertainty due to lack of knowledge and that due to inherent randomness (Apostolakis 1990).

The clearest example of inherent randomness (referred to as aleatory uncertainty, from the Latin alea meaning “dice game”) can be illustrated by the flip of a coin or the roll of dice. Even if the coin or dice are fair, and fairly tossed, the exact result each time is inherently random, and not reducible by further study. In the context of nuclear war or terrorism, an example of this random uncertainty is the possibility of a false alarm from a detector. Even if a good estimate of the rate of false alarms is known, whether one will occur on a given day is inherently random. That uncertainty cannot be reduced on the spot (and therefore, at that precise time, reflects randomness) but can be addressed later by more testing of the detector.

Systematic uncertainty due to lack of knowledge (referred to as epistemic uncertainty, from the Greek episteme meaning “knowledge”) can, in principle, be refined (increased or reduced, even if not eliminated) through research if the basis for this uncertainty is recognized and better understood. Decision makers rely on the intelligence community to refine systematic, epistemic uncertainty about the goals, motivations, and capabilities of potential attackers. Their examination of the signals may result in either an increase or a decrease of uncertainties, which in either case must be properly communicated to the decision maker. More generally, decision makers rely on the broader research community to refine understanding of systematic uncertainties about natural, human, and engineered processes.

For systematic uncertainty, the sources of uncertainty need to be recognized or at least anticipated in order to be studied and—once understood—potentially adjusted. The intelligence community often requires at least two independent means of characterization of a phenomenon—for example, “dual phenomenology” in documenting an attack. But unless the sources of uncertainty are fully understood, the messages may not be entirely independent, and in that case are still subject to systematic error. Biases, unrecognized assumptions, and cultural perspectives are among the common sources of systematic biases, which include both systematic errors in measurement and lapses due to conceptual uncertainty (i.e., relevant variables are not even being considered, let alone documented). In addition, pressures to reduce uncertainties may lead to catastrophic misinformation.

In practice, of course, both epistemic and aleatory uncertainties typically come into play in a single decision or question, and can both be addressed by Bayesian probability. For example, there may be systematic uncertainty about the rate of false alarms from a particular detection technology, and that uncertainty might be addressed through experimental research, but randomness still exists about the timing of a false alarm, even after research has been done to address the level of epistemic uncertainty. In some cases, aleatory uncertainty cannot be reduced; for example, one cannot change dice (unless by cheating and loading them), but one can improve the quality of a sensor and one’s understanding of its frequency of errors.

The distinction between systematic and random uncertainty is not just philosophical—it is important for decision making (Bier and Lin 2013). If a particular decision is difficult because of a recognized source of epistemic uncertainty, further research or intelligence gathering might be desirable before making a final decision (assuming that time allows and the cost of information gathering is not prohibitive). If a decision is anticipated to be difficult because of aleatory uncertainty, further testing is needed to test the sensors to better qualify the uncertainty, which it may reduce or increase. The primary way to improve the effects of that uncertainty would be to switch to a more reliable or independent detection technology. Still, more or better information does not necessarily reduce uncertainties. In fact, it may increase it by uncovering previously unrecognized sources of uncertainty, and that may be critical to a better decision.