Dealing with uncertainty in analyzing decisions

---

Lecture 03

September 22, 2025

Overview

• Uncertainty in Earth systems modeling
• Neglecting uncertainty can bias decisions
• Uncertainty characterization and sensitivity analysis
• Uncertainty analysis choices follow from problem framing
• Upcoming Schedule

Uncertainty in Earth systems modeling

What is uncertainty?

A lack of certainty; an inability to exactly describe current or future states and values

Parphrased from the Wikipedia entry on Uncertainty

Two overarching types of uncertainty

Aleatoric uncertainty results from randomness

Epistemic uncertainty results from lack of knowledge

Fundamental limits to Earth systems modeling

Earth systems are open, posing fundamental limitations on our ability to reduce all relevant areas of epistemic uncertainty.

Verification and validation of numerical models of natural systems is impossible… because natural systems are never closed and because model results are always nonunique…

Thus, the primary value of models is heuristic: Models are representations, useful for guiding further study but not susceptible to proof.

— Oreskes, N., Shrader-Frechette, K., & Belitz, K. (1994). Verification, validation, and confirmation of numerical models in the earth sciences. Science, 263(5147), 641-646.

Example: Climate scenario uncertainty is often called “deep”

Hausfather and Peters (2020). RCP8.5 is a problematic scenario for near-term emissions

Example: Scenario and model choices reflect “deep” uncertainty in sea-level rise projections

Doss-Gollin and Keller (2023). A subjective Bayesian framework for synthesizing deep uncertainties in climate risk management

Neglecting uncertainty can bias decisions

Example: Triggering managed retreat

Hegde et al., (2025). Timing managed retreat for robust coastal adaptation strategies. Preprint.

Myopic strategies

Hegde et al., (2025). Timing managed retreat for robust coastal adaptation strategies.. Preprint

Uncertainty characterization and sensitivity analysis

Schematic Representation

Reed et al., (2025). Addressing Uncertainty in multisector dynamics research

Many purposes - a few highlights for decision analyses

• How can we simplify our model to run it faster and sample more scenarios?
• What are worst-case scenarios?
• Which actions perform well in a wide range of scenarios?
• What causes actions to fail?
• What types of observations should we collect to improve our model and better evaluate actions?

Note

Note the iterative nature of using uncertainty characterization and sensitivity analysis to inform and assess strategies

Types of analyses

Reed et al., (2025). Addressing Uncertainty in multisector dynamics research

Analysis methods

Reed et al., (2025). Addressing Uncertainty in multisector dynamics research

Revisiting the managed retreat example

Hegde et al., (2025). Timing managed retreat for robust coastal adaptation strategies.. Preprint

Hegde et al., (2025). Timing managed retreat for robust coastal adaptation strategies.. Preprint

Uncertainty analysis choices follow from problem framing

Example from my research

Because of the stated goals of the Justice40 Initiative, our main analytical goal was to evaluate different funding rules on equity and economic objectives and identify which rules perform well. To make claims about policy performance, our decision analysis was built on the following modeling chain…

Pollack et al., (2025). Funding rules that promote equity in climate adaptation outcomes

One reviewer complimented us for our extensive accounting of uncertainty. However, our (narrowly scoped!) research and policy questions did not call for factor mapping or prioritization (though we call for this in our discussion).

Some personal reflections

• Avoid inconsistent uncertainty framing and analysis (e.g., calculating expected values when framing uncertainties as “deep”)
• I feel like there’s not good guidance out there on saying for decision analysis purposes, how to represent and analyze various types of uncertainty
• Don’t lose sight of why uncertainty matters for your decision analysis (i.e., don’t ignore uncertainty but don’t overcomplicate how you handle it)
• Remember that peer-reviewed studies are units of overall decision analysis projects (i.e., don’t overstuff these studies with too many analyses and points)

Upcoming Schedule

This week

• We will only meet today this week - prioritize your project proposal and schedule office hours as needed
• Let me know if you are testing out the lab and need any help

Next week

• Starting RDM module
• Journal club next Wednesday (stay on track with 9/29 readings)