What makes a forecast with uncertainty valuable to practitioners?

by Sarah Michaels (University of Nebraska)

“the extent to which forecasts shape decision making under uncertainty is the true measure of the worth of a forecast”

For forecasts to be useful requires they be scientifically justified, the information conveyed valuable and action can be taken based on them that lead to desired outcomes. While users may or may not be able to assess the calibre of the science and the modeling, they can more readily determine if the outputs of the model are applicable to their decision making and whether or not they can exploit a particular forecast by identifying actions they can afford to take based on them. Lead-time, scale and ease of uptake become essential forecast properties.

What users value are forecasts that meet or can be readily modified to fit their specific requirements since decisions about different operational flood management functions have different information needs. For example, to operate hydropower dams and to manage lakes total discharge volumes from medium range forecasts with lead times of 3-5 day are valuable. For flood management planning and response, peak flow timing and peak discharge volume are vital. Flood defense measures are initiated and flood alerts delivered based on short to very short range forecasts including specific information on peak time, peak discharge, and possibly areas inundated and into which observed data can be assimilated.

Emergency managers are concerned that when the probability of an adverse event is low conveying that low probability will reduce how many people will heed a warning about that event. While it may be tempting to withhold uncertainty as a means of reducing such a possibility and/or simplifying the information relayed, there are good reasons not to do so.

When estimates of forecast uncertainty are not provided, individuals generate their own estimates and often do so in such a way that does not take full advantage of the forecast provided. Individuals for whom forecast uncertainty information is not given adopt more risk averse stances than when uncertainty information is provided. People are more likely to make sound decisions in sync with other people’s decision making when they receive uncertain information than when they don’t.
When users receive a forecast including upper and lower bounds of the predictive interval they may conclude forecast providers acknowledge the forecast’s uncertainty and still consider taking protective action is justified. This is particularly important for extreme events when it is vital for people to trust the forecast and to take the recommended actions. Thus admitting the uncertainties and limitations associated with the science underpinning the forecasts may become a means of building trust between scientists and decision makers.

Pragmatically, the extent to which forecasts shape decision making under uncertainty is the true measure of the worth of a forecast. What counts is the benefits acquired and the losses avoided that would not have accrued if the forecast had not been used.

This blog draws from Michaels, S. 2015. Probabilistic forecasting and the reshaping of flood risk management. Journal of Natural Resources Policy Research 7:1:41-51. DOI: 10.1080/19390459.2014.970800