On decisions under uncertainty

by Jan Danhelka, a HEPEX 2015 Guest Columnist

There was an interesting side meeting organized by the WMO on Multi-hazard Early Warning Systems during the UN World Disaster Risk Reduction Conference in Sendai, Japan last month.

All participants agreed on the need to prepare impact-based forecasts and warning instead of phenomenon-based ones. I think everyone agrees. At the same time, it means a great challenge to meteorology and hydrology. Here are some of my thoughts about this topic.

Impact-based forecasting

In hydrology, we have a long term experience in impact-based forecasting (I guess more than in meteorology). An example is the flow forecast for navigation for the Elbe River. Every day, a special forecast is prepared that defines the minimum water stage guaranteed for the critical section of the river if stage is below 270 cm (a limit for full loading of ships).

At the same time, the flood stages are mostly defined based on the potential damage in the given area, and not based on flow probabilities. Therefore, the highest flood alert threshold for Prague equals to 1 500 m³.s^-1 (< 5-year flood), while, e.g., in Ostrava, it corresponds to the 20-year flood. These thresholds are well known and periodically tested. It is probably more difficult to do the same in meteorology, although I am aware that, for example, winds or tornados classifications were originally based on the evaluation of their impacts (by the way, the same was a cause of earthquakes intensity definition). But, for some phenomena, it looks like that the human adaptation is more flexible, such as for temperature (see the paper of Guo et al., 2014 here). Detailed impact-based forecast is however not possible (at least currently) in global products, as potential impact information is not available or not applicable. This is a reason why local knowledge in forecasting is necessary.

Deep uncertainty in decision

Recently, I have read a great World Bank document, Agreeing on Robust Decisions. It deals with the deep uncertainty in decisions on climate change adaptation measures. In general, there are two ways on how to agree on solutions and measures for deeply uncertain future:

• The first one, let’s say the traditional one, is “agree-on-assumptions”. It demands for an agreement on forecasts, predictions or scenarios, and then it analyses the decision options.
• The second option is “agree-on-decisions”, which might be characterized as seeking the robust decision that performs well across a wide range of possible future scenarios.

In my opinion, the decision process in real-time flood management and protection is more often based on the “agree-on-assumptions” principle, especially on local level. But, as forecasters, we also mostly try to decrease the uncertainty of our forecast by agreeing on which scenario (ensemble member) is not realistic and, as such, we apply the “agree-on-assumption” way of thinking.

“Hope for the best and prepare for the worst”

At the beginning of the floods in 2013, I was called by the Minister of Environment, who is a responsible Head of the National Flood Committee. It was a very stressful morning; a mesoscale convergence zone developed during the night and caused extreme precipitation and runoff (unfortunately, it had not been forecasted before it occurred). All the streams were rising and flood protection measures started to be taken in Prague.

The Minister simply asked me what the forecast for Prague was. After I have explained all the known information, forecasts and uncertainties, I gave him my best guess of the peak flow. But his response was: “No, no, no, give me the worst case scenario; don’t tell me numbers you cannot guarantee as not being exceeded”. Well, obviously, he was a good crisis manager and followed the motto “Hope for the best and prepare for the worst”.

The question is what is the worst case scenario? In general, for Prague, the worst case scenario is the 2002 Flood water level plus 35 cm (as the protection is built for 2002+30 cm). But that was not the answer he was demanding for (and there was no time for kidding, especially with the minister).

So, what is the worst case scenario under a given circumstance?

Now, I think that when we find the worst case scenario, than, in the case of flood forecasting, we more or less fuse the “agree-on-assumptions” with the “agree-on-decisions” systems. The worst case scenario is definitely an assumption, but, at the same time, any decision that doesn’t perform under the worst case scenario is not robust enough to be accepted (and not politically justifiable).

The only problem that remains is the way how to derive a worst case scenario. I don’t think the use of the ensemble upper envelope is the right solution, nor the forecast plus X% is. I am convinced that this is something we need experienced and skillful forecasters to do.

While writing this post I realized that a lack of experience is bad, but experience might as well result in excessive certitude of how to decide. That leads me to the conclusion that it is better to learn what you have done wrong during the last time than to learn what you have done right. The second may suggest repeating the solution in the next time, without looking for a better solution under the new given circumstances.