Risk aversion and decision making using ensemble forecasts
by Marie-Amélie Boucher and Vincent Boucher
Assessing the value of forecasts is a very popular topic among the HEPEX community. The assessment of forecast value is highly dependent on the purpose served by the forecasts. For the specific problem of decision-making related to flood mitigation, Murphy (1976, 1977) proposed the use of the cost-loss ratio framework. The vast majority of papers related to the assessment of forecast value for flood mitigation adopt this framework, so one could think that everything is pretty much solved… except that the cost-lost ratio has very important flaws!
One such flaws is the fact that the cost-loss ratio assumes that the decision maker is risk neutral. Risk neutral individuals only care about the expected outcome, and disregard the spread of the distribution of outcomes. Risk neutral individuals are very rarely encountered in real life. Indeed, most of us are risk averse. That is, for the same expected outcome, we prefer less risky distributions. This is (among other things) why we buy insurance. Informally, most people dislike risk and would be willing to spend resources (e.g. money) in order to reduce the amount of risk faced.
Utility theory
Economists (and statisticians and mathematicians) have been studying those issues for a long time, and came up with many models and concepts which could be used in hydrology, as an alternative to the cost-loss ratio. The central framework is based on “utility theory”. To put it’s development a bit in a context, here is a (very) pseudo-historical adaptation of a conversation between Nicolau I Bernoulli and his cousin Daniel [1]:
It didn’t happen exactly like that, but the general idea is preserved. The game that Nicolau is referring to is now known as the St-Petersburg paradox and was first described in a letter. The first person to present it more formally was Daniel Bernoulli in 1738. He was also the first to suggest the idea of risk aversion and exposed (without any mathematics) that different persons faced with the same decision problem and the same information could take different decisions, because of different preferences.
It is only much later, in 1944, that the concept of risk-based individual preferences was formalized as a mathematical theory, by John von Neumann and Oskar Morgenstern [2]:
Utility theory is not perfect (and indeed, many generalizations and extensions exist), but it has at least two advantages over the cost-loss ratio:
- It allows for decisions to consider a finite number or a continuum of decisions (e.g. “protect” vs “don’t protect” for floods, or alternatively, “amount spent in protection”).
- It allows to account explicitly for the decision maker’s level of risk aversion in the assessment of forecasts value.
In hydrology
Krzysztofowicz suggested using utility theory in hydrology as early as in 1986. It is not a miraculous solution to the problem of assessing the value of forecasts for flood mitigation, but it would be an improvement over the cost-loss ratio. It can explain real-life behaviors that the cost-loss ratio cannot. This was the case in our recent application for the assessment of forecasts value on the Montmorency River in Canada (Matte et al. 2017). Other examples of accounting for risk-aversion for decision-making in hydrology and meteorology include Shorr (1966), and Cerdá and Quiroga Gómez (2008).
References:
- Bernoulli D. (1738) Specimen Theoriae Novae de Mensura Sortis, Commentarii academiae scientiarum imperialis Petropolitanae, 5, 175-192.
- Cerdá Tena E. and Quiroga Gómez S. (2008) Cost-Loss Decision Models with Risk Aversion, Working paper no. 01, Instituto Complutense de Estudios Internaciolales, 28 pages.
- Krzysztofowicz R. (1986) Expected utility, benefit, and loss criteria for seasonal water supply planning, Water Resources Research, 22(3), 303-312.
- Matte S., Boucher M-A, Boucher V. and Fortier-Filion T-C (2017) Moving beyond the cost-loss ratio, economic assessment of streamflow forecasts for a risk-averse decision maker, Hydrology and Earth System Sciences (Accepted)
- Murphy A.H. (1977) Value of climatological, categorical and probabilistic forecasts in cost-loss ration situation, Monthly Weather Review, 105(7), 803-816
- Murphy A.H. (1976) Decision-making models in cost-loss ratio situation and measures of values of probability forecasts, Monthly Weather Review, 104(8), 1058-1065
- Shorr B. (1966) The cost/loss utility ratio, Journal of Applied Meteorology, 5(6), 501-803.
- von Neumann J. and Morgenstern O. (1944) Theory of games and economic behavior, vol. 60, Princeton University Press Princeton, 625 pages.
Figure captions:
[1] The original pictures were taken from these websites: http://www.famous-mathematicians.com/daniel-bernoulli/ (right) and http://www2.stetson.edu/~efriedma/periodictable/html/Bi.html (left). The schematic drawing of D. Bernoulli’s blood pressure experiment apparatus was taken from https://plus.maths.org/content/daniel-bernoulli-and-making-fluid-equation
[2] The original pictures were taken from these websites: http://www.tinbergen.nl/oskar-morgenstern-and-john-von-neumann-shelby-white-and-leon-levy-archives-center/ (left) and http://www.karbosguide.com/books/pcarchitecture/chapter02.htm (right)
June 6, 2017 at 11:54
I fully agree with the concept of using Bayesian approaches and utility theory in water resources management. The basic problem is that meteorological ensembles do not correctly describe the predictive density to be used in order to compute expected utilities, which can severely distort the expected values and the consequent decisions. Ensembles are biased and under-dispersive (Hamill and Colucci 1997; Eckel and Walters 1998; Schwanenberg et al. 2015; Todini, 2017) and tests on the obtained predictive densities, as advocated by Diebold et al., (1998) or Laio and Tamea (2007), clearly show their inadequacy (Todini, 2017). In order to derive more realistic and correct predictive densities, alternative approaches should be advocated either using BMA (Raftery, 1993) or by modifying the existing post processors such as HUP ( Krzysztofowicz, 1999), or MCP (Todini, 2008), in order to take into account the time variability of the ensemble spread.
June 6, 2017 at 16:55
Thank you for this very relevant comment! This is precisely what we show in our paper (Matte et al. 2017). Yes, meteorological (and hydrological) ensemble forecasts are flawed in many ways and this is clearly a problem that needs to be addressed. But regardless, they are still used operationally (sometimes without any post-processing). In Matte et al. (2017), we illustrate how « imperfections » in ensemble forecasts affect the behavior of a risk-averse decision maker, using a real-life example and for various levels of risk aversion. Those imperfections lead to « bad » spending decisions, but we can actually relate that to characteristics of the forecasts! In our conclusions, we indeed mention that statistical post-processing could improve the forecasts, and hence their economic value. Different post-processing methods could be compared in that same framework, involving utility theory. Obviously, there is still a lot to do!
April 15, 2021 at 13:31
You have written a great blog about RISK AVERSION AND DECISION MAKING USING ENSEMBLE FORECASTS . Thanks for sharing such type of information. I want to know more about……….. Would you please give me any other resources so i could go in depth about it.