EXTRA – Ensemble predictions are not real forecasts – EXTRA
Contributed by McFools
Forecasting has since the beginning of time been the process of making statements about events whose actual outcomes (typically) have not yet been observed. Consequently, adding uncertainty to your forecast permitted you to make many statements about the future.
However, theorists claim that you cannot simply add uncertainty as you wish: no, the forecast has to be reliable and skilful! It seems like an awful lot of work!
Previously, we used deterministic, high resolution forecasts and changed them according to our intuition and experience. Most of the time that worked pretty well, and since we showed a single forecast everybody believed that we knew exactly what we were doing.
In those days we could easily derive a simple traffic light system with green (all OK), yellow (take care), red (flooding imminent). With ensemble forecasts everything has changed (how do you explain that there is a 10% chance of getting flooded, it just does not make sense!!!) and we just look like fools, having no clue about the future.
So, following this reasoning, it is clear that ensemble predictions are not real forecasts, they are just an excuse for hydrologists to shift responsibility and avoid getting the blame and being fired.
This might sound a bit harsh, but it is the truth, and it is about time to wake up from this uncertainty nightmare. Other ludicrous claims and assumptions behind ensemble/probabilistic predictions confirm this:
- They never ever give you an exact and unique answer to your problem
- They give you forecast hydrographs, but you cannot recognize your peak flow and they tell you it’s normal because it is a ‘spaghetti plot’!
- They keep telling you that you need to wait some time to get a verification score and finally learn about the quality of the forecasts – I don’t want to wait, I want to know the skill of my forecast today!
- They say they have economic value, but you still have no extra money in your pocket at the end of the month
- They push hydrologists to work with meteorologists in the same room and tell you that it will be okay and nobody will get hurt
- They make your computer crash with low memory space by the time you start believing you can run ensemble predictions and get the first results ready for the next risk prone season (after the season is over of course)
- When you solve your memory problem, they come again with a 20-year reforecast to run!
- They promise you a PhD time of scientific glory and when you publish your results, reviewers tell you that your paper is rejected because confidence intervals around the Brier Score are missing
- They ask you to be post-processed but when you get better accuracy with a state-of-the-art technique, the reliability score is worsened (and vice versa)
- They tell you they are serious predictions made by serious people and recognized scientists, but when you go to their website, you find games, YouTube videos (just look at the xmas one, which proves my point) and a photo Gallery full of amazing people interested in planning another workshop!
Therefore, we believe it is time to demand that hydrologists and meteorologists are given again a proper education (and re-education for all the lost cases out there), which above all ends the nonsense of uncertainty in forecasting.
Join in and leave a comment in support!
Want to know more? Why not subscribe to our mailing list, follow us on twitter, facebook or LinkdIn. Before citing this article we strongly suggest to check the author’s name and date of publication.