Ensembles of hydrological modellers can foster advances – Example with a participative workshop

Contributed by Guillaume Thirel, member of the IRSTEA Columnist Team

The 2013 IAHS General Assembly in Göteborg saw two very interesting events happening:

• The biennal IAHS ultimate frisbee tournament, organised by Irstea since 2007 [1], that featured some of the Prediction in Ungauged Basins decade participants and some of the Hepex Blog columnists,
• A participative workshop on modelling changing catchments, also organised by Irstea, that featured modellers willing to apply their beloved models under changing and challenging conditions.

These two topics, ultimate frisbee and participative workshops, represent some of the major focuses of our team: ultimate frisbee is played on a very regular basis at noon at Irstea or during conferences; we organised another participative workshop during the 2011 IAHS GA in Melbourne, in which a large panel of hydrologists was asked to analyse a set of hydrographs (see results by Crochemore et al., 2015 here); we regularly contribute to develop the HEPEX games you can find here; we recently participated in the multi-model exercise described by de Boer et al. (2016), among many other examples.

While posting about the frisbee tournament would be for sure much fun, I am going to post here about the participative workshop.

A workshop for modellers to confront their models to changing catchments

I was asked more than a year ago to post in this blog about this 2013 IAHS workshop, where we wanted modellers to confront their models to changing catchments. The rationale behind this is that we are often asked by water authorities to produce hydrological projections under IPCC scenarios, but in spite of knowing that models could have difficulties to deal with the very different future conditions (lack of robustness), we often tell ourselves and to others: “Come on, that will be fine, my model is good enough and physics is not going to change…”, without verifying much, or at all, the actual robustness of the model.

I must recognize that this topic may not be of direct relevance to what HEPEX is about, but several common points can be raised:

• First, it is all about hydrological models (but simulation models rather than prediction models). They are of course at the centre of our research, regardless if we are forecasters or researchers in any other hydrological domain. For the workshop, we asked modellers to perform calibrations and simulations with their own models a couple of weeks before the workshop.
• Second, we believe in collaborations to enhance the efforts of individuals (which is a strong common point with HEPEX). As a consequence, we organised this event as a real workshop, with lots of time dedicated to discussions and even working groups, instead of the usual presentation-only conferences format.
• Third, we know the importance of modelling protocols. As recently stated in an HEPEX blog post, rigorous protocols are necessary to evaluate theories or algorithms. In the 2013 workshop, we defined a specific calibration protocol that each modeller had to apply to their model.
• Fourth, we want to generalize our findings through the use of multiple catchments datasets. As large sample hydrology is now largely advocated, forecasting models and model developments for global changes purposes applied to multiple basins are now more and more common. In the workshop, we gathered a dataset of 14 catchments whose changes were precisely known and described.
• Finally, many of HEPEX collaborators are also involved in model development and climate change applications and many improvements of hydrological models can benefit to hydrological forecasting.

To go into more details, we prepared for this workshop a dataset gathering lumped precipitation, temperature and discharge for 14 catchments. We also described the changes these catchments had faced (climate change, building of a dam, progressive or sudden land cover modifications). We asked the modellers to calibrate their models on six different periods: the complete period of available data, plus five equal-length subperiods, some of them being before the changes, some of them being after the change (Figure 2).

We then proposed the modellers to send to me their simulations so that I could produce a large range of metrics and analysis plots (eg. Figure 3). The details about the protocol, metrics, plots and dataset can be found in Thirel et al. (2015).

By doing that, we provided the modellers with a good framework for:

• analysing their models with common tools/methods,
• assessing how the parameters and performances of models evolve with changes,
• determining to which extent a model prepared before a change can provide realistic simulations after it,
• proposing and evaluating solutions to improve the robustness of models.

With more than 40 abstracts submitted, the outcomes from this workshop were numerous and it would be unfair to emphasize one specific work. The curious reader is invited to have a look at the Special Issue we organised in the Hydrological Sciences Journal and at the Editorial that summarises it (Thirel et al., 2015).

In my opinion, and I actually agree a lot with the outcomes from the Bertinoro workshop, common modelling protocols and datasets are absolutely necessary to better validate our advances in hydrological forecasting or hydrological modelling robustness. The combinations of these two elements allow for the reproducibility of studies, which is essential to serve as benchmarks to further advances.

[1] Check here for a history of all the IAHS games. The next game will take place in Port Elizabeth in July 2017 during next IAHS GA.