HEPEX Verification Testbed and the Ensemble Verification System (EVS)

Contributed by Maria-Helena Ramos, James Brown and Julie Demargne

Meteorological and hydrologic forecasts, as well as operational forecast products, must be verified at multiple temporal and spatial scales. Rigorous and objective verification help us answer:

• Are the forecasts unbiased/accurate?
• Are the forecasts more skilful than a chosen baseline?
• What are the strengths and weaknesses in the forecasts?
• What are the sources of error, bias and uncertainty in the forecasts?
• How are new science and technology improving the forecasts and the verifying observations?
• What is the most effective (e.g. cost effective) way to improve the forecasts?
• What are user’s needs for forecast quality and verification information?

The verification testbed conducted within HEPEX was launched in 2005. It acknowledged the need for specific verification metrics and products adapted to hydrology (Demargne et al., 2010). See an introductory presentation here.

The Ensemble Verification System (EVS)

EVS was built, in part, to meet the needs of the verification test bed (Brown et al., 2010). It was originally developed by the Office of Hydrologic Development of the U.S. National Weather Service, and was designed to be flexible, modular and open to accommodate enhancements and additions by its users. The first version was released in 2008 and it has been continuously updated.

EVS is a Java application for evaluating the quality of ensemble forecasts issued at discrete locations (points or areas). The binary executable, user’s manual, example data, source code and developer’s documentation are all freely available to download. The source code is released under the GNU Lesser General Public License (GPL) Version 3.0.

If you have used the EVS, please leave a comment here about your experience or any suggestions for improvements. For specific questions about the system, you may also contact the developers.

Way forward on hydrologic ensemble prediction verification

We would be grateful for your thoughts on future priorities for ensemble verification of hydrologic and hydrometeorological forecasts. For instance:

• What should we look at now?
• Are new metrics needed?
• How can forecast verification better interact with other HEPEX topics (pre- or post-processing, data assimilation, estimation of forecast value)?

Your suggestions are welcome!