Tracing The Origins of ESP

HEPEX Historical Hydrology Series, Edition 1

Contribution by Andy Wood, Tom Pagano, and Maury Roos, with special thanks to Mike Anderson

Every day in the western US and elsewhere in the world, reservoir release decisions are made based in part on seasonal reservoir inflow forecasts created using a operational technique called Ensemble Streamflow Prediction (ESP, which originally stood for Extended Streamflow Prediction).

ESP has become a key component of operational long lead prediction in a number of agencies (in the US, Russia, Czechoslovakia, Germany, Sweden, to name a few countries) in the last few decades. The earliest common references for ESP are a US Western Snow Conference presentation (Twedt et al., 1977) and Day et al. (1985) journal article.

But where did ESP really begin?

We have to go back a little over 40 years, when the State of California was about to grapple with a series of long lead ‘water supply forecast’ (WSF) busts in one of the worst droughts to hit parts of the western US in the 20^th Century – the two year drought of 1976-77.

In the fall of 1977, California water resources were dire condition, leading to intense interest in predictions for the coming winter and 1978 water supply (i.e., snowmelt runoff) outcomes. California’s winter rain and snowfall typically provide the bulk of the water used by the state throughout the year.

The 1970s were an exciting period of experimentation with seasonal, regional climate forecasting. California was working with private consultants on a program called Project Hydrospect, the goal of which was to use long-range weather forecasts to help with water supply management.

Researchers at Scripps (including Jerry Namais, who in the 1960s had developed monthly and seasonal forecasts at the U.S. Weather Bureau ) were making innovative forecasts, leveraging new computer capabilities in linear algebra (eg, inverting matrices!) that opened doors to objective analysis techniques such as relating EOFs of Pacific Ocean SSTs to California climate. ENSO indices had not yet been defined … but that’s another interesting story, perhaps one for another blog.

The very first ESP river forecasts were simultaneously being developed in the mid-1970s by a joint federal-state operations center that housed personnel from the State of California Department of Water Resources (CADWR) and the California-Nevada River Forecast Center (CNRFC) of the National Weather Service (NWS).

Flood forecasting was a federal responsibility, and in theory, longer-range forecasts were a joint activity. The two parts of longer range forecasts were:

1. a continuous soil moisture accounting model (the Sacramento model, developed by NWS; Burnash et al, 1973) and
2. the concept of combining estimated initial hydrologic conditions (IHCs) with future meteorological forecasts based on historical sequences or seasonal climate predictions to drive hydrologic simulations extending out to seasonal lead times.

A depiction of a seasonal ensemble forecasts from one of the first publications in which it was called ESP is below.

The ideation of ESP and initialization date of the first ESP forecast in the US is unclear, but the likely first operational ESP forecaster was a woman named Helen Joyce Peters.

Ms. Peters headed the State side of the joint flood forecasting program at the time, and was running the long-lead forecasts as well. Anecdotal reports suggest that her forecasts predated those from the NWS side of the center (made by Bob Burnash) by several weeks to months.

Ms. Peters was a groundbreaker in more than one way: she was the California DWR’s first female engineer as well as an internationally-renowned groundwater specialist. Others involved in the effort included consultants Jack Hannaford, Joe Burns, Bob Zettlemoyer from the State, and John Schaake, who joined the NWS Hydrology Laboratory (HL) in 1974. In the same year, Dave Smith of NWS coined the term Extended Streamflow Prediction, and the NWS HL soon after began developing an NWS version of ESP for use in other RFCs.

The two first major applications of ESP were both drought related – predictions made during the California drought and also for a smaller but intense drought in Virginia, affecting the Occoquan Reservoir, which formed part of the water supply for the Washington DC area. Based on the ESPs and other analysis techniques, probabilistic predictions for reservoir target level failures could be compiled (figure below). Today, we take such approaches for granted, but at the time they were quite innovative applications of stochastic methods in hydrology.

It’s interesting that the first ESP forecasts were made just an hour up the road from the source of so much modern-day innovation at companies such as Google and Apple.  Is there something special in the water there?

In any case, ESPs — born in Sacramento, and focusing on the water supply drainages of the California Sierras — were due in no small measure to Joyce Peters, a little-known original forecaster and program lead. Ms. Peters passed away in 2002 after a remarkable career with DWR lasting 37 years.

NB: This reconstruction of the ESP origin is based on notes from interviews Tom Pagano (AU BOM) conducted with John Schaake, former Chief Scientist of the NWS Office of Hydrology, and on correspondence between Andy Wood (NCAR), Eric Rosenberg (Hazen and Sawyer), and Tom Pagano with Maury Roos, the former Director of the California Department of Water Resources. California’s State Climatologist, Mike Anderson, was also extremely helpful in providing digitized copies of Hydrospect Project reports from the mid to late 1970s, which are a fascinating read. Like any history based on long-ago memories and incomplete documentation, it is likely to be just one possible version of the actual events, and may contain inaccuracies. It is also possible that ESP evolved independently in other countries. Please send comments if you know more details of this history, or a related forecasting history that would be of interest to HEPEX readers. 

References

• Burnash, R.J.C., Ferral, R.L., and McGuire, R.A. (1973). “A Generalized Streamflow Simulation System – Conceptual Modeling for Digital Computers,” U.S. Department of Commerce, National Weather Service and State of California, Department of Water Resources.
• Day, G. (1985). “Extended Streamflow Forecasting Using NWSRFS.” J. Wat. Res. Plan. Mgmt., 10.1061/(ASCE)0733-9496(1985)111:2(157), 157-170.
• Hirsch, R, J Schaake and D Sheer (1977). Assessment of Current Occoquan Water Supply Situation. Pub: # ICPRB-M-4, Interstate Commission on the Potomac River Basin.
• Twedt, T. M., J. C. Schaake, Jr., and E. L. Peck (1977). National Weather Service extended streamflow prediction. Proc., Western Snow Conference, 52 – 57.