The aim of the CEMS-Flood post-processing methodology is to adjust the CEMS-Flood medium-range ensemble forecasts at specific locations, so they become predictors of future observed river discharge values. Because of the requirement of near-real time river discharge observed time series to generate the product, It is applied only to the EFAS system.

The CEMS-Flood post-processing methodology is based on a combination of two post-processing techniques: the Model Conditional Processor (MCP; Todini, 2008) and the Ensemble Model Output Statistics (EMOS; Gneiting et al., 2005) method. The post-processed forecast is represented by a probability distribution that is dependent on recent observations, simulation forced by observations (also known as the EFAS reanalysis), and forecasts. The output of this process is the 'Real-time Hydrograph' which is available in the pop-out windows of the Reporting Point layer for static reporting points where near real-time and past river discharge observations are available. Since EFAS version 4.5, the post-processing has been performed at 6-hourly timesteps where possible.

In CEMS-Floo, the post-processing is composed of two parts; the calibration (offline), and the forecast update (online):

Calibration (offline)

The offline calibration of the post-processing is performed twice a year to include the most recent observations.

Data: The off-line calibration requires at least 2 years of river discharge observations and the simulation forced by observations for the same time period. Where possible 6-hourly observations and simulations are used (as this allows the forecasts to be post-processed at a 6-hourly timestep in the forecast update part); daily observations are used otherwise and the simulation is aggregated to a daily timestep. For each station, the simulation comes from the most recent LISFLOOD historical run (available https://cds.climate.copernicus.eu/cdsapp#!/dataset/efas-historical?tab=overview).

The off-line procedure has two main objectives:

  • Estimation of separate river discharge distributions for the observed and simulated river discharge values. This estimation is performed by fitting a Generalised Pareto distribution to the extreme river discharge values and applying a kernel density estimation procedure for the remainder of the distribution (see Figure 1). 

Figure 1: An example of the estimated river discharge distribution for a station from the off-line calibration. Orange shows the part estimated by the Generalised Pareto distribution. Purple shows the main part of the distribution. Small black lines show the individual river discharge values. Modified from Matthews et al. (2022).


  • Estimation of a joint probability distribution of observations and simulations across multiple timesteps. The joint probability distribution describes the relationship between observed and simulated values at different times over a 55-day period. Figure 2 shows an example of a joint distribution between 2 variables (a simulated variable (model) and an observed variable (reality)). The joint-distribution defined in the off-line calibration is between 440 variables for the 6-hourly stations and 110 variables for the daily stations.


b

Figure 2: Representation of the joint probability distribution of observations (reality) and simulations (model), figure from Biondi et al. (2018).

The distributions defined in the offline calibration are used in the forecast update part of the post-processing method. The length of the observation record and the quality of the observations can impact the accuracy of the distributions.

Forecast Update (online)

The online part of the post-processing method is performed for each station where the offline calibration was successful and near real-time river discharge data are available. 

Data: The forecast update step requires the observations, simulation forced with observations (water balance), and CEMS-Flood ensemble forecasts for the past 40 days (although some leniency is given for missing values). It also requires the current CEMS-Flood ensemble forecast (i.e., the forecast that is being post-processed). The distributions defined in the offline calibration are also required. Where possible 6-hourly observations are used and daily observations are used otherwise. However, the offline calibration and the real-time post-processing must use the same timestep.

The forecast update step is further split into 3 steps:

  1. The joint probability distribution defined in the off-line calibration is used to condition the river discharge distributions defined in the off-line calibration on recent river discharge observations (see Fig. 3a). Using the recent river discharge values in this way restricts the forecast probability distribution to the values that are likely given the recent state of the river. This is the MCP portion of the method and it is used to correct errors and uncertainty due to the hydrological model and initial conditions.
  2. The current CEMS-Flood ensemble forecast is spread corrected (see Fig. 3b). This is done by calculating the average spread correction parameter need to match the spread of the CEMS-Flood ensemble forecasts with the root mean square error of the ensemble mean for the past 40 days. This is the EMOS portion of the method and it is used to correct errors and uncertainty due to meteorological forcings.
  3. Steps 1 and 2 results in two probabilistic distributions which are combined using a Kalman filter (see Fig.3) . The Kalman filter weights the two distributions from steps 1 and 2 depending on their uncertainty (or spread). This creates a probability distribution that is consistent with recent observations and influenced by the predicted meteorological forcings. 
  4. The Real-Time Hydrograph product is created (see below). 

Figure 3: The forecast update part of the post-processing method uses a) the MCP method and b) the EMOS method. The output from these two methods is combined using the Kalman Filter to produce the Real-time Hydrograph'.

Real-time Hydrograph

CEMS-Flood post-processed forecasts are available for stations with at least 2 years of river discharge data and that provide near real-time river discharge observations to the CEMS Hydrological Data Collection Centre (HYDRO). The post-processed forecast is shown by the 'Real-time Hydrograph' product shown in the pop-up window of the 'Reporting Points' layer. Stations for which the Real-time hydrograph is available are represented by light blue points in the Reporting Points layer.

The main panel shows the probability distribution (blue shading) for each timestep of the forecast and the recent observations (black dots). Darker blues show values closer to the forecast median. The two panels on the right show the probability of exceeding the mean annual maximum flow (MHQ; top) and the mean flow (MQ; bottom) thresholds respectively. These thresholds are calculated from past river discharge observations.

Two examples are shown below, for stations Gaulfoss, Gaula in Norway (ID 1099) at 6-hourly timesteps, and Clonmel, Suir in Ireland (ID 1409) at daily timesteps.  

Figure 4 - Real-time hydrographs for stations Gaulfoss (left), and Clonmel (right).


Note: The post-processed forecasts have the tendency to slightly underestimate peaks, particularly in catchments with quick hydrological response times. We are investigating improvements to the method.

Known Issues

The CEMS-Flood post-processing method is highly dependent on both the past and near real-time observed and simulated river discharge values. Issues can arise if either the observed or simulated river discharge values are much higher than those previously recorded or if an insufficient number of near real-time observations are available at the time the forecast is created. 

If the ensemble forecast predicts river discharge values higher than those recorded in the CEMS-Flood historical river discharge simulation, the Real-time hydrograph will show as:

If the post-processed forecast predicts river discharge values higher than those recorded in the observed record made available to CEMS-Flood, the Real-time hydrograph will show as:

If an insufficient number of near real-time river discharge observations are made available to CEMS-Flood, the Real-time hydrograph will show as:



References

Biondi, Daniela & Todini, Ezio. (2018). Comparing Hydrological Postprocessors Including Ensemble Predictions Into Full Predictive Probability Distribution of Streamflow. Water Resources Research. 10.1029/2017WR022432. https://doi.org/10.1029/2017WR022432

Gneiting, T., Raftery, A. E., Westveld, A. H., & Goldman, T. (2005). Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Monthly Weather Review, 133(5), 1098-1118.

Matthews, G., Barnard, C., Cloke, H., Dance, S. L., Jurlina, T., Mazzetti, C., & Prudhomme, C. (2022). Evaluating the impact of post-processing medium-range ensemble streamflow forecasts from the European Flood Awareness System. Hydrology and Earth System Sciences, 26(11), 2939-2968.

Todini, E. (2008). A model conditional processor to assess predictive uncertainty in flood forecasting. International Journal of River Basin Management, 6(2), 123-137.