...

...

Obtaining the exercises

The exercises described below are available as a set of Metview macros with the accompanying data. This is available as a downloadable tarfile for use with Metview. It is also available as part of the OpenIFS/Metview virtual machine, which can be run on different operating systems.

For more details of the OpenIFS virtual machine and how to get the workshop files, please contact: openifs-support@ecmwf.int.

ECMWF operational forecasts

At the time of this case study in 2012, ECMWF operational forecasts consisted of:

...

Please follow this link to see more details on changes to the ECMWF IFS forecast system (http://www.ecmwf.int/en/forecasts/documentation-and-support/changes-ecmwf-model)

Virtual machine

If using the OpenIFS/Metview virtual machine with these exercises the recommended memory is at least 6Gb, the minimum is 4Gb. If using 4Gb, do not use more than 2 parameters per plot.

These exercises use a relatively large domain with high resolution data. Some of the plotting options can therefore require significant amounts of memory. If the virtual machine freezes when running metview, please restart the VM and increase the memory assigned to the VM

Starting up metview

To begin:

metview &

...

Saving images and printing

 To save images during these exercises for discussion later, you can either use:

...

ksnapshot

Exercise 1. The ECMWF analysis

Hurricane Nadine and the cut-off low

Task 1: Mean-sea-level pressure and track

Right-click on the 'an_1x1.mv' icon and select the 'Visualise' menu item (see figure right)

...

Task 2: MSLP and 500hPa geopotential height

This task creates Figure 2. from Pantillon et al.

...

Task 3: Changing geographical area

Right-click on 'an_1x1.mv' icon and select  'Edit'.

...

Animate the storm on this smaller geographical map.

Task 4: Wind fields, sea-surface temperature (SST)

The 'an_2x2.mv' icon allows for plotting up to 4 separate figures on a single frame. This task uses this icon to plot multiple fields.

...

Task 5: Satellite images

Open the folder 'satellite' by doubling clicking (scroll the window if it is not visible).

...

Use the an_1x1.mv and/or the an_2x2.mv macros to compare the ECMWF analyses with the satellite images.

Task 6: Cross-sections

The last task in this exercise is to look at cross-sections through Hurricane Nadine and the cut-off low.

...

Changing forecast time

Cross-section data is only available every 24hrs.

...

steps=[2012-09-22 00:00]

Changing fields

A reduced number of fields is available for cross-sections: temperature (t), potential temperature (pt), relative humidity (r), potential vorticity (pv), vertical velocity (w), wind-speed (speed; sqrt(u*u+v*v)) and wind vectors (wind3).

Changing cross-section location

#Cross section line [ South, West, North, East ]
line = [30,-29,45,-15]

...

Remember that if the forecast time is changed, the storm centres will move and the cross-section line will need to be repositioned to follow specific features. This is not computed automatically, but must be changed by altering the coordinates above.

Exercise 2: The operational HRES forecast

Recap

The ECMWF operational deterministic forecast is called HRES. At the time of this case study, the model ran with a spectral resolution of T1279, equivalent to 16km grid spacing.

...

Before looking at the ensemble forecasts, first understand the performance of the operational HRES forecast of the time.

Available forecast

Data is provided for a single 5 day forecast starting from 20th Sept 2012, as used in the paper by Pantillon et al.

HRES data is provided at the same resolution as the operational model, in order to give the best representation of the Hurricane and cut-off low iterations. This may mean that some plotting will be slow.

Available parameters

A new parameter is total precipitation : tp.

The parameters available in the analyses are available in the forecast data.

Available plot types

Forecast performance

Task 1: Forecast error

In this task, we'll look at the difference between the forecast and the analysis by using "root-mean-square error" (RMSE) curves as a way of summarising the performance of the forecast.

...

Task 2: Compare forecast to analysis

Use the hres_to_an_diff.mv icon and plot the difference map between the HRES forecast and the analysis first for z500 and then mslp (change plot1 from z500 to mslp).

...

If time: look at other fields to study the behaviour of  the forecast.

Task 3: Precipitation over France

This task produces a plot similar to Figure 2 in Pantillon et al.

...

Suggested plots for discussion

The following is a list of parameters and plots that might be useful to produce for later group discussion. Choose a few plots and use both the HRES forecast and the analyses.

...

Exercise 3 : The operational ensemble forecasts

Recap

• ECMWF operational ensemble forecasts treat uncertainty in both the initial data and the model.
• Initial analysis uncertainty: sampled by use of Singular Vectors (SV) and Ensemble Data Assimilation (EDA) methods. Singular Vectors are a way of representing the fastest growing modes in the initial state.
• Model uncertainty: sampled by use of stochastic parametrizations.
• Ensemble mean : the average of all the ensemble members. Where the spread is high, small scale features can be smoothed out in the ensemble mean.
• Ensemble spread : the standard deviation of the ensemble members, represents how different the members are from the ensemble mean.

The ensemble forecasts

In this case study, there are two operational ensemble datasets and additional datasets created with the OpenIFS model, running at lower resolution, where the initial and model uncertainty are switched off in turn. The OpenIFS ensembles are discussed in more detail in later exercises.

...

• Control forecast (unperturbed)
• Perturbed ensemble members. Each member will use slightly different initial data conditions and include model uncertainty pertubations.

2012 Operational ensemble

ens_oper: This dataset is the operational ensemble from 2012 and was used in the Pantillon et al. publication. A key feature of this ensemble is use of a climatological SST field (you would have seen this in the earlier tasks).

2016 Operational ensemble

ens_2016: This dataset is a reforecast of the 2012 event using the ECMWF operational ensemble of March 2016. Two key differences between the 2016 and 2012 operational ensembles are: higher horizontal resolution, and coupling of NEMO ocean model to provide SST from the start of the forecast.

The analysis was not rerun for 20-Sept-2012. This means the reforecast using the 2016 ensemble will be using the original 2012 analyses. Also only 10 ensemble data assimilation (EDA) members were used in 2012, whereas 25 are in use for 2016 operational ensembles, so each EDA member will be used multiple times for this reforecast. This will impact on the spread and clustering seen in the tasks in this exercise.

Ensemble exercise tasks

Visualising ensemble forecasts can be done in various ways. During this exercise we will use a number of visualisation techniques in order to understand the errors and uncertainties in the forecast,

Key parameters: MSLP, z500, and total precipitation (tp) over France.  We suggest concentrating on viewing these fields. If time, visualize other parameters (e.g. PV320K).

Available plot types

Group working

If working in groups, each group could follow the tasks below with a different ensemble forecast. e.g. one group uses the 'ens_oper', another group uses 'ens_2016' and so on.

...

#The experiment. Possible values are:
# ens_oper = operational ENS
# ens_2016 = 2016 operational ENS

expId="ens_oper"

Ensemble forecast performance

In these tasks, the performance of the ensemble forecast is studied.

Task 1: RMSE "plumes"

This is similar to task 1 in exercise 2, except the RMSE curves for all the ensemble members from a particular forecast will be plotted.

...

• Explore the plumes from other variables.
• Do you see the same amount of spread in RMSE from other pressure levels in the atmosphere?

Task 2: Ensemble spread

In the previous task, uncertainty in the forecast by starting from different initial conditions and the stochastic parameterizations can result in significant differences in the RMSE (for this particular case and geographical region).

...

Task 3: Spaghetti plots - another way to visualise spread

A "spaghetti" plot is where a single contour of a parameter is plotted for all ensemble members. It is another way of visualizing the differences between the ensemble members and focussing on features.

...

Experiment with changing the contour value and (if time) plotting other fields.

Task 4: Visualise ensemble members and differences

So far we have been looking at reducing the information in some way to visualise the ensemble.

...

Make sure clustersId="off" for this task.

Precipitation over France

Use stamp.mv and plot total precipitation ('tp') over France (mapType=2) for 00Z 24-09-2012 (compare with Figure 2 in Pantillon).

Note, stamp_diff.mv cannot be used for 'tp' as there is no precipitation data in the analyses.

Difference stamp maps

Use the stamp_diff.mv plot to look at the differences between the ensemble members and the analysis. It can be easier to understand the difference in the ensembles by using difference stamp maps.

Compare ensemble members to analysis

After visualizing the stamp maps, it can be useful to animate a comparison of individual ensemble members to the analyses.

...

param="mslp"
ensType="pf30"

ensType="cf"

Further analysis using ensembles

param="mslp"
pf=[30,50]

...

Task 5:  Cumulative distribution function

Recap

...

Cumulative distribution function (CDF)

The figures above illustrate the relationship between a normal distribution and its associated cumulative distribution function. The CDF is constructed from the area under the probability density function.

...

For a forecast ensemble where all values were the same, the CDF would be a vertical straight line.

Plot the CDFs

This exercise uses the cdf.mv icon. Right-click, select 'Edit' and then:

...

Exercise 4: Cluster analysis

The paper by Pantillon et al describes the use of clustering to identify the main scenarios among the ensemble members.

...

• Construct your own qualitative clusters by choosing members for two clusters
• Generate clusters using principal component analysis (similar to Pantillon et al).

Task 1: Create your own clusters

Clusters can be created manually from lists of the ensemble members.

...

...

#ENS members (use ["all"] or a list of members like [1,2,3]
members_1=["cl.example.1"]
members_2=["cl.example.2"]

...

Task 2: Empirical orthogonal functions / Principal component analysis

A quantitative way of clustering an ensemble is by a principal component analysis using empirical orthogonal functions. These are computed from the differences between the ensemble members and the ensemble mean, then computing the eigenvalues and eigenfunctions of these differences (or variances) over all the members such that the difference of each member can be expressed as a linear combination of these eigenfuctions, also known as empirical orthogonal functions (EOFs).

...

Exercise 5. Percentiles and probabilities

To further compare the 2012 and 2016 ensemble forecasts, plots showing the percentile amount and probabilities above a threshold can be made for total precipitation.

...

Both these macros will use the 6-hourly total precipitation for forecast steps at 90, 96 and 102 hours, plotted over France.

Task 1. Plot percentiles of total precipitation

Edit the percentile_tp_compare.mv icon.

...

Run the macro and compare the percentiles from both the forecasts.  Change the percentiles to see how the forecasts differ.

Task 2: Plot probabilities of total precipitation

This macro will produce maps showing the probability of 6-hourly precipitation for the same area as in Task 1.

...

Exercise 6. Exploring the role of uncertainty

To further understand the impact of the different types of uncertainty (initial and model), some forecasts with OpenIFS have been made in which the uncertainty has been selectively disabled. These experiments use a 40 member ensemble and are at T319 resolution, lower than the operational ensemble.

As part of this exercise you may have run OpenIFS yourself in the class to generate another ensemble; one participant per ensemble member.

Recap

• • EDA is the  Ensemble Data Assimilation.
  • SV is the use of Singular Vectors to perturb the initial conditions.
  • SPPT is the stochastic physics parametrisation scheme.
  • SKEB is the stochastic backscatter scheme applied to the model dynamics.

Experiments available:

• Experiment id: ens_both.  EDA+SV+SPPT+SKEB  : Includes initial data uncertainty (EDA, SV) and model uncertainty (SPPT, SKEB)
  
• Experiment id: ens_initial.  EDA+SV only  : Includes only initial data uncertainty
• Experiment id: ens_model. SPPT+SKEB only : Includes model uncertainty only

...

Task 1. RMSE plumes

Use the ens_exps_rmse.mv icon and plot the RMSE curves for the different OpenIFS experiments.

...

Task 2. Ensemble spread and spaghetti plots

Use the ens_exps_to_an.mv icon and plot the ensemble spread for the different OpenIFS experiments.

...

• use the ens_part_to_all.mv icon to compare a subset of the ensemble members to that of the whole ensemble. Use the stamp_map.mv icon to determine a set of ensemble members you wish to consider (note that the stamp_map icons can be used with these OpenIFS experiments. See the comments in the files).

Task 3.  What initial perturbations are important

The objective of this task is to identify what areas of initial perturbation appeared to be important for an improved forecast in the ensemble.

...

Use the large geographical area for this task. Use the MSLP and z500 fields (and any others you think are useful).

Task 4. Non-linear development

Ensemble perturbations are applied in positive and negative pairs. This is done to centre the perturbations about the control forecast.

...

• Plot PV at 320K. What are the differences between the forecast? Upper tropospheric differences played a role in the interaction of Hurricane Nadine and the cut-off low.

Appendix

Further reading

For more information on the stochastic physics scheme in (Open)IFS, see the article:

Shutts et al, 2011, ECMWF Newsletter 129.

Acknowledgements

We gratefully acknowledge the following for their contributions in preparing these exercises. From ECMWF: Glenn Carver, Sandor Kertesz, Linus Magnusson, Iain Russell, Simon Lang, Filip Vana. From ENM/Meteo-France: Frédéric Ferry, Etienne Chabot, David Pollack and Thierry Barthet for IT support at ENM. We also thank the students who have participated in the training and workshop using this material for helping to improve it!

...