...

...

• HRES : spectral T1279 (16km grid) highest resolution 10 day deterministic forecast.
• ENS :   spectral T639 (34km 31km grid) resolution ensemble forecast (50 members) is run for days 1-10 of the forecast, T319 (70km) is run for days 11-15.

In 2016, the ECMWF operational forecasts has been was upgraded compared to 2012 and consisted of:

...

These exercises use a relatively large domain with high resolution data. Some of the plotting options can therefore require significant amounts of memory. If the virtual machine freezes when running metviewMetview, please try increasing the memory assigned to the VM.

...

To save any other images during these exercises for discussion later, you can either use:

...

In this exercise, the development of Hurricane Nadine and the cut-off flow up to the 20th September 2012 is studied.

Begin by entering the folder labelled 'Analysis':

...

This task will look at the synoptic development of Hurricane Nadine and the cutoff low up to 00Z, 20th September 2012. The forecasts in the next exercises start from this time and date.

...

...

plot1=["pv320K"]

plot1=["z500.s","pv320K","mslp"]

Task 3: Changing the map geographical area

Task 3: Changing the map geographical area

Right-click on Right-click on an_1x1.mv icon and select  'Edit'.

...

...

...

This task focuses on the fate of Nadine and examines vertical PV cross-sections of Nadine and the cutoff at different forecast times to characterize the diabatic warm core PV tower of Nadine compared to the upper level PV cold core of the cutoff.

Right-click on the icon 'hres_xs.mv' icon, select 'Edit' and push the play button.

...

Changing forecast time

Cross-section data is only available every 24hrs until the 30th Sept 00Z (step 240).

...

steps=[2012-09-26 00:00]

Changing cross-section location

To change the forecast length for hres_1x1.mv and hres_2x2.mv, right-click, select Edit and change:

fclen=5

to

fclen=10

Changing cross-section location

#Cross section line [ South, West, North, East ]
line = [30,-29,45,-15]

...

If the forecast time is changed, the storm centres will move and the cross-section line will need to be repositioned to follow specific features. This is not computed automatically, but must be changed by altering the coordinates above. Use the cursor data icon Image Added to find the new position of the line.

Change the forecast time again to day+8 (28th Sept), or a different date if you are interested, relocate and plot the cross-section of Nadine and the low pressure system. Use the hres_1x1.mv icon from task 1 if you need to follow location of Nadine.

...

Cyclone phase space (CPS) diagrams

An objectively defined cyclone phase space (CPS) is described using the storm-motion-relative thickness asymmetry (symmetric/non-frontal versus asymmetric/frontal) and vertical derivative of horizontal height gradient (cold- versus warm-core structure via the thermal wind relationship). A cyclone's life cycle can then be analyzed within this phase space, providing insight into the cyclone structural evolution.

...

...

Task 1: Ensemble spread

Use the ens_maps.mv icon and plot the MSLP and z500. This will produce plots showing: the mean of all the ensemble forecasts, the spread of the ensemble forecasts and the operational HRES deterministic forecast.

...

Compare ensemble members to the deterministic and control forecast

After visualizing the stamp maps, it can be useful to animate a comparison of individual ensemble members to the HRES and ensemble control deterministic forecastforecasts.

This can help in identifying individual ensemble members that produce a better different forecast than the control or HRES forecast.

Icons Use the ens_to_ref_diff.mv and icon to compare an ensemble member to the HRES forecast. Use pf_to_cf_diff.mvcan be used to compare ensemble members to the control forecast.

param="mslp"
ensType="pf30"

ensType="cf"

Further analysis using ensembles

param="mslp"
pf=[30,50]

Sea-surface temperature

Compare the SST parameter used for the ens_oper and ens_2016 ensemble forecasts. The 2016 reforecast of this case study used a coupled ocean model unlike the 2012 ensemble and HRES forecast that used climatology for the first 5 days.

Cross-sections of

...

an ensemble member

To show a cross-section of a particular ensemble member, use the macro ens_xs.mv.

This works in the same way as the hres_xs.mv macros.

Identifying sensitive regions

Find ensemble members that appear to produce a better forecast and look to see how the initial development in these members differs.

...

Exercise 4: CDF, percentiles and probabilities

Recap

...

Exercise 4: CDF, percentiles and probabilities

Recap

Cumulative distribution function

...

The figures

...

Image Removed

_{Figure from Wikipedia.}

Cumulative distribution function

The figures above illustrate the relationship between a normal distribution and its associated cumulative distribution function. The CDF is constructed from the area under the probability density function.

...

Using the probability map, click the cursor data icon and move the pointer over the map for +96h and select choose a location in the region of highest rainfall. Do this for both the 2012 or and 2016 ensemble map.

Make a note of the latitude and longitude coordinates. The highest rainfall area was approximately over the Cévennes mountains , approximately ( 44°25′34″N 03°44′21″E ).

Edit prob_tp_compare.mv and set the location:

...

and replot the map. A small purple dot will appear at the location specified. If the dot is not in the right location, change it and replot.

...

You should now have the probability values that total precipitation will exceed 10mm, 20mm and 30mm, for both the 2012 and 2016 ensembles, for forecast time +96 hours for your chosen location.

Task 2: Plot the CDF

This exercise uses the cdf.mv icon.

...

Set the percentile for the total precipitation to 70% and specify the location as in Task  Tasks 1 & 2:

#The percentile of ENS precipitation forecast
perc=70

location=[44.0,4.1]   # [ lat, lon ] -- use your own values!

...

Exercise 5: Cluster analysis

...

• Construct your own qualitative clusters by choosing members for two clusters.
• Generate clusters using principal component analysis.

...

You can choose any parameter to construct the clusters from, if you think another parameter shows a clear clustering behaviour.

How to create your own cluster

...

The filename is important!
The first part of the name 'ens_oper' refers to the ensemble dataset and must match the name used in the plotting macro. 
The 'example' part of the filename can be changed to your choice and should match the 'clustersId'.
As an example a filename of: ens_both_cluster.fred.txt would require 'expId=ens_both', 'clustersId=fred' in the macro.

Plot

...

maps of parameters as clusters

Image Added

The macro cluster_to_ref.mv can be used to plot maps of parameters as clusters and compared to the ensemble control forecast and the HRES forecast.

Use cluster_to_ref.mv to plot z500 maps of your two clusters.

If your

Use the clusters of ensemble members you have created in ens_oper_cluster.example.txt.

Set clustersId='example' in each of the ensemble plotting macros to enable cluster highlighting.

Replot ensembles:

Stamp maps: the stamp maps will be reordered such at the ensemble members will be groups according to their cluster. Applies to stamp.mv icon. This will make it easier to see the forecast scenarios according to your clustering.

Spaghetti maps: with clusters enabled, two additional maps are produced which show the contour lines for each cluster.

Plot maps of parameters as clusters

The macro cluster_to_an.mv can be used to plot maps of parameters as clusters and compared to the HRES forecasts. TODO: rename cluster_to_an.mv to cluster_hres.mv and comment out plot of AN.

Use cluster_to_an.mv to plot z500 maps of your two clusters.

If your cluster definition file is called 'ens_oper_cluster.example.txt', then Edit cluster_to_anref.mv and set:

#ENS members (use ["all"] or a list of members like [1,2,3]
members_1=["cl.example.1"]
members_2=["cl.example.2"]

If your cluster definition file is has another name, e.g. ens_oper_cluster.fred.txt, then members_1=["cl.fred.1"].

Plot other parameters:

Plot total precipitation for France (mapType=2). Compare with Figure 8. in Pantillon et al.

has another name, e.g. ens_oper_cluster.fred.txt, then members_1=["cl.fred.1"].

Plot ensembles with clusters

In this part of the task, redo the plots from the previous exercise which looked at ways of plotting ensemble data, but this time with clustering enabled.

Use the clusters of ensemble members you have created in ens_oper_cluster.example.txt.

Set clustersId='example' in each of the stamp.mv and ens_to_ref_spag.mv to enable cluster highlighting.

If time, also try the ens_part_to_all.mv icon. This compares the spread and mean of part of the ensemble to the full ensemble.

Plot other parameters

Use the stamp.mv icon and change it to plot the total precipitation over France with clusters enabled.e.g.

param="tp"
expId="ens_oper"
mapType=2
clustersId="example"

If you choice of clustering is accurate, you should see a clear separation of precipitation over France between the two clusters.

Task 2: Empirical orthogonal functions / Principal component analysis

A quantitative way of clustering an ensemble is by computing uses empirical orthogonal functions from the differences between the ensemble members and the control forecast. and then using an algorithm to determine the clusters from each ensemble as projected in EOF space (mathematically).

As a smooth dynamical field, Although geopotential height at 500hPa at 00 00Z 24/9/2012 is recommend (it used in the paper by Pantillon et al.), but the steps described below can be used for any parameter at any step.

Image Removed

Image Added

The eof.mv macro computes the EOFs and the clustering.

...

param="z500"
expId="ens_oper"
steps=[2012-09-24 00:00]

...

ens_oper_cluster.eof.txt

...

...

#ENS members (use ["all"] or a list of members like [1,2,3]
members_1=["cl.eof.1"]
members_2=["cl.eof.2"]

...

If time, change the date/time used to compute the clusters. How does the variance explained by the first two clusters change?

...

Is geopotential the best parameter to use?

...

Changing the number of clusters

To change the number of clusters created by the EOF analysis,

...

edit eof.mv.

...

Change:

  clusterNum=2

to

  clusterNum=3

Now if you run the eof.mv macro, it will generate a text file, such as ens_oper.eof.txt with 3 lines, one for each cluster. It will also show the 3 clusters as different colours.

You can use the 3 clusters in the cluster_to_

...

ref.mv macro, for example:

param="z500.s"
expId="ens_oper"
members_1=["cl.eof.1"]
members_2=["cl.eof.3"]

would plot the mean of the members in the first and the third clusters (it's not possible to plot all three clusters together).

You can have as many clusters as you like but it does not make sense to go beyond 3 or 4 clusters.

Exercise 6. Assessment of forecast errors

Exercise 6. Assessment of forecast errors

In this exercise, the analyses covering the forecast period are now available to see how Nadine and the cut-off low actually behaved.

Various In this exercise, various methods for presenting the forecast error are presentedused in the tasks below.  The clusters created in the previous exercise above can also be used.

...

TODO:

...

Enter the 'Forecast errors' folder in the openifs_2018 folder to start work on this exercise.

Image Added

Task 1: Satellite images

Open the folder 'satellite'the folder 'satellite' (scroll the window if it is not visible).

This folder contains satellite images (water vapour, infra-red, false colour) for 00Z on 20-09-2012 and animations of the infra-red and water vapour images.

Double click the images to display them and watch the observed behaviour of Nadine and the cut-off low.

Task 2: Analyses from 20th Sept.

The first task is to now look at the analyses from 20th Sept to observe what actually happened.

Enter the folder 'Forecast_errors'  (TODO: include icon graphic)

.

Task 2: Analyses from 20th Sept.

Now look at the analyses from 20th Sept to observe what actually happened.

As in Exercise 1, task 1. Right-click an_1x1.mv, Edit and set the plot to show MSLP and geopotential at 500hPa:

plot1=["z500.s","mslp"]

Click the play button and animate the plot button and animate the plot to watch how Nadine and the cut-off low behave.

Drop the mv_track.mv icon to overlay the track of Nadine onto the map.

If time, use the other icons such as an_2x2.mv and an_xs.mv to look at the cross-section through the analyses and compare to the forecast cross-sections from the previous exercises.

Task 3: Compare forecast to analysis

Plot forecast difference maps to see how and when the forecast differed from the analyses.

Use the hres_to_an_diff.mv icon and plot the differences between the z500, MSLP and other fields to how the forecast differences evolve.

Also try the ctrl_to_an_diff.mv icon which plots the difference but this time using the ensemble control forecast.

Task 4: Forecast error curve

...

Image Removed

...

In this task, we'll look at the difference between the forecast and the analysis by using "root-mean-square error" (RMSE) curves as a way of summarising the performance of the forecast.

Root-mean square error curves are a standard measure to determine forecast error compared to the analysis and several of the exercises will use them. The RMSE is computed by taking the square-root of the mean of the forecast difference between the HRES and analyses. RMSE of the 500hPa geopotential is a standard measure for assessing forecast model performance at ECMWF (for more information see: http://www.ecmwf.int/en/forecasts/quality-our-forecasts)-our-forecasts).

Right-click the hres_rmse.mv icon, select 'Edit' and plot the RMSE curve for z500.

...

would use the cluster definitions in the file: ens_oper_cluster.eof.txt (for the 2012 operational ensemble).

The cluster files are 'linked' from the Cluster folder, but if they do not work, just copy the cluster file (e.g. ens_oper_cluster.eof.txt) to the Forecast_errors folder.

...

Appendix

Further reading

For more information on the stochastic physics scheme in IFS, see the article:

Shutts et al, 2011, ECMWF Newsletter 129.

Acknowledgements

We gratefully acknowledge the following for their contributions in preparing these exercises. From ECMWF: Glenn Carver, Gabriella Szepszo, Sandor Kertesz, Linus Magnusson, Iain Russell, Simon Lang, Filip Vana. From ENM/Meteo-France: Frédéric Ferry, Etienne Chabot, David Pollack and Thierry Barthet for IT support at ENM.

...