...

Obtaining the exercises

The exercises described below are available as a set of Metview macros with the accompanying data. This is available as a downloadable tarfile for use with Metview (if installed). It is also available as part of the OpenIFS/Metview virtual machine, which can be run on different operating systems.

For more details of the OpenIFS virtual machine and how to get the workshop files, please contact: openifs-support@ecmwf.int.

ECMWF operational forecasts

At the time of this case study in 2012, ECMWF operational forecasts consisted of:

...

At the time of this workshop (in 2016), the ECMWF operational forecasts has been upgraded compared to 2012 and consisted of:

...

Please follow this link to see more details on changes to the ECMWF IFS forecast system (http://www.ecmwf.int/en/forecasts/documentation-and-support/changes-ecmwf-model)

Virtual machine

If using the OpenIFS/Metview virtual machine with these exercises the recommended memory is at least 6Gb, the minimum is 4Gb. If using 4Gb, do not use more than 2 parameters per plot.

These exercises use a relatively large domain with high resolution data. Some of the plotting options can therefore require significant amounts of memory. If the virtual machine freezes when running metview, please try increasing restart the VM and increase the memory assigned to the VM.

Starting up metview

To begin:

metview &

Saving images and printing

 To save images during these exercises for discussion later, you can either use:

...

or use the following command to take a 'snapshot' of the screen:

ksnapshot

Exercise 1. The ECMWF analysis

...

...

...

Hurricane Nadine and

...

the cut-off low

...

Panel
bgColor

Hurricane Nadine and the cut-off low

Task 1: Mean-sea-level pressure and track

Plot. Right-click the mouse button on the 'an_1x1.mv' icon and select the 'Visualise' menu item (see figure right)

After a pause, this will generate a map showing mean-sea-level pressure (MSLP).

Image RemovedImage Added

Plot track of Nadine. Drag the mv_track.mv icon onto the map.

This will add the track of Hurricane Nadine. Although the full track of the tropical storm is shown from the 10-09-2012 to 04-10-2016, the ECMWF analyses (for the purpose of this study) only show 15-09-2012 to 25-09-2012.

In the plot window, use the play button in the animation controls  to animate the map and follow the development and track of Hurricane Nadine.

...

Task 2: MSLP and 500hPa geopotential height

This task creates Figure 2. from Pantillon et al.

...

Task 3: Changing geographical area

Right-click the mouse button on the 'an_1x1.mv' icon and select the select  'Edit' menu item.

In the edit window that appears

#Map type: 0=Atl-an, 1: Atl-fc, 2: France 
mapType=0

...

Animate the storm on this smaller geographical map.

Task 4: Wind fields, sea-surface temperature (SST)

The 'an_2x2.mv' icon allows for plotting up to 4 separate figures on a single frame. This task uses this icon to plot multiple fields.

Right-click on the 'an_2x2.mv' icon and select the 'Edit' menu item.

#Define plot list (min 1- max 4)
plot1=["mslp"]
plot2=["wind10"]
plot3=["speed500","z500"]
plot4=["sst"]

Click the play button at the top of the window to run this macro with the existing plots as shown above.

Note that each Each plot can be a single field or overlays of different fields as in the an_1x1.mv macro.

Wind parameters can be shown either as arrows or as wind flags ('barbs') by adding '.flag' to the end of variable name e.g. "wind10.flag".

Task 5: Satellite images

Open the folder 'satellite' by doubling clicking (scroll the window if it is not visible).

...

Use the an_1x1.mv and/or the an_2x2.mv macros to compare the ECMWF analyses with the satellite images.

Task 6: Cross-sections

The last task in this exercise is to look at cross-sections through Hurricane Nadine and the cut-off low.

Right click on the icon 'an_xs.mv', select 'Edit' and push the play button.This generates a plot with a map of MSLP, a red line and underneath a cross-section plot along that red-line.

The default plot shows potential vorticity (PV), wind vectors and potential temperature roughly through the centre of the Hurricane and the cut-off low.

Changing forecast time

The red line on the map of MSLP shows the location of the cross-section.

Changing forecast time

Cross-Cross-section data is only available every 24hrs.

This means the 'steps' value in the macros is only valid for the times:  [2012-09-20 00:00], [2012-09-21 00:00], [2012-09-22 00:00], [2012-09-23 00:00], [2012-09-24 00:00], [2012-09-25 00:00]

To change the date/time of the plot, edit the macro and change the line:

steps=[2012-09-22 00:00]

Changing fields

A smaller set reduced number of fields is available for cross-sections: temperature (t), potential temperature (pt), relative humidity (r), potential vorticity (pv), vertical velocity (w), wind-speed (speed; sqrt(u*u+v*v)) and wind vectors (wind3).

Changing cross-section location

#Cross section line [ South, West, North, East ]
line = [30,-29,45,-15]

The cross-section location (red line) can be changed in this macro by defining the end points of the line as shown above.

Remember that if the forecast time is changed, the storm centres will move and the cross-section line will need to be repositioned to follow specific features. This is not computed automatically, but must be changed by altering the coordinates above.

Exercise 2: The operational HRES forecast

Exercise 2: The operational HRES forecast

Recap

The ECMWF operational deterministic forecast is called HRES. At the time of this case study, the model ran with a spectral resolution of T1279, equivalent to 16km grid spacing.

...

Before looking at the ensemble forecasts, first understand the performance of the operational HRES forecast of the time.

Available forecast

Data is provided for a single 5 day forecast starting from 20th Sept 2012, as used in the paper by Pantillon et al.

HRES data is provided at the same resolution as the operational model, in order to give the best representation of the Hurricane and cut-off low interationsiterations. This may mean that some plotting will be slow.

...

Available parameters

A new fieldparameter is total precipitation : tp.

The fields ( parameters ) available in the analyses are available in the forecast data.

...

Available plot types

Available plot types

Forecast performance

Task 1: Forecast error

In this task, we'll look at the difference between the forecast and the analysis by using "root-mean-square error" (RMSE) curves as a way of summarising the performance of the forecast.

...

Repeat for both geographical regions: mapType=1 (Atlantic) and mapType=2 (France).

Task 2: Compare forecast to analysis

Use the hres_to_an_diff.mv icon and plot the difference map between the HRES forecast and the analysis first for z500 and then mslp (change plot1 from z500 to mslp).

If time: look at other fields to study the behaviour of  the forecast.

Task 3: Precipitation over France

This task produces a plot similar to Figure 2 in Pantillon et al.

...

As for the analyses, the macros hres_1x1.mv, hres_2x2.mv and hres_xs.mv can be used to plot and animate fields or overlays of fields from the HRES forecast.

Suggested plots for discussion

The following is a list of parameters and plots that might be useful to produce for later group discussion. Choose a few plots and use both the HRES forecast and the analyses.

...

...

Exercise 3 : The operational ensemble forecasts

Recap

Exercise 3 : The operational ensemble forecasts

Recap

• ECMWF operational ensemble forecasts treat uncertainty in both the initial data and the model.
• Initial analysis uncertainty: sampled by
• ECMWF operational ensemble forecasts treat uncertainty in both the initial data and the model.
• Initial analysis uncertainty: sampled by use of Singular Vectors (SV) and Ensemble Data Assimilation (EDA) methods. Singular Vectors are a way of representing the fastest growing modes in the initial state.
• Model uncertainty: sampled by use of stochastic parametrizations. In IFS this means the 'stochastically perturbed physical tendencies' (SPPT) and the 'spectral backscatter scheme' (SKEB)
• Ensemble mean : the average of all the ensemble members. Where the spread is high, small scale features can be smoothed out in the ensemble mean.
• Ensemble spread : the standard deviation of the ensemble members, represents how different the members are from the ensemble mean.

The ensemble forecasts

In this case study, there are two operational ensemble datasets and additional datasets created with the OpenIFS model, running at lower resolution, where the initial and model uncertainty are switched off in turn. The OpenIFS ensembles are discussed in more detail in later exercises. Please see below for more details.

An ensemble forecast consists of:

• Control forecast (unperturbed)
• Perturbed ensemble members. Each member will use slightly different initial data conditions and include model uncertainty pertubations.

2012 Operational ensemble

ens_oper: This dataset is the operational ensemble from 2012 and was used in the Pantillon et al. publication. A key feature of this ensemble is use of a climatological SST field (you should would have seen this in the earlier tasks!).

2016 Operational ensemble

ens_2016: This dataset is a reforecast of the 2012 event using the ECMWF operational ensemble from of March 2016. Two key differences between the 2016 and 2012 operational ensembles are: higher horizontal resolution, and coupling of NEMO ocean model to provide SST from the start of the forecast.

Note that the The analysis was not rerun for 20-Sept-2012. This means the reforecast using the 2016 ensemble will be using the original 2012 analyses. Also only 10 ensemble data assimilation (EDA) members were used in use at that time2012, whereas 25 would be used in the are in use for 2016 operational ensembleensembles, so each EDA member will be used multiple times for this reforecast. This will impact on the spread and clustering seen in the tasks in this exercise.

Ensemble exercise tasks

Visualising ensemble forecasts can be done in various ways. During this exercise , we will use a number of visualisation techniques in order to understand the errors and uncertainties in the forecast, we will use a number of visualisation techniques.

Key parameters: MSLP and z500, z500, and total precipitation (tp) over France.  We suggest concentrating on viewing these fields. If time, visualize other parameters (e.g. PV320K).

Image Added

Available plot types

Group working

If working in groups, each group could follow the tasks below with a different ensemble forecast. e.g. one group uses the 'ens_oper', another group uses 'ens_2016' and so on.A key question is: which is the better ensemble?

Choose your ensemble dataset by setting the value of 'expId':', either 'ens_oper' or 'ens_2016' for this exercise.

One of the OpenIFS ensembles could also be used but it's recommended one of the operational ensembles is studied first.

#The experiment. Possible values are:
# ens_oper = operational ENS
# ens_2016 = 2016 operational ENS
# ens_both = OpenIFS (EDA+SV+SPPT+SKEB)
# ens_initial = OpenIFS (EDA+SV)
# ens_model = OpenIFS (SPPT+SKEB) 

expId="ens_oper"

Ensemble forecast performance

In these tasks, the performance of the ensemble forecast is studied.

Task 1: RMSE "plumes"

This is similar to task 1 in exercise 2, except now the RMSE curves for all the ensemble members from a particular forecast will be plotted.

Using Right-click the ens_rmse.mv icon, right-click, select 'Edit' and plot the curves first for 'mslp' and then for 'z500' .Turn off clustering in the macro(change the param field to mslp, run the macro and then change to z500 and run again).

Change 'expID' for your choice of ensemble.

useClustersclustersId="off"

Clustering will be used in later tasks.

There might be some evidence of clustering in the ensemble plumes.

...

• Explore the plumes from other variables.
• Do you see the same amount of spread in RMSE from other pressure levels in the atmosphere?

Task 2: Ensemble spread

In the previous task, we have seen that introducing uncertainty into in the forecast by starting from different initial conditions and enabling the stochastic parameterizations in IFS can result in significant differences in the RMSE (for this particular case and geographical region).

This task will explore the difference in another way by looking at the 'ensemble spread'.

Use the ens_to_an.mv icon and first plot the MSLP and then z500 (set param to mslp, run the macro, then change param to z500 and run again).

This will produce plots showing: the mean of z500. This will produce plots showing: the mean of  all the ensemble forecasts, the spread of the ensemble forecasts, the operational HRES deterministic forecast and the analysis.

Change 'expId' if required.

Animate this plot to see how the spread grows.

This macro can also be used to look at collections clusters of ensemble members. It will be used later in the clustering tasks. For this task, make sure all the members of the ensemble are used.

#ENS members (use ["all"] or a list of members like [1,2,3]
members=["all"]        #[1,2,3,4,5] #or ["all"] or ["cl1cl.example.1"]

If time:

• try setting the 'members=' option to change the number of members in the spread plots.
  e.g. try a "reduced" ensemble by only using the first 5 ensemble members: "members=[1,2,3,4,5]".

Task 3: Spaghetti plots - another way to visualise spread

Task 3: Spaghetti plots - another way to visualise spread

A "spaghetti" plot is where a single contour of a parameter is plotted for all ensemble A "spaghetti" plot is where a single contour of a parameter is plotted for all ensemble members. It is another way of visualizing the differences between the ensemble members and focussing on features.

Use the ens_to_an_runs_spag.mv icon. Plot and animate either the MSLP and or z500 fields using your suitable choice for the contour level. Find a value that highlights the low pressure centres. Note that not all members may reach the low pressure set by the contour.

Note that this macro may animate slowly because of the computations required.

Experiment with changing the contour value and (if time) plotting other fields.

...

The red contour line shows the control forecast of the ensemble.

Note that this macro may animate slowly because of the computations required.

Experiment with changing the contour value and (if time) plotting other fields.

Task 4: Visualise ensemble members and differences

So far we have been looking at reducing the information in some way to visualise the ensemble.

To Stamp maps are used to visualise all the ensemble members as normal maps, we can use stamp maps. These are small, stamp sized contour maps plotted for each ensemble member using a small set of contours.

There are two icons to use, stamp.mv and stamp_diff.mv.

Plot Use stamp.mv to first plot the MSLP and then z500 fields in the ensemble and other fields of interest(set param='mslp', run the macro, then change to 'z500' and run again).

The stamp map is slow to plot as it reads a lot of data. Rather than animate each forecast step, a particular date can be set by changing the 'steps' variable.

#Define forecast steps
steps=[2012-09-24 00:00,"to",2012-09-24 00:00,"by",6]

Make sure clustersId="off" for this task.

Precipitation over France

Use stamp.mv and plot total precipitation ('tp') over France (mapType=2) for 00Z 24-09-2012 (compare with Figure 2 in Pantillon).

Note, stamp_diff.mv cannot be used for 'tp' as there is no precipitation data in the analyses. 

Difference stamp maps

Use the stamp_diff.mv plot to look at the differences between the ensemble members and the analysis. It can be easier to understand the difference in the ensembles by using difference stamp maps. 

If time:

Compare ensemble members to analysis

After visualizing the stamp maps, it can be useful to animate a comparison of individual ensemble members to the analyses.

Use the macros to see how the perturbations are evolving; use ens_to_an_diff.mv and pf_to_cf_diff.mv can be used to compare individual ensemble members to the analyses.

Find ensemble members that appear to produce a better forecast and look to see how the initial development in these members differs. Start by using a single lead time and examine the forecast on the 28th.

...

.

...

param="mslp"
ensType="pf30"

ensType="cf"

Further analysis using ensembles

...

...

Task 5:  Cumulative distribution function

Recap

Cumulative distribution function (CDF)

The figures above illustrate the relationship between a normal distribution and its associated cumulative distribution function.The CDF is constructed from the area under the probability density function.

The CDF gives the probability that a value on the curve will be found to have a value less than or equal to the corresponding value on the x-axis. For example, in the figure, the probability for values less than or equal to X=0 is 50%.

The shape of the CDF curve is related to the shape of the normal distribution. The width of the CDF curve is directly related to the value of the standard deviation of the probability distribution function.

For an ensemble, the width is therefore related to the 'ensemble spread'.

For a forecast ensemble where all values were the same, the CDF would be a vertical straight line.

Plot the CDFs

Image Removed

This exercise uses the cdf.mv icon. Right-click, select 'Edit' and then:

• Plot the CDF of MSLP for the 3 locations listed in the macro.e.g. Reading, Amsterdam, Copenhagen.  TODO; what parameters and what location(s)?
• If time, change the forecast run date and compare the CDF for the different forecasts.

 Forecasting during HyMEX : Work in teams for group discussion

Ensemble forecasts can be used to help forecasting. This exercise discusses a real-world case of forecasting during HyMEX.

Exercise 4: Computing the EOFs and clusters

TO BE DONE

Exercise 4. Exploring the role of uncertainty using OpenIFS forecasts

To further understand the impact of the different types of uncertainty (initial and model), some forecasts with OpenIFS have been made in which the uncertainty has been selectively disabled. These experiments use a 40 member ensemble and are at T319 resolution, lower than the operational ensemble.

As part of this exercise you may have run OpenIFS yourself in the class to generate another ensemble; one participant per ensemble member.

Recap

• • EDA is the  Ensemble Data Assimilation.
  • SV is the use of Singular Vectors to perturb the initial conditions.
  • SPPT is the stochastic physics parametrisation scheme.
  • SKEB is the stochastic backscatter scheme applied to the model dynamics.

Experiments available:

• Experiment id: ens_both.  EDA+SV+SPPT+SKEB  : Includes initial data uncertainty (EDA, SV) and model uncertainty (SPPT, SKEB)
  
• Experiment id: ens_initial.  EDA+SV only  : Includes only initial data uncertainty
• Experiment id: ens_model. SPPT+SKEB only : Includes model uncertainty only
• Experiment id: ens_oifs. SPPT+SKEB only, class ensemble  : this is the result of the previous task using the ensemble run by the class

These are at T319 with start dates: 24/25/26/27 Oct at 00UTC for a forecast length of 5 days with 3hrly output.

The aim of this exercise is to use the same visualisation and investigation as in the previous exercises to understand the impact the different types of uncertainty make on the forecast.

A key difference between this exercise and the previous one is that these forecasts have been run at a lower horizontal resolution. In the exercises below, it will be instructive to compare with the operational ensemble plots from the previous exercise.

Task 1. RMSE plumes

Use the ens_exps_rmse.mv icon and plot the RMSE curves for the different OpenIFS experiments.

Q. Compare the spread from the different experiments.
Q. The OpenIFS experiments were at a lower horizontal resolution.  How does the RMSE spread compare between the 'ens_oper' and 'ens_both' experiments?

If time: change the 'run=' line to select a different forecast lead-time (run).

Task 2. Ensemble spread and spaghetti plots

Use the ens_exps_to_an.mv icon and plot the ensemble spread for the different OpenIFS experiments.

Also use the ens_exps_to_an_spag.mv icon to view the spaghetti plots for MSLP for the different OpenIFS experiments.

Q. What is the impact of reducing the resolution of the forecasts? (hint: compare the spaghetti plots of MSLP with those from the previous exercise).
Q. How does changing the representation of uncertainty affect the spread?
Q. Which of the experiments ens_initial and ens_model gives the better spread?
Q. Is it possible to determine whether initial uncertainty or model uncertainty is more or less important in the forecast error?

If time:

• use the ens_part_to_all.mv icon to compare a subset of the ensemble members to that of the whole ensemble. Use the stamp_map.mv icon to determine a set of ensemble members you wish to consider (note that the stamp_map icons can be used with these OpenIFS experiments. See the comments in the files).

Task 3.  What initial perturbations are important

The objective of this task is to identify what areas of initial perturbation appeared to be important for an improved forecast in the ensemble.

Using the macros provided, find an ensemble member(s) that gave a consistently improved forecast and take the difference from the control. Step back to the beginning of the forecast and look to see where the difference originates from. 

Use the large geographical area for this task. It is only possible to use the MSLP parameter for this task as it is the only one provided for the ensemble.

Task 4. Non-linear development

Ensemble perturbations are applied in positive and negative pairs. This is done to centre the perturbations about the control forecast.

So, for each computed perturbation, two perturbed initial fields are created e.g. ensemble members 1 & 2 are a pair, where number 1 is a positive difference compared to the control and 2 is a negative difference.

• Choose an odd & even ensemble pair (use the stamp plots). Use the appropriate icon to compute the difference of the members from the ensemble control forecast.
• Study the development of these differences using the MSLP and wind fields. If the error growth is linear the differences will be the same but of opposite sign. Non-linearity will result in different patterns in the difference maps.
• Repeat looking at one of the other forecasts. How does it vary between the different forecasts?

If time:

• Plot PV at 330K. What are the differences between the forecast? Upper tropospheric differences played a role in the development of this shallow fast moving cyclone.

Notes from Frederic: email 7/4/16

day 2

See above. My preference after talking with people here is to use the comparison between 2012 operational ensemble and 2016 operational ensemble. The lower res (T319) ensembles; control (EDA+SV), (SPPT_SKEB) ensembles for this case are running now and we can include the data (as long as filesize does not become an issue). But honestly, I do not think there will be time. I will leave it to you to decide!

...

Exercise 2.

Extended deterministic forecast : 20-28 September just for MSLP : Etienne told me that the ECMWF model of the 20 000UTC proposed a very extreme situation on the 28th, with a storm over Gibraltar. This would be a way to illustrate the limits of a deterministic approach.

.

As I am writing I am beginning to wonder if we should not make 2 groups : one for task 4 and one for task 5. Tasks 1-3 would be for all students. This would allow to keep the CDO task. What do you think ?

On 04/05/16 15:25, FERRY Frédéric wrote:

*T1279 Analysis* : 20121020 0000UTC to 20121025 0000UTC --> Only the 20
september analysis will be looked at but I assume you need to get the
other analysis to compute the RMSE in day 2 ?
*Extended analysis* : 15-20 September just for MSLP and T2m (or better
the SST) --> Nadine tracking before the 20th

...

*Extended deterministic forecast* : 20-28 September just for MSLP :
Etienne told me that the ECMWF model of the 20 000UTC proposed a very
extreme situation on the 28th, with a storm over Gibraltar. This would
be a way to illustrate the limits of a deterministic approach.

...

All ok apart from:

3 : Equivalent potential temperature at 850 hPa  + winds at 850 hPa +
vertical velocity at 600hPa + MSLP in background --> focussing on the

Can we use 700hPa VV instead of 600, to be consistent with data on other levels? We will need VV on multiple levels in order to plot the x-sections (see below), though these will only be available 00Z on each day. The horiz. maps will have VV available 6hrly but on selected levels only (we're proposing 200, 500, 700, 850).

Proposed tasks for Day 1 :
5 : Beyond D+5 deterministic scenario : MSLP only

...

Concerning the ensemble runs, 6 hourly data is OK. If you have space on
the VM it would be interesting to go up to D+10 (or D+15). This would
allow to try and look at the extreme member over Gibraltar on the 28
September.

...

param="mslp"
pf=[30,50]

Task 5:  Cumulative distribution function

Recap

Cumulative distribution function (CDF)

The figures above illustrate the relationship between a normal distribution and its associated cumulative distribution function. The CDF is constructed from the area under the probability density function.

The CDF gives the probability that a value on the curve will be found to have a value less than or equal to the corresponding value on the x-axis. For example, in the figure, the probability for values less than or equal to X=0 is 50%.

The shape of the CDF curve is related to the shape of the normal distribution. The width of the CDF curve is directly related to the value of the standard deviation of the probability distribution function.

For an ensemble, the width is therefore related to the 'ensemble spread'.

For a forecast ensemble where all values were the same, the CDF would be a vertical straight line.

Plot the CDFs

Image Added

This exercise uses the cdf.mv icon. Right-click, select 'Edit' and then:

• Plot the CDF of MSLP for Toulouse for your choice of ensemble
• Find a latitude/longitude point in the area of intense precipitation on 12Z 24/9/2012 (see Figure 2(c) Pantillon et al) and plot the CDF for MSLP (set station=[lat,lon] in the macro cdf.mv)

Note that only MSLP, 2m temperature (t2) and 10m wind-speed (speed10) are available for the CDF.

Make sure useClusters='off'.

Exercise 4: Cluster analysis

The paper by Pantillon et al describes the use of clustering to identify the main scenarios among the ensemble members.

This exercise repeats some of the plots from the previous one but this time with clustering enabled.

Using clustering will highlight the ensemble members in each cluster in the plots.

In this exercise you will:

• Construct your own qualitative clusters by choosing members for two clusters
• Generate clusters using principal component analysis (similar to Pantillon et al).

Task 1: Create your own clusters

Clusters can be created manually from lists of the ensemble members.

Choose members for two clusters. The stamp maps are useful for this task.

From the stamp map of z500 at 24/9/2012 (t+96), identify ensemble members that represent the two most likely forecast scenarios.

It is usual to create clusters from z500 as it represents the large-scale flow and is not a noisy field. However, for this particular case study, the stamp map of 'tp' (total precipitation) over France is also very indicative of the distinct forecast scenarios. You might also try using other fields, such as 'mslp' or 'pv320K' to compare.

1#   2  3  4  9  22 33 40
2#   10 11 12 31 49

#ENS members (use ["all"] or a list of members like [1,2,3]
members_1=["cl.example.1"]
members_2=["cl.example.2"]

Task 2: Empirical orthogonal functions / Principal component analysis

A quantitative way of clustering an ensemble is by a principal component analysis using empirical orthogonal functions. These are computed from the differences between the ensemble members and the ensemble mean, then computing the eigenvalues and eigenfunctions of these differences (or variances) over all the members such that the difference of each member can be expressed as a linear combination of these eigenfuctions, also known as empirical orthogonal functions (EOFs).

Although geopotential height at 500hPa at 00 24/9/2012 is used in the paper by Pantillon et al. as it gives the best results, the steps described below can be used for any parameter at any step.

Image Added

The eof.mv macro computes the EOFs and the clustering.

param="z500"
expId="ens_oper"
steps=[2012-09-24 00:00]

ens_oper_cluster.eof.txt

#ENS members (use ["all"] or a list of members like [1,2,3]
members_1=["cl.eof.1"]
members_2=["cl.eof.2"]

  clusterNum=2

  clusterNum=3

param="z500.s"
expId="ens_oper"
members_1=["cl.eof.1"]
members_2=["cl.eof.3"]

Exercise 5. Percentiles and probabilities

To further compare the 2012 and 2016 ensemble forecasts, plots showing the percentile amount and probabilities above a threshold can be made for total precipitation.

Use these icons:

Image Added

Both these macros will use the 6-hourly total precipitation for forecast steps at 90, 96 and 102 hours, plotted over France.

Task 1. Plot percentiles of total precipitation

Edit the percentile_tp_compare.mv icon.

Set the percentile for the total precipitation to 75%:

#The percentile of ENS precipitation forecast
perc=75

Run the macro and compare the percentiles from both the forecasts.  Change the percentiles to see how the forecasts differ.

Task 2: Plot probabilities of total precipitation

This macro will produce maps showing the probability of 6-hourly precipitation for the same area as in Task 1.

In this case, the maps show the probability that total precipitation exceeds a threshold expressed in mm.

Edit the prob_tp_compare.mv and set the probability to 20mm:

#The probability of precipitation greater than
prob=20

Run the macro and view the map. Try changing the threshold value and run.

Exercise 6. Exploring the role of uncertainty

To further understand the impact of the different types of uncertainty (initial and model), some forecasts with OpenIFS have been made in which the uncertainty has been selectively disabled. These experiments use a 40 member ensemble and are at T319 resolution, lower than the operational ensemble.

As part of this exercise you may have run OpenIFS yourself in the class to generate another ensemble; one participant per ensemble member.

Recap

• • EDA is the  Ensemble Data Assimilation.
  • SV is the use of Singular Vectors to perturb the initial conditions.
  • SPPT is the stochastic physics parametrisation scheme.
  • SKEB is the stochastic backscatter scheme applied to the model dynamics.

Experiments available:

• Experiment id: ens_both.  EDA+SV+SPPT+SKEB  : Includes initial data uncertainty (EDA, SV) and model uncertainty (SPPT, SKEB)
  
• Experiment id: ens_initial.  EDA+SV only  : Includes only initial data uncertainty
• Experiment id: ens_model. SPPT+SKEB only : Includes model uncertainty only

The aim of this exercise is to use the same visualisation and investigation as in the previous exercises to understand the impact the different types of uncertainty make on the forecast.

A key difference between this exercise and the previous one is that these forecasts have been run at a lower horizontal resolution. In the exercises below, it will be instructive to compare with the operational ensemble plots from the previous exercise.

Task 1. RMSE plumes

Use the ens_exps_rmse.mv icon and plot the RMSE curves for the different OpenIFS experiments.

Compare the spread from the different experiments.

Task 2. Ensemble spread and spaghetti plots

Use the ens_exps_to_an.mv icon and plot the ensemble spread for the different OpenIFS experiments.

Also use the ens_exps_to_an_spag.mv icon to view the spaghetti plots for MSLP for the different OpenIFS experiments.

If time:

• use the ens_part_to_all.mv icon to compare a subset of the ensemble members to that of the whole ensemble. Use the stamp_map.mv icon to determine a set of ensemble members you wish to consider (note that the stamp_map icons can be used with these OpenIFS experiments. See the comments in the files).

Task 3.  What initial perturbations are important

The objective of this task is to identify what areas of initial perturbation appeared to be important for an improved forecast in the ensemble.

Using the macros provided:

• Find an ensemble member(s) that gave a consistently improved forecast and take the difference from the control.
• Step back to the beginning of the forecast and look to see where the difference originates from. 

Use the large geographical area for this task. Use the MSLP and z500 fields (and any others you think are useful).

Task 4. Non-linear development

Ensemble perturbations are applied in positive and negative pairs. This is done to centre the perturbations about the control forecast.

So, for each computed perturbation, two perturbed initial fields are created e.g. ensemble members 1 & 2 are a pair, where number 1 is a positive difference compared to the control and 2 is a negative difference.

• Choose an odd & even ensemble pair (use the stamp plots). Use the appropriate icon to compute the difference of the members from the ensemble control forecast.
• Study the development of these differences using the MSLP and wind fields. If the error growth is linear the differences will be the same but of opposite sign. Non-linearity will result in different patterns in the difference maps.
• Repeat looking at one of the other forecasts. How does it vary between the different forecasts?

If time:

• Plot PV at 320K. What are the differences between the forecast? Upper tropospheric differences played a role in the interaction of Hurricane Nadine and the cut-off low.

Appendix

Further reading

For more information on the stochastic physics scheme in (Open)IFS, see the article:

Shutts et al, 2011, ECMWF Newsletter 129.

Acknowledgements

We gratefully acknowledge the following for their contributions in preparing these exercises. From ECMWF: Glenn Carver, Sandor Kertesz, Linus Magnusson, Iain Russell, Simon Lang, Filip Vana. From ENM/Meteo-France: Frédéric Ferry, Etienne Chabot, David Pollack and Thierry Barthet for IT support at ENM. We also thank the students who have participated in the training and workshop using this material for helping to improve it!

...

Task 4 : PCA and clustering

...

So day 2 « menu » would be :
Looking at the ensemble products and the cluster products and making a decision for Hymex field campaign —>  They will have Etienne's forecaster feedback the day after. 
I’ll ask Etienne his ideas for the workshop tasks on this topic.
Looking at the impact of ocean coupling on the ensemble prediction.

Tell me if you manage to redo the clustering and the composites in Metview, I hope it will work.
If you manage to redo figures 5 6 7 8 and 10 I think I’ll have to tell Jean-Pierre to focus more during his presentation on the vortex-vortex interaction and the CRM sensitivity experiments he made. This will leave the cluster analysis for the students to discover.
 

Hello Glenn,

Here are a few comments concerning your previous emails :

2- Véronique Ducrocq could play the role of an HyMeX operation director being the client of the students' forecast. This forecasting exercise could be done by the 8 students following the forecasting option (with me as their "teacher"), whereas the 18 others (informatic or statistic options) could keep doing more sensitivity tests while manipulating the code of the model (with Frédéric and you).
3- It would be very interesting to briefly tackle with the ECMWF Data Targeting System which was one of the observation strategies used during HyMeX SOP1. 
I precisely asked Véronique Ducrocq to speak about DTS during her presentation on Day 1.
4- ARPEGE and IFS deterministic charts are available at the French Met School between 18th and 24th sept (except the 20th runs unfortunately !). As far as I was the HyMeX forecaster myself before the 24th sept. event, I would be very interesting in the MSLP fields from the 20th 00UTC run between 25th and 28th sept. , in order to be able to illustrate (in my own Day 3 presentation) the propagation of this impressive "Gibraltar storm" I mentionned into my daily meeting report. A 6h step would be perfect, even if it is only a paper-scanned version...

Appendix

Datasets available

The following datasets are available on the Virtual Machine for this workshop:

Further reading

For more information on the stochastic physics scheme in (Open)IFS, see the article:

Shutts et al, 2011, ECMWF Newsletter 129.

Acknowledgements

We gratefully acknowledge the following for their contributions in preparing these exercises. From ECMWF: Glenn Carver, Sandor Kertesz, Linus Magnusson, Iain Russell, Simon Lang, Filip Vana. From ENM/Meteo-France: Frédéric Ferry, Etienne Chabot, David Pollack and Thierry Barthet for IT support at ENM.

...