This page describes how to extract forcing data to run the Single Column Model from the ECMWF data archive.

Note that you need an account on the ECMWF 'ecgate' system in order to be able to run the extraction steps below.

Initial setup

There are a number of required steps before the extraction can be done. These are steps that only need to be done once.

Access to 'ifs' group

You need to be in the 'ifs' unix group in order to run the scripts to generate the SCM forcing.

To check you have access, login to ecgate, and type:

groups

If you do not see 'ifs' listed in the output, please contact ECMWF user support (advisory@ecmwf.int) and ask to be added, explaining why.

ssh keys

Make sure you have set up your ssh_keys so that you don't need to use password to go from ecgate to cca and from cca to ecgate:

ssh-keygen (no passphrase)
ssh-copy-id -i ~/.ssh/id_rsa.pub name_of_remote_host

Install workflow suite

The forcing data is created using a 'workflow' developed at ECMWF.

A workflow is a defined order in which a number of separate scripts are run. The workflow software used at ECMWF is called 'sms'. SMS is a 'workflow manager' that a user interacts with to control the executation of scripts (or 'tasks') in the workflow.

The workflow 'suite', the sequence of the scripts that will be run, is defined in a .def file.

The following steps explain how to install and customize this workflow and it's scripts for your use.

Download the workflow suite: sms.tar.gz and unpack it to your ecgate user home directory:
```
cd $HOME
mkdir sms
tar zxvf sms.tar.gz
```
Choose either ECMWF operational analysis (OPER-cray.def) or the ERA-Interim analyses (ERAI-cray.def) and copy to 'cray.def'.
e.g. for operational analyses
```
cp OPER-cray.def cray.def
```
Edit cray.def to correct it for your user details:

i) replace all '/home/ms/se/usr' by your actual home directory path where 'usr' is your username (e.g. /home/ms/se/batman)
ii) replace 'acnt' by your billing units account.
To find your account, type the command:
```
account -s your_user_name
```
where your_user_name is replaced by your username (e.g. batman).
iii) replace 'usr' in OWNER and USER to your username.
iv) set LOGDIR to a directory on the HPC (where this job runs) where you want to store the log files.
In the directory, 'sms/cray', edit the files: sms/cray/getdata_scm.sms and sms/cray/getmars_scm.sms to set the unix directory paths to your user account:
Change these lines in both files:
```
export SCRATCH=/ws/scratch/rd/rdx     # only useful for $SCMDATA definition
export SCMDATA=$SCRATCH/pafv/data     # will define the place your resulting profiles are stored
TMPDIR=/scratch/rd/pafv/scm_cray      # working directory on cca
```
It is important that these directories are set the same in both getdata_scm.sms and getmars_scm.sms.
In the directory sms/include_cray, you need copy the script files from either 'OPER' if you want to use operational analyses, or 'ERAI-0_12' if you want to use ERA-Interim.
Copy the files into the sms/include_cray directory:
e.g. for operational analyses
```
cd sms/include_cray
cd OPER					# we will use the ECMWF operational analysis to create the forcing files
cp getini1c* ..         # copy the files to sms/include_cray
```
Edit the files: sms/include_cray/getini1c_setup_cycle_validation_SCM_FC_cray*, and replace the text 'usr' in "EXPVER=usr" to your real username.
Invoke the sms server to find your 'sms identifier', which is the number in the command output after the PROGNUM parameter, needed to log in to ecgate and set up server.
```
/usr/local/apps/sms/current/bin/sms_start
```
This command starts up your personal workflow manager on ecgate.
Set up server in the xcdp viewer. This is a visual view of the workflow and the tasks as they execute.
```
xcdp &  # run the workflow viewer 'xcdp' in the background
```
Once the xcdp window appears, add your server to the list of servers known to xcdp by the menus: Edit → Preferences → Servers
Choose 'Name', state 'Host server' (e.g. ecgate) and set PROGNUM value (step 7) as Number'.

Set up the job to retrieve forcing

Define the starting day(s) of your experiment(s) by adapting the YMD variable in sms/cray.def. The current setting uses the starting date of 1st of April 2016:
```
define rcmd { repeat string YMD 20160401 }
```
If you want to do multiple experiments simply add other dates in the following way:
```
define rcmd { repeat string YMD 20160401 20160402 20160403 }
```

Edit following two files:

sms/include_cray/getini1c_setup_cycle_validation_SCM_FC_cray
sms/include_cray/getini1c_setup_cycle_validation_SCM_FC_cray.dataonly

between the lines labelled as "definition area" to reflect your desired points.

If using only one time step keep the forward slash '/' afterwards for correct use of sed command.
If you request fields at 1 hour frequency, some fields won't be available from the MARS archive. It is better to use a 3 hour interval for the operational forecast.
Note the start time has to be either 00 or 12 for the operational forecast archive. While multiple dates can be specified, there is no way to have multiple starting time in the similar way.
getini1c_mars_run_validation_SCM_FC has information on which variables are read from MARS archive

Run the job to retrieve forcing

Make sure the sms server is already running by using the following command. If it is, this command will do nothing:
```
/usr/local/apps/sms/current/bin/sms_start
```

Load your suite into the SMS server and start running it:

cd ~/sms                               #  change to the directory with the SMS scripts
cdp                                    #  start the workflow manager 'cdp'
CDP> set SMS_PROG your_sms_identifier  #  the command prompt will change to 'CDF'; set the SMS_PROG value

where your_sms_identifier is given in the the output after invoking the sms server, look for variable called PROGNUM.

login -t 60 ecgate your_user_name 314159    # password 314159, can also be 1 instead of 314159

If errors occur, likely need to invoke sms server

Run (or 'play') the workflow suite:

CDP>  play -r /cray<cray.def		# load the workflow suite into the workflow manager
CDP>  begin /cray<cray.def          # start running the suite
CDP>  ctrl-D                        # CTRL-D to quit the CDP workflow manager

To cancel a running application:
```
CDP> cancel -y -f /cray
```
before entering the play command.
Start the xcdp viewer, a graphical display of the running tasks:
```
xcdp &        # run the viewer in the background
```
In the viewer, xcdp window, expand the branch of jobs on the host by middle mouse click on the symbol to right. Get further information on the job by right mouse button.

Color coding explanation for xcdp:

grey/light blue: queuing job

green: job is running

yellow: job has successfully completed

red: job was aborted with errors

more information on errors can be found in either getmars_scm.* or getmars_scm.* in folder LOGDIR on cca (cray) depending on which process crashes

before restarting job check via

--> qstat -u your_user_name

if job is still running, if yes, kill by $ qdel process_id

violet: right click and select ”Restart”

all scripts in tree below will also be restarted and exectuables again copied to cca (changes included on ecgate will be activated)

however: no restart of cray.def file, need to login to cdp to change date

After all your boxes turns to yellow your task is completed and you should find your results in the directory specified in $SCMDATA

Download data (more information https://software.ecmwf.int/wiki/display/UDOC/File+Transfer)

e.g. http://ecaccess.ecmwf.int

Browsing → ECscratch files → data → click on symbol to left (page with red arrow) to download

Adapting extraction to other operational systems than current one (from 8 March 2016)

change scripts:

sms/include_cray/getini1c_setup_cycle_validation_SCM_FC_cray

sms/include_cray/getini1c_setup_cycle_validation_SCM_FC_cray.dataonly

in the way to modify there (for period June 2013 – March 2016)

CYCLE=40r3
dataid2="t1279_2010"
CLIMVERSION=climate.v011
GTYPE=l_2
and substitute there:
$TMPDIR/run_parallel /home/rd/nag/bin/getini1c_TCo1279
by older script (as used for ERA-I):
$TMPDIR/run_parallel /home/rd/nag/ bin/getini1c

Extracting operational forecasts or analyses further back in time needs different adaptions to the two include_cray/getini1c_setup_cycle* scripts.

Horizontal resolution updates:

21 Nov 2000: T511
1 Feb 2006: T799
26 Jan 2010: T1279
8 March 2916: T2047

Vertical resolution updates:
21 Nov 2000: L60
1 Feb 2006: L91
June 2013: L137

If changing horizontal resolution:
Also change ZDELTA with coarser grid so that point is found in circle around desired location.
Choose large enough but not to large so that point further away is detected.

TSTEP is not used in getini1c script, not as important to set.

If changing vertical resolution:
In addition to LEVELS change option “-l 137” in getini1c_mars_run_validation_SCM_FC call in include_cray/getini1c_setup_cycle_validation_SCM_FC_cray.dataonly according to new number of levels (default 137)

Extraction from ERA-I:
use scripts in directory include_cray/ERA-I instead and also ERAI-cray.def
Data availability:
00: forecast step 0, 3, 6, 9, 12
06: analysis step 0
12: forecast step 0, 3, 6, 9, 12
18: analysis step 0

Acknowledgements

Filip Váña, Nils Wedi, Paul Dando, Kerstin Hartung and Glenn Carver

On this page...

Space shortcuts

Page tree

Initial setup

Access to 'ifs' group

ssh keys

Install workflow suite

Set up the job to retrieve forcing

Run the job to retrieve forcing

Adapting extraction to other operational systems than current one (from 8 March 2016)

Acknowledgements

Space shortcuts

Page tree

How to extract Single Column Model forcing data from ECMWF analysis

Initial setup

Access to 'ifs' group

ssh keys

Install workflow suite

Set up the job to retrieve forcing

Run the job to retrieve forcing

Adapting extraction to other operational systems than current one (from 8 March 2016)

Acknowledgements