Versions Compared

Old Version 30

changes.mady.by.user Unknown User (nagc)

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (nagc)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Section


Column


 

Panel
borderColorblue
bgColor#e6ffff
borderWidth1
borderStylesolid

This tutorial explains how to create a new experiment from an existing one by copying the initial files and setting up the experiment directory.

ECMWF HPCF

ECMWF operates a Cray based High Performance Computing Facility (HPCF). It consists of two identical Cray XC40 clusters with their own storage, but with equal access to the high performance working storage of the other cluster. This provides the benefit of having one very large system but the dual clusters add significantly to the resiliency of the system.

For more details about the ECMWF HPCF, please see: http://www.ecmwf.int/en/computing/our-facilities/supercomputer

All the OpenIFS experiments will make use of the ECMWF HPCF. During the workshop you will login to the 'front-end' machines and submit 'batch job's to the HPCF.

The scripts described below are provided to make preparing and submitting ensemble batch jobs easier. Useful commands are listed at the end of this tutorial.


Column
width40%


Panel
borderStyledotted
titleOn this page...

Table of Contents



...

Code Block
languagebash
titleTo copy multiple dates. Example 1: first 5 days
cd inidir/ob01
cp -rL /perm/rd/openifs/oifs_workshop_2017/expts-inidata/ob00/2015110[1-5]* .     # this will take some time

Please only copy the initial dates you intend to use. Each date uses 0.5 Gbyte of file storage.

...

...

cd inidir/ob01
cp -rL /perm/rd/openifs/oifs_workshop_2017/expts-inidata/ob00/2015110[1-3]* .      # this will take some time.

Change experiment id

OpenIFS forecasts are identified by an 'experiment id', a four letter string.

...

Code Block
languagebash
troifs0@cca-login3:~> createEX -d 2015110100 -e ob01 -i scratch/inidir -m 10 -i scratch/inidir -m 10

We tell this command it can find our new initial files in scratch/inidir. By default, this command will create the experiment directory also in scratch. Be careful not to confuse the initial data directory with the experiment directory. These should be kept separate.

Info

Note, one ensemble member '00' is always created, so the -m argument lists the total number of members and defaults to 1. Note directories Directories are numbered starting from zero.

Only use 10 members in total otherwise you will not be able to compare with the climatological SST experiment which uses 10 members.

...

The only namelist variable that needs changing in these experiments is NENSFNB, which the run_all_ens command does for you.

Note

The stochastic backscatter scheme LSTOPH_SPBS should be disabled for OpenIFS 43r3 and beyond as it's use has been deprecated since 40r1. It's impact is very small, less so on the newer cubic grids, and is not recommended.


Create and submit Cray batch job

...

The monthly mean files can be transferred to ICTP with the sftp command, to be used on the classroom PC with metview.

Useful Unix commands

 


Code Block
titleSubmit a job to the Cray..
qsub job

...

Code Block
Quota for $HOME and $PERM:
Disk quotas for user troifs0 (uid 16144): 
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
cnasa1:/vol/home
                   226M    480M    500M             132   20000   22000        
Disk quotas for user troifs0 (uid 16144): 
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
cnasa2:/vol/perm
                      0  26624M  27648M               1    200k    210k        

Quota for $SCRATCH ($TEMP) including $SCRATCHDIR ($TMPDIR):
Disk quotas for user troifs0 (uid 16144):
     Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
  /lus/snx11062 767099824  32212254720 32212254720       -  116570  5000000 5000000       -
Disk quotas for group ectrain (gid 1400):
     Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
  /lus/snx11062 767105640       0       0       -  118078       0       0       -

...


 Excerpt Include
Credits
Credits
nopaneltrue