This page contains a guide for building and using OpenIFS 48r1 at model version 48R1.1 in a linux based container using Docker.

While this page focuses on the containerised installation, the details presented can be used as a recipe for the underlying dependencies required to install and run OpenIFS 48r1 on a linux system.

Contents:

Creation of a docker build directory

The first step is to make a directory from which the OpenIFS container can be created and cd into that directory, e.g.

mkdir <openifs-docker-build-dir>
cd <openifs-docker-build-dir>

The instructions presented assume that docker is installed on the system of interest. If docker is not installed please speak with local system administration to find out if installation is permitted and how to do it. The specifics of this page have been tested with docker desktop on ubuntu 22.04 and Apple Macbook M1

While this page focuses on the containerised installation, the details presented, particularly the list of libraries to install, can be used as a recipe for the underlying dependencies required to install and run OpenIFS 48r1 on a linux system.

...

Contents:

...

Creation of a docker build directory

The first step is to make a directory from which the OpenIFS container can be created and cd into that directoryOnce inside the <openifs-docker-build-dir>  download the OpenIFS release package, e.g.

mkdir <openifs-docker-build-dir>
cd <openifs-docker-build-dir>

Once inside the <openifs-docker-build-dir>  download the OpenIFS release package from https://sites.ecmwf.int/openifs/openifs-data/src/48r1/openifs-48r1.tar.gz

...

,  e.g.

Further, if the SCM is required, then also download the SCM package to <openifs-docker-build-dir>

wget# download the openifs-48r1.tar.gz model package from https://sites.ecmwf.int/openifs/openifs-data/scmsrc/48r1/scm_openifs_-48r1.1.tar.gz
tar -xzvf scm_openifs_48r1.1 to <openifs-docker-build-dir> and untar :
tar -xvzf openifs-48r1.tar.gz

Then also download the SCM package Once OpenIFS 48r1 has been extracted to your <openifs-docker-build-dir> ,  then copy the Dockerfile from the release to <openifs-docker-build-dir>  from https://sites.ecmwf.int/openifs/openifs-data/scm/48r1/scm_openifs_48r1.tar.gz . 

cp#  <openifs-docker-build-dir>/openifs-48r1.1/scripts/docker/gcc-docker-48r1.1/Dockerfiledownload the scm_openifs_48r1.tar.gz https://sites.ecmwf.int/openifs/openifs-data/scm/48r1/scm_openifs_48r1.tar.gz to <openifs-docker-build-dir>/.

This cp step is important because the Dockerfile  and the OpenIFS release and SCM package, i.e., openifs-48r1.1 and scm_openifs, need to be in the same directory to build the container.

Build the OpenIFS docker image

...

 and untar :
tar -xzvf scm_openifs_48r1.tar.gz

At present, the SCM package is required for the build of the container.

Once OpenIFS 48r1 and the SCM package have been extracted to your <openifs-docker-build-dir> ,  then copy the Dockerfile from the release to <openifs-docker-build-dir> 

dockercp build <openifs--tag "docker-build-dir>/openifs-48r1/scripts/docker/gcc-docker-48r1.1" . 

Note the trailing '.' to build in the current dir, which is required.

...

/Dockerfile <openifs-docker-build-dir>/.

This cp step is important because the Dockerfile  and the OpenIFS release and SCM package, i.e., openifs-48r1 and scm_openifs, need to be in the same directory to build the container.

Build the OpenIFS docker image

Before starting to build the container, it is necessary to set the OIFS_EXPT environment variable in oifs-config.edit_me.sh, i.e. 

open <openifs-docker-build-dir>/openifs-48r1/oifs-config.edit_me.sh  with a text editor, then change the following line

export OIFS_EXPT=${PERM}

to 

export OIFS_EXPT=/home/openifs/openifs-expt

The above change will ensure that the defined experiment directory is consistent to the experiment directory set-up during the build of the container. 

The following command builds the docker image from the <openifs-docker-build-dir> , with the docker command executed in the same directory as the Dockerfile .

docker build --tag "openifs-48r1" . 

Note the trailing '.' to build in the current dir, which is required.

This command runs the build process for the OpenIFS image using gcc:11.2.0-bullseye as the base image, which is a debian Linux distribution with gcc 11.2 (at the time of writing, similar to the gcc on the ATOS). After downloading the image, the Dockerfile installs the following software

    apt install -y git && \
    apt install -y cmake && \
    apt install -y python3 python3-ruamel.yaml python3-yaml python3-venv && \
    apt install -y libomp-dev && \
    apt install -y     apt install -y git && \
    apt install -y cmake && \
    apt install -y python3 python3-ruamel.yaml python3-yaml python3-venv && \
    apt install -y libomp-dev && \
    apt install -y libboost-dev libboost-date-time-dev libboost-filesystem-dev libboost-serialization-dev libboost-program-options-dev&& \
    apt install -y netcdf-bin libnetcdf-dev libnetcdff-dev && \
    apt install -y libatlas-base-dev && \
    apt install -y liblapack-dev && \
    apt install -y libeigen3-dev && \
    apt install -y bison && \ 
    apt install -y flex && \
    apt install -y vim && \
    apt install -y wget bc 
## Install python3-pip and the dependencies to run 
## plotscm.py with SCM output
    apt install -y python3-pip && \
    pip3 install numpy && \
    pip3 install netcdf4 && \
    pip3 install matplotlib && \
    pip3 install pandas && \
    pip3 install xarray
## Download and build openmpi
https://download.open-mpi.org/release/open-mpi/v4.1/openmpi-4.1.5.tar.gz 

Once these repository packages and the python librarie are installed, open-mpi is downloaded and built. Then the openifs user is created and the openifs-48r1.1  directory is copied to the image.

...

docker build --no-cache --tag "openifs-48r1.1" .

...

docker images

REPOSITORY          TAG       IMAGE ID       CREATED       SIZE
openifs-48r1.1      latest    f72ea92f010f   5 hours ago   2.15GB

...

docker run -it 'openifs-48r1.1'

This command will open an interactive session in the new container, in which the entry directory is the openifs-48r1 .1 directory, e.g.

docker run -it 'openifs-48r1.1'

OpenIFS environment variables are:
------------------------------------------------------
OIFS_ARCH=./arch/ecmwf/hpc2020
OIFS_CYCLE=48r1
OIFS_DATA_DIR=/perm/openifs/oifs_data/48r1/48r1
OIFS_EXEC=/home/openifs/openifs-48r1.1/build/bin/ifsMASTER.DP
OIFS_EXPT=/home/openifs/openifs-expt
OIFS_HOME=/home/openifs/openifs-48r1.1
OIFS_LOGFILE=/home/openifs/openifs-48r1.1/oifs_test_log.txt
OIFS_RUN_SCRIPT=/home/openifs/openifs-48r1.1/scripts/exp_3d
OIFS_TEST=/home/openifs/openifs-48r1.1/scripts/build_test


SCM environment variables are:
------------------------------------------------------
SCM_EXEC=/home/openifs/openifs-48r1.1/build/bin/MASTER_scm.DP
SCM_LOGFILE/MASTER_scm.DP
SCM_LOGFILE=/home/openifs/openifs-expt/scm_openifs/48r1/scm-projects/ref48r1/scm_run_log.txt
SCM_PROJDIR=/home/openifs/openifs-expt/scm_openifs/48r1/scm-projects
SCM_RUNDIR=/home/openifs/openifs-expt/scm_openifs/48r1/scm-projects/ref48r1/scm_run_log.txt
SCM_PROJDIRTEST=/home/openifs/openifs-expt/scm_openifs/48r148r1/scripts/scm-projects
SCM_RUNDIRVERSIONDIR=/home/openifs/openifs-expt/scm_openifs/48r1/scm-projects/ref48r1
SCM_TEST=/home/openifs/openifs-48r1.1/scripts/scm
SCM_VERSIONDIR=/home/openifs/openifs-expt/scm_openifs/48r1



openifs@d1bd89ccc47f:~/openifs-48r1$ ls
CHANGES  COPYING  NOTICE  arch        ifs-source  oifs-config.edit_me.sh  scripts
CITE     LICENSE  README  bundle.yml  ifs-test    openifs-bundle

Once inside the container, the openifs-test.sh  script can be run, e.g., 

openifs@d1bd89ccc47f:~/openifs-48r1.1$ ls
CHANGES  COPYING  NOTICE  arch        ifs-source  oifs-config.edit_me.sh  scripts
CITE     LICENSE  README  bundle.yml  ifs-test    openifs-bundle

Once inside the container, the openifs-test.sh  script can be run, e.g., 

openifs@d1bd89ccc47f:~/openifs-48r1.1$ $OIFS_TEST/openifs-test.sh -cbt --arch=""

where :

-c   creates source directory in $OIFS_HOME, which is used to bring all the sources, e.g. ifs-source, ifs-test and any packages in the bundle (ecbuild, eccodes etc.), together in preparation for the build

-b  builds the source. This step creates the directory build in $OIFS_HOME, which is used to build and store

• the OpenIFS double and single precision master executables (ifsMASTER.DP and ifsMASTER.SP, respectively) , which are used to run 3-D OpenIFS. The executables are located in  $OIFS_HOME/build/bin.
  • The location and name of the executable for OpenIFS is defined in the platform configuration file (oifs-config.edit_me.sh) as $OIFS_EXEC.

• the double and single precision Single Column Model (SCM) executables (MASTER_scm.DP and MASTER_scm.SP, respectively), which are used to run the SCM derived from OpenIFS. The executables are located in  $OIFS_HOME/build/bin.
  • The location and name of the executable for the SCM is defined in the platform configuration file (oifs-config.edit_me.sh) as $SCM_EXEC.

-t will run the ifs-test t21 tests, which comprise of

48r1$ $OIFS_TEST/openifs-test.sh -cbt -j 2 --arch=""

where :

-c   creates source directory in $OIFS_HOME, which is used to bring all the sources, e.g. ifs-source, ifs-test and any packages in the bundle (ecbuild, eccodes etc.), together in preparation for the build

-b  builds the source. This step creates the directory build in $OIFS_HOME, which is used to build and store

• the OpenIFS double and single precision master executables (ifsMASTER.DP and ifsMASTER.SP, respectively) , which are used to run 3-D OpenIFS. The executables are located in  $OIFS_HOME/build/bin.
  • The location and name of the executable for OpenIFS is defined in the platform configuration file (oifs-config.edit_me.sh) as $OIFS_EXEC.

• the double and single precision Single Column Model (SCM) executables (MASTER_scm.DP and MASTER_scm.SP, respectively), which are used to run the SCM derived from OpenIFS. The executables are located in  $OIFS_HOME/build/bin.
  • The location and name of the executable for the SCM is defined in the platform configuration file (oifs-config.edit_me.sh) as $SCM_EXEC.

-t will run the ifs-test t21 tests, which comprise of

• 21 3-D OpenIFS forecast-only tests with and without chemistry
• 1 SCM test (based on TWP-ICE)

-j is an optional argument that allows the user to define the number of threads used to create and build openifs, with the above example using 2 threads. It should be noted that 

• If -j is not supplied as a command line argument, as in the above example, the number of threads will default to 8 threads, which can be problematic for older and lower spec systems. For example, it will not work with a 4 core intel m3. The above example uses 2 threads, which should work on most systems.
• The number of threads can be increased to speed up the build step, but how much the number of threads can be increased to will depend on the system spec. Hence, please speak the local system administration for guidance about appropriate settings. 
• 21 3-D OpenIFS forecast-only tests with and without chemistry
1 SCM test (based on TWP-ICE)

--arch="" means no arch file is required because all the libraries and paths have been set-up locally in the docker container. 

...

1. List running containers

   Code Block
   language bash theme Midnight
   $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d1bd89ccc47f openifs-48r1.1 "bash" 15 hours ago Up 5 seconds beautiful_pasteur

    If no container is running on your system, then only CAPITAL headings are returned with docker ps 

2. List all containers (running and exited)

   Code Block
   language bash theme Midnight
   $ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d1bd89ccc47f openifs-48r1.1 "bash" 15 hours ago Exited (0) 4 seconds ago beautiful_pasteur

   Notice that the STATUS  is Exited, rather than Up, as in (1)

...

$ docker start -i <Container ID> 
# e.g. 
$ docker ps -a

CONTAINER ID   IMAGE               COMMAND       CREATED        STATUS                     PORTS     NAMES
d1bd89ccc47f   openifs-48r1.1      "bash"        15 hours ago   Exited (0) 4 seconds ago             beautiful_pasteur

$ docker start -i  d1bd89ccc47f

...