This page contains a guide for building and using OpenIFS 48r1 at model version 48R1.1 in a linux based container using Docker.

While this page focuses on the containerised installation, the details presented can be used as a recipe for the underlying dependencies required to install and run OpenIFS 48r1 on a linux system.

If you have access to the ECMWF HPCF (ATOS hpc2020) and are planning to run OpenIFS on that system please refer to OpenIFS 48r1 User Guide.

Contents:

Creation of a docker build directory

The first step is to make a directory from which the OpenIFS container can be created and cd into that directory, e.g.

mkdir <openifs-docker-build-dir>
cd <openifs-docker-build-dir>

Once inside the <openifs-docker-build-dir> clone or branch from the git repository with the OpenIFS release package, e.g.

git clone --single-branch --branch release ssh://git@git.ecmwf.int/oifs/openifs-48r1.1.git

Please note that currently access to this repository is restricted to a limited number of users only.
It is important that the OpenIFS package in the docker build directory is a fresh clone/branch, i.e., not built on your local system.

Once OpenIFS 48r1 has been extracted to your <openifs-docker-build-dir> , then copy the Dockerfile from the release to <openifs-docker-build-dir>

cp  <openifs-docker-build-dir>/openifs-48r1.1/scripts/docker/gcc-docker-48r1.1/Dockerfile <openifs-docker-build-dir>/.

This cp step is important because the Dockerfile and the OpenIFS release, i.e., openifs-48r1.1, need to be in the same directory to build the container (see the openifs-48r1.1/scripts/docker/gcc-docker-48r1.1/Dockerfile)

Build the OpenIFS docker image

The following command builds the docker image

docker build --tag "openifs-48r1.1" .

Note the trailing '.' to build in the current dir, which is required.

This command runs the build process for the OpenIFS image using gcc:11.2.0-bullseye as the base image, which is a debian Linux distribution with gcc 11.2 (at the time of writing, similar to the gcc on the ATOS). After downloading the image, the Dockerfile installs the following software

    apt install -y git && \
    apt install -y cmake && \
    apt install -y python3 python3-ruamel.yaml python3-yaml python3-venv && \
    apt install -y libomp-dev && \
    apt install -y libboost-dev libboost-date-time-dev libboost-filesystem-dev libboost-serialization-dev libboost-program-options-dev&& \
    apt install -y netcdf-bin libnetcdf-dev libnetcdff-dev && \
    apt install -y libatlas-base-dev && \
    apt install -y liblapack-dev && \
    apt install -y libeigen3-dev && \
    apt install -y bison && \ 
    apt install -y flex && \
    apt install -y vim && \
    apt install -y wget bc

Once these packages are installed, open-mpi is downloaded and built, the openifs user is created and the openifs-48r1.1 directory is copied to the image.

On a Macbook pro (M1) the initial build of the container, takes about 5 minutes. If you decide to change the Dockerfile following a build, you can just execute the same command and, depending on what you change, subsequent builds can be a lot quicker.

At the time of writing open-mpi is downloaded and built as part of the image creation. This is quite slow, so in the future we will investigate the use of a standard library
If a fresh build is required but an image has already been built, then execute the following command, which is the same as above but with no-cache

docker build --no-cache --tag "openifs-48r1.1" .

The tag name does not need to be provided but it is useful for identifying images.

Running OpenIFS-test using the docker image

Once a build has completed, the image can be checked by typing docker images , e.g.,

docker images

REPOSITORY          TAG       IMAGE ID       CREATED       SIZE
openifs-48r1.1      latest    f72ea92f010f   5 hours ago   2.15GB

And the container can be run using the following

docker run -it 'openifs-48r1.1'

This command will open an interactive session in the new container, in which the entry directory is the openifs-48r1.1 directory, e.g.

docker run -it 'openifs-48r1.1'

OpenIFS environment variables are:
------------------------------------------------------
OIFS_CYCLE=48r1
OIFS_DATA_DIR=/perm/openifs/oifs_data/48r1/48r1
OIFS_HOME=/home/openifs/openifs-48r1.1
OIFS_LOGFILE=/home/openifs/openifs-48r1.1/oifs_test_log.txt

openifs@d1bd89ccc47f:~/openifs-48r1.1$ ls
CHANGES  COPYING  NOTICE  arch        ifs-source  oifs-config.edit_me.sh  scripts
CITE     LICENSE  README  bundle.yml  ifs-test    openifs-bundle

In the above code block the openifs config (oifs-config.edit_me.sh) is sourced automatically so the container has the correct environment for OpenIFS

Once inside the container, the openifs-test.sh script can be run, e.g.,

openifs@d1bd89ccc47f:~/openifs-48r1.1$ ./scripts/openifs-test.sh -cbt -s docker

where :

-c will clean the directory (not necessary for a fresh image or new container)

-b will configure the bundle and build OpenIFS, i.e.,

the OpenIFS double and single precision master executables (ifsMASTER.DP and ifsMASTER.SP, respectively) , which are used to run 3-D OpenIFS. The executables are located in $OIFS_HOME/build/bin.
the double and single precision Single Column Model (SCM) executables (MASTER_scm.DP and MASTER_scm.SP, respectively), which are used to run the SCM derived from OpenIFS. The executables are located in $OIFS_HOME/build/bin.

-t will run the ifs-test

21 3-D OpenIFS forecast-only tests with and without chemistry
1 SCM test (based on TWP-ICE)

-s docker defines the system as docker, which makes the script use the correct build commands

On a Macbook pro (M1) the build of OpenIFS, takes about 10 minutes, while successful completion of ifstest takes just over 3 minutes.

Basic docker commands and functionality

Check for existing containers

From a terminal on the host system, it is possible to list running and exited containers with the following

List running containers

$ docker ps

CONTAINER ID   IMAGE            COMMAND   CREATED        STATUS         PORTS     NAMES
d1bd89ccc47f   openifs-48r1.1   "bash"    15 hours ago   Up 5 seconds             beautiful_pasteur

If no container is running on your system, then only CAPITAL headings are returned with docker ps

List all containers (running and exited)

$ docker ps -a

CONTAINER ID   IMAGE               COMMAND       CREATED        STATUS                     PORTS     NAMES
d1bd89ccc47f   openifs-48r1.1      "bash"        15 hours ago   Exited (0) 4 seconds ago             beautiful_pasteur

Notice that the STATUS is Exited, rather than Up, as in (1)

Start an existing container

If docker ps -a shows an exited container it can be restarted using the following

$ docker start -i <Container ID> 
# e.g. 
$ docker ps -a

CONTAINER ID   IMAGE               COMMAND       CREATED        STATUS                     PORTS     NAMES
d1bd89ccc47f   openifs-48r1.1      "bash"        15 hours ago   Exited (0) 4 seconds ago             beautiful_pasteur

$ docker start -i  d1bd89ccc47f

Exiting the container

The container can be exited by typing exit from the active container.
A container can be stopped from a terminal on the host system
docker stop <Container ID>
It is also possible to stop multiple containers at once by adding more than one ID.
Using docker stop will attempt a clean shutdown. It is also possible to stop the container by killing it
docker kill <Container ID>

Removing containers and images

Remove a container with the following command

docker rm <Container ID>

It is possible to remove multiple containers at once by adding more than one ID.

It is worth noting that if a container is running it cannot be removed without being forced or being stopped first

Removing a container will result in a loss of all data and any code changes etc from the container