Docker support
Docker is not supported on TEMS directly for security reasons.
You may use Singularity if you wish to run containerised workloads. It does not need root privileges to run the containers, and it supports running its own "SIF" container images as well as standard docker containers pulled from any registry such as Docker Hub. Those will get translated automatically into a SIF image before they run.
Official reference documentation
Visit the Singularity User Guide for further details
The basics
First, you will need to load the singularity module so you can start using it:
$ module load singularity
Here's a quick example running a simple command within the official latest Ubuntu Docker image, pulled straight from Docker Hub:
$ singularity exec docker://ubuntu:latest cat /etc/os-release INFO: Converting OCI blobs to SIF format INFO: Starting build... Getting image source signatures Copying blob 345e3491a907 done Copying blob 57671312ef6f done Copying blob 5e9250ddb7d0 done Copying config 7c6bc52068 done Writing manifest to image destination Storing signatures 2021/06/07 17:51:35 info unpack layer: sha256:345e3491a907bb7c6f1bdddcf4a94284b8b6ddd77eb7d93f09432b17b20f2bbe 2021/06/07 17:51:36 info unpack layer: sha256:57671312ef6fdbecf340e5fed0fb0863350cd806c92b1fdd7978adbd02afc5c3 2021/06/07 17:51:36 info unpack layer: sha256:5e9250ddb7d0fa6d13302c7c3e6a0aa40390e42424caed1e5289077ee4054709 INFO: Creating SIF file... NAME="Ubuntu" VERSION="20.04.2 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.2 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal
You can easily get the SIF image file from a docker container with:
$ singularity pull docker://ubuntu:latest INFO: Converting OCI blobs to SIF format INFO: Starting build... Getting image source signatures Copying blob 345e3491a907 done Copying blob 57671312ef6f done Copying blob 5e9250ddb7d0 done Copying config 7c6bc52068 done Writing manifest to image destination Storing signatures 2021/06/07 17:51:35 info unpack layer: sha256:345e3491a907bb7c6f1bdddcf4a94284b8b6ddd77eb7d93f09432b17b20f2bbe 2021/06/07 17:51:36 info unpack layer: sha256:57671312ef6fdbecf340e5fed0fb0863350cd806c92b1fdd7978adbd02afc5c3 2021/06/07 17:51:36 info unpack layer: sha256:5e9250ddb7d0fa6d13302c7c3e6a0aa40390e42424caed1e5289077ee4054709 INFO: Creating SIF file...
Once you have the SIF image, you can run a shell on it :
$ singularity shell ./ubuntu_latest.sif Singularity> cat /etc/os-release NAME="Ubuntu" VERSION="20.04.2 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.2 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal Singularity> exit exit
SIF images are themselves executable, and when run standalone will basically execute what it is defined in their "runscript" section of their singularity definition file, or just a shell if nothing is defined (such as when pulled from a docker registry):
$ ./ubuntu_latest.sif Singularity> exit $ ./ubuntu_latest.sif cat /etc/os-release NAME="Ubuntu" VERSION="20.04.2 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.2 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal
Environment and filesystems in the container
By default, the container inherits the environment of the host, and the usual user filesystems are also available by default: $HOME, $PERM, $HPCPERM, $SCRATCH, $SCRATCHDIR and $TMPDIR.
$ singularity shell docker://ubuntu:latest INFO: Using cached SIF image Singularity> ls -1d $HOME $PERM $HPCPERM $SCRATCH $SCRATCHDIR $TMPDIR /etc/ecmwf/ssd/ssd1/tmpdirs/21/usxa.2339821.20210608_133807.854 /home/usxa /lus/pfs1/hpcperm/usxa /lus/pfs1/scratch/usxa /lus/pfs1/scratchdir/usxa/1/at1-11.2339821.20210608_133807.854 /perm/usxa Singularity>
You may bind additional directories with the --bind option.
Building your containers
You will not be able to build your singularity images on TEMS from a definition file, since it requires sudo privileges. However, you can do so on your own computer where you do root privileges and transfer the final SIF image ready to run. Check the Singularity User Guide for more details.
MPI Support
For maximum portability, the recommended approach is the Hybrid Model. That requires installing MPI within the container, and having a compatible MPI or launcher on the host.
Performance note
Installing stock MPI implementations from official repositories will usually work, but they may not be able to fully exploit the Infiniband Fabric for inter-node communication. If you intend to run MPI workloads across multiple nodes, make sure you build the appropriate support for the underlying fabric.