Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Section


Column

Introduction

The ECMWF ecCodes software library provides a set of functions/subroutines and command line tools for encoding and decoding working with WMO FM-92 GRIB edition 1 and edition 2 messages. The OpenIFS models use GRIB for their input and output files.

The ecCodes software library must be built and installed before compiling any OpenIFS model that needs it.

To download and install the software please visit the ecCodes website.

If you have a version of ecCodes or the older (no longer supported) ECMWF grib_api library on your system already that would most likely work, though check the minimum grib-api version required for each model.  It is recommended that you build and install ecCodes as described below to ensure the correct configuration is used. We also recommend compiling your own to ensure the same compiler and compiler version is used for the model, as otherwise, error messages about conflicting module versions when compiling can arise.

Info

The older ECMWF grib_api library can be used with OpenIFS, but as it's no longer developed by ECMWF, ecCodes is recommended. Versions of OpenIFS before 40r1v2 may need a small code change to work with ecCodes.

Before you start

Decide where you will put the ecCodes software.

For a single user, ecCodes might be installed in your home directory. For a multi-user environment, say a cluster or High Performance Computing Facility (HPCF), installing ecCodes centrally once only is probably best and , then point your build configuration to this location.

In the walk-through example below, a single-user download and install is assumed.

Info

OpenIFS does not need the 'JPEG' ir 'jasperPNG' library libraries in ecCodes . To as the model does not use any image based data. They can be disabled to avoid linking against the jasper library it can be disabled them (see below).



Column
width50px

 


Column
width280px


Panel
bgColorwhite
titleBGColorlightlightgrey
titleOn this page

Table of Contents
maxLevel2
printablefalse



...

Code Block
languagebash
titleCreate directory and unpack source
mkdir -p ecmwf/src
cd ecmwf/src
tar zxf eccodes-2.9.0-Source.tar.gz

...

Code Block
languagebash
titleSteps to configure grib-api with CMake assuming gnu compilers
mkdir bld
mkdir eccodes
cd bld
cmake ../eccodes-2.9.0-Source   \
      -DCMAKE_C_COMPILER=gcc      \
      -DCMAKE_Fortran_COMPILER=gfortran \
      -DCMAKE_INSTALL_PREFIX="..$HOME/ecmwf/eccodes"   \
      -DENABLE_NETCDF=ON              \
      -DENABLE_JPG=OFF                \
      -DENABLE_PNG=OFF                \
      -DENABLE_PYTHON=ON              \
      -DENABLE_FORTRAN=ON             \
      -DBUILD_SHARED_LIBS=BOTH

Explanation of lines. Note that all options are prefixed by '-D':

CMAKE_C_COMPILER=gcc
CMAKE_Fortran_COMPILER=gfortran

These lines set

This sets the choice of compiler. CMake is often able to determine the available compilers for itself.
However, in cases where multiple compilers are available, or where compiler wrappers are used (as on HPC systems), it's preferable to set these explicitly.

Note the CMake variables are preferred rather than set environment variables CC and FC.

CMAKE_INSTALL_PREFIX="
..
$HOME/ecmwf/eccodes"

This specifies the location where the 'make install' command will place the eccodes installation.

ENABLE_NETCDF=ONEnabling this option ensures the grib_to_netcdf command can be used. Note: a netCDF library must be available on the system.
ENABLE_JPG=OFF
ENABLE_PNG=OFF

As OpenIFS models do not deal with any image based GRIB data these options can be disabled, unless you have other GRIB data containing images that you need to work with.

Setting this removes the need for the Jasper or OpenJPG libraries.

ENABLE_PYTHON=ONSome of the utilities that come with the OpenIFS model make use of the python interface to ecCodes e.g. the tools to create and manipulate the model data.
For this option to work, ensure that a python installation is available.
ENABLE_FORTRAN=ONThis enables the Fortran interface to ecCodes and should always be on as OpenIFS uses this.
BUILD_SHARED_LIBS=BOTHThe ecCodes library can be built with both 'shared' and 'static' libraries for linking, by default only shared libraries are built. Shared libraries may be needed for the python interface depending on your installation. Some compilers will link to the shared version by default if both shared and static are found (e.g. the GNU compiler). Disabling shared libraries from being generated is not recommended as it may prevents the ecCodes command line tools from working. Note some IBM users may experience problems if shared libraries are built.

Available options

To get additional help on the available options and build configuration run the command:

...

Info
titleParallel make

If your computer has multiple cores as many, use the -j flag to make to build ecCodes faster. e.g.

Code Block
make -j 2 | tee make.out


...

At the end of this step, in the directory $HOME/ecmwf/ that we use in this example, you should now have a directory called eccodes (or some other compiler suffix) which contains the following:

Code Block
% ls eccodes
bin  include  lib  share
% ls eccodes/lib
libgrib_apilibeccodes.a  libgrib_api.la  libgrib_api_f77libeccodes_f90.a  libgrib_api_f77.lapkgconfig  libgrib_api_f90.a  libgrib_api_f90.la  pkgconfig

After this stage, next steps are to download and install FCM and then download the OpenIFS code and accompanying files.

python2.7

Possible problemsPossible problems

Info
titleFor help contact...

 OpenIFS support: openifs-support@ecmwf.int.

...

1. If problems occur, first try using the option "--disable-shared" (configure) or "-DBUILD_SHARED_LIBS=OFF" ) to only build statically linked libraries.

...

LIBRARY_PATH is only used at compile/link time. It may also be necessary to add this directory to the LD_LIBRARY_PATH environment variable which is searched for libraries at runtime.

Intel compiler

We currently recommend not using a higher optimization level than -O1 with the Intel compiler. Failures have been seen with grib_api and the Intel compiler when compiled with -O2.

Some versions of the Intel compiler (v14.0.x) can produce an error "unknown option -soname" when using 'configure. If this occurs try using "--disable-shared" or try the cmake command instead.

IBM compiler

For the IBM xlc compiler we recommend disabling the creation of 'shared libraries' which is known to cause problems with some versions of grib_api.

In this example, note the use of the '_r' form of the IBM compiler to ensure grib_api is compiled 'thread-safe'.

Building on High Performance Computer Systems

Some HPC batch systems have a different hardware architecture for their login (or frontend node) to the batch node, but the frontend compilation system is targeted at the batch nodes. This is known as cross-compilation. If this is the case you may see failures in the 'make check' stage because the checks, although compiled for the backend batch nodes, are being run on the frontend nodes and therefore may not work correctly. If this is the case on your system, we recommend using a batch job to do the 'configure; make; make check; make install' steps.

runtim

Building on High Performance Computer Systems

Some HPC batch systems have a different hardware architecture for their login (or frontend node) to the batch node, but the frontend compilation system is targeted at the batch nodes. This is known as cross-compilation. If this is the case you may see failures in the 'make check' stage because the checks, although compiled for the backend batch nodes, are being run on the frontend nodes and therefore may not work correctly. If this is the case on your system, we recommend using a batch job to do the 'cmake; make; make check; make install' stepsIn some cases, the batch system cannot be used for compilation at all. In this case, you have to compile on the frontend but without extra flags 'configure' will assume the build is for the frontend. You can make use of the --host option to ensure the build is correct for the architecture of the batch system. Again though, the tests will fail, a small serial batch job is recommended to make sure grib_api is installed correctly.

Note if you plan on using the grib_api eccodes software in your own software that runs on the frontend nodes you will may need to compile and install grib_api eccodes twice; one for the batch system and again for the frontend system.

If you have any questions installing grib_api eccodes in this type of environment, please contact: openifs-support@ecmwf.int for assistance.

...

To verify that the installation was successful, first ensure that the grib_api eccodes bin directory is added to your PATH environment variable and then run the gribcodes_info command e.g.

Code Block
% export PATH=$PATH:$HOME/ecmwf/grib_api_gcceccodes/bin
% gribcodes_info

grib_apiecCodes Version 12.148.0

Default definition files path is used: ..../grib_api_gcc/share/home/myname/ecmwf/eccodes/share/eccodes/definitions
Definition files path can be changed setting GRIBECCODES_DEFINITION_PATH environment variable

Default SAMPLES path is used: ..../grib_api_gcc/share/home/myname/ecmwf/eccodes/share/eccodes/samples
SAMPLES path can be changed setting GRIBECCODES_SAMPLES_PATH environment variable

If the installation was successful, gribcodes_info will report the installation directories. Any problems, please contact openifs-support@ecmwf.int.

...