...
Section | ||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Compiling with FCM
The Flexible Configuration Management (FCM) software, from the UK Meteorological Office, is used to compile the model. FCM offers a number of advantages: configuration files are easy to understand, model object files are compiled 'out-of-source', source code changes can be 'out-of-source', fortran interface blocks can be generated automatically. FCM replaces the traditionally 'make' or 'gmake' command and is specifically written for compiling large fortran or mixed fortran/C codes like OpenIFS.
A quick introduction to FCM for OpenIFS users is available. A more detailed description of the 'fcm make' command is also available.
Build types
The supplied configuration files support 3 different build types:
...
Full debugging ('nansC') - this configuration is intended for debugging only. As well as setting the lowest optimization, -O0, it also enables array bound checking and initialisation of variables with special values that will trigger 'not-a-number' exceptions useful for trapping variables used before initialized. This will cause the model to run very slowly and is normally only used for short runs for tracing bugs.
Config files and variables
The FCM software uses configuration files identified by the suffix '.cfg'. They can be found in the directory oifs/make/cfg
. The master configuration file for OpenIFS is in cfg/oifs.cfg. It sets general configuration options for FCM, lets FCM know where the source code is and reads a single architecture and compiler specific configuration file. No compiler options are contained in the oifs.cfg file - these are always in the architecture specific configuration files.
...
Info |
---|
Newer versions of OpenIFS (38r1v05 and on) introduced an environment variable |
How to change build type.
Before compiling the OpenIFS model, the build configuration must first be set by several environment variables:
...
changes the build type to the 'optimized' build. This means instead the file 'cfg/gnu-opt.cfg
' will be included by cfg/oifs.cfg
instead of the file cfg/gnu-noopt.cfg.
Create your own config files
Suppose we wanted to run the model on MacOS X with the intel compiler and a optimized setting. We could set:
...
Code Block |
---|
% cd oifs/make % mkdir mycfg % ln -s cfg/oifs.cfg mycfg/oifs.cfg # the oifs.cfg must exist in the same directory as your custom .cfg file. % cp cfg/intel-opt.cfg mycfg/osx_intel_v13-opt.cfg (edit osx_intel_v13-opt.cfg) % export OIFS_COMP=osx_intel_v13 % export OIFS_BUILD=opt % fcm make -f mycfg/oifs.cfg |
Change compiler options globally
All the compiler flags and options are contained in the build specific files (e.g. intel-opt.cfg). As above, a number of environment variables can be defined to override the default options set in these files. As a general rule, any variable with a {?} can be overriden by setting an environment variable.
...
User configurable variables | Description |
---|---|
OIFS_GRIB_API_DIR | Location of the grib_api installation directory. This must be changed or overridden to the location of your grib_api install before OpenIFS can be compiled. |
OIFS_GRIB_API_INCLUDE | Location of the 'include' sub-directory in the grib_api installation. This does not normally need changing. |
OIFS_GRIB_API_LIB | Location of the 'lib' sub-directory in the grib_api installation. This does not normally need changing. Note: although grib_api builds both shared and static libraries, OpenIFS always links to the static library in case different versions of grib_api exist on the user system. |
OIFS_LAPACK_LIB | Compiler options to set the directory of the LAPACK and BLAS libraries. This must be changed or overriden as the supplied default is unlikely to be correct on most systems. e.g. OIFS_LAPACK_DIR{?} = -L/opt/apps/lapack/LP64/ -llapack -lblas |
OIFS_EXTRA_LIB | Available in case additional libraries need to be specified. This is most likely set when vendors supply special optimized libraries. One example is IBM's optimized math function library MASS. e.g. OIFS_EXTRA_LIB{?} = -L/usr/lib/mass -lmass -lmassvp6 |
OIFS_FC | Specifies the fortran compiler. The normal default is 'mpif90' but might be 'mpiifort' for the Intel compiler or more generally 'mpifc'. We recommend that mpi compiler wrappers are used whenever possible to simplify the compilation e.g. OIFS_FC = mpif90. |
OIFS_FFLAGS | Fortran compiler flags for free format source files. These flags will vary between the 'opt', 'noopt' and 'nansC' build configurations. e.g. OIFS_FFLAGS = -g -O2 -m64 -march=native -fconvert=big-endian -fopenmp |
OIFS_FFIXED | Extra fortran compiler flags, in addition to OIFS_FFLAGS, for fixed source form Fortran. There are a small number of fortran source files in OpenIFS that are still in fixed format. These options will be added to the options in OIFS_FFLAGS. |
OIFS_FCDEFS | Fortran preprocessor DEFINE flags. This is a list, separated by spaces, of the DEFINE flags required by the code in preprocessor statements like #ifdef. The user is advised not to change these |
OIFS_LFLAGS | Linker options. These are separate flags to be passed to the linker. For example, a common requirement is to generate a loadmap or increase stack memory size. e.g. OIFS_LFLAGS{?} = -bloadmap:map -bmaxstack:0x80000000 |
OIFS_CC | Specifies the C compiler. As for the Fortran compiler this would normally be a compiler wrapper supplied by the MPI system in use. e.g. mpicc for GCC or mpiicc for Intel. |
OIFS_CFLAGS | C compiler flags. As for the Fortran flags, these will vary between the 'opt', 'noopt' and 'nansC' build options. |
OIFS_CCDEFS | C preprocessor flags. As for the OIFS_FCDEFS, the user is advised not to change these. |
Examples
Set minimum required options.
The only two compilation variables that need to be changed before building OpenIFS are the location of the grib_api installation and the location of the LAPACK and BLAS libraries.Code Block title e.g. export OIFS_GRIB_API_DIR=/home/me/ecmwf/grib_api export OIFS_LAPACK_LIB="/opt/apps/lapack/current/LP64 -llapack -lblas" fcm make -f cfg/oifs.cfg
Would override the values of these variables in the FCM config files. In this example, FCM would use the gnu-noopt.cfg build as this is the default specified in the oifs.cfg file (in the oifs/make/cfg directory).Changing compiler and compiler options.
In this example, the build type is changed and the choice of underlying fortran compiler is altered along with some compiler options:Code Block export OIFS_COMP=gnu export OIFS_BUILD=opt export OIFS_GRIB_DIR=/opt/local/ecmwf/grib_api_xe6 export OIFS_LAPACK_LIB="-L/opt/apps/scal/lib -llapack -lblas" export OIFS_FC="mpif90 --fc=crayftn" export OIFS_FFLAGS="-g -O1 -m64 -convert big-endian"
In this example, the compiler has been changed although FCM will still use the gnu-opt.cfg configuration file. This might be because a user wants to temporarily try a different compiler. However, it's recommended that each compiler has its own set of .cfg files rather than alter it this way.
Another example is where an optimized build is required but the optimization level is dropped to -O1 instead of -O2.
Changing options per file
The description above covers changing the compiler options globally, that is, for all files in the compilation. There may be instances when options for one or a few files need to be changed. For instance, in debugging where array bound checking is required for a handful of changed subroutines.
Changing options per file cannot be done using environment variables. There are several ways in which it can be done:
Edit the oifs.cfg file
The oifs.cfg file is the place to put all file specific options. Some can be found in this file already e.g.
...
Info |
---|
As this is a new .cfg file for FCM it will do a full build of the code from scratch. Any subsequent builds will be incremental - only compiling changed files. |
Using the 'include' statement
FCM configuration files support an 'include' statement. The oifs.cfg file uses this to include architecture and compiler specific information. If you need to include several lines changing the options for multiple files then another option would be to add an additional 'include' statement to point to a new file containing the build.prop lines you need. For example:
...
Info |
---|
The $HERE variable is a special FCM specific variable that expands to the directory name containing the file. In this example the oifs.cfg and your debug.cfg are in the same directory. |
Inherited build
A third way to customize at the file level is through FCM's 'inherited build' mechanism. This simple but powerful facility is described in more detail in the HowTo modify and add new code. In brief this allows you to leave the current configuration and source untouched and 'use' or 'inherit' the OpenIFS configuration in a completely separate directory. This makes it easier to update the OpenIFS code for new releases and keeps any changes you make completely separate.
The reader is referred to the separate HowTo modify and add new code for examples on how to use it.
Recommendations
As described above, the build environment uses 3 types of build:
...