Versions Compared

Old Version 64

changes.mady.by.user Carsten Maass

Saved on

compared with

New Version Current

changes.mady.by.user Marcus Koehler

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Confirmed.

...


 

Section


Please note that grib-api is being phased out in preference for ecCodes. All the grib_* commands shown below are still available with ecCodes.

grib_to_netcdf

This command will convert one or more GRIB files to netCDF and is available with the ECMWF ecCodes /grib-api software (for grib-api, versions above 1.11. 0 are recommended). For more details please see: grib_to_netcdf description.

grib_to_netcdf only works correctly when a single level coordinate is present in the GRIB file. Often the model output files have fields on multiple level types (ie. hybrid model levels and pressure levels).

Column


Excerpt

This article show different approaches to convert GRIB to netCDF for use in other analysis and visualisation software.

It also describes different methods to interpolate the reduced Gaussian grid to regular grids.

What are GRIB files

GRIB files are the file format used at ECMWF.  GRIB is a WMO standard and consists of GRIB Edition 1 and Edition 2.

The OpenIFS/IFS models output in GRIB format. These files are a mix of GRIB-1 & GRIB-2 messages, the multi-level fields are encoded as GRIB-2, whereas surface fields are GRIB-1. GRIB files can also contain multiple vertical coordinates: pressure levels, model levels, sub-surface levels etc. This can cause a problem with some 3rd party tools, as the same GRIB variable code is used for each axis. The instructions below show how to work around this by splitting the file to separate fields on different vertical axes.

OpenIFS model output

The OpenIFS model outputs two types of files: those beginning with ICMSH contain fields represented as spherical harmonics, those that begin ICMGG contain gridpoint fields.  The ICMSH files are the spherical harmonics of the wind fields, pressure and temperature and require a spectral transform to convert to gridded data. Also see controlling the OpenIFS output for more details of output fields and options.

Retrieving data from MARS archive

Please note that if using the MARS archive (apps.ecmwf.int) (e.g. for reanalysis products such as ERA-Interim/ERA-5), it is possible to download the files in netCDF format as well as GRIB.

ecCodes

/grib-api

tools

Info
Code Block
titleHow to split GRIB file into separate level types
grib_copy ICMGGftkm+001440 ICMGG_[typeOfLevel].grb

In this example, the GRIB model output file ICMGGftkm+001440 contains a number of different model level types. The special square bracket "[ ]" syntax is recognised by grib_copy (and other grib commands such as grib_filter) and can contain any valid GRIB key.

This example will copy the original file, separating the level types into their own file: ICMGG_hybrid.grb, ICMGG_isobaricInhPA.grb, ICMGG_surface ...and so on.

grib_to_netcdf can then be used on the individual files:

Code Block
titleConvert GRIB file to netCDF
grib_to_netcdf ICMGG_hybrid.grb -o ICMGG_hybrid.nc

By default, grib_to_netcdf will pack the data into scaled integers with an offset to optimize space.

If you prefer data stored as floats then use:

Code Block
languagebash
titleConvert GRIB to netCDF storing values as floats rather than scaled integers...
grib_to_netcdf -D NC_FLOAT ICMGG_hybrid.grb -o ICMGG_hybrid.nc

To convert from vorticity and divergence to wind u and v, please see CDO instructions below.

Note that grib_to_netcdf does not do any regridding. If the fields use a reduced Gaussian latitude grid, they will not be converted to a regular grid. Use CDO instead to do this as described below.


Column
width350px


Panel
bgColorwhite
titleBGColorlightlightgrey
titleOn this page

Table of Contents
indent10px


Info
The MetView Metview application is available for analysis and visualisation of OpenIFS output and will both convert spectral to gridded parameters and plot reduced Gaussian grid directly.
Please see 'Using Metview with OpenIFS' for more details.


Panel
titleSee also...

ecCodes - grib tools

MARS user documentation

EMOSLIB interpolation library



...

CDO supports GRIB-2 but needs either the grib_api or ecCodes library from ECMWF to be included. This is necessary in order to work correctly with OpenIFS model output.

Code Block
languagebash
titleCheck CDO is compiled with grib-api (or ecCodes)
cdo --version
Climate Data Operators version 1.79.26 (http://mpimet.mpg.de/cdo)
...........
Libraries: HDF5/1.10.4 proj/5.2 xml2/2.9.4
Filetypes: srv CDIext libraryieg versiongrb1 : 1.7.2 of Aug 10 2016 09:22:26
 CGRIBEXgrb2 nc1 nc2 nc4 nc4c nc5 
     CDI library version : 1.79.56
 ofcgribex Junlibrary version 3 2016 14:44:00
GRIB_API: 1.9.2
 ecCodes library version : 12.1512.05
  NetCDF library version : 4.46.12 of AugFeb 14 32019 2016 1110:1013:4927 $
    hdf5 library version : 1.10.4 threadsafe
    exse library version : 1.4.1
    FILE library version : 1.8.3
.........

If you find the CDO commands below do not work, you can either build CDO yourself, making sure a recent version of grib_api or ecCodes is used, or you can use the workaround below.

...

Info

If you have files with a mix of GRIB-1 and GRIB-2, then either split the file first or compile cdo with "--disable-cgribex --with-grib_apieccodes=yes"

If variable names are lost, add the  "-t ecmwf" option.

...

Another way to split the file would be using the grib_copy command from the ecCodes (or grib_api) software as shown above:

Code Block
grib_copy ICMSHg4a4+000000.grb ICMSHg4a4+000000_[typeOfLevel].grb

...

There are various commands for inspecting the contents of a GRIB file. The GRIB_API installation as part of ecCodes software installation required for OpenIFS has useful commands like grib_ls and grib_dump.

...

Note

cdo commands can be combined into a single command for greater efficiency.

 


Using EMOSLIB to interpolate to regular grid

The ECMWF interpolation software library EMOSLIB also provides the capability to interpolate spectral data to regular gaussian grids or regular lat-lon grids, and interpolate regular gaussian grids to regular lat-lon grids.

...

The grid specified for the --regular option follows the grid naming convention for EMOSLIB here: Reduced Gaussian Grids.

Another command can be found in the 'tools' directory. This allows the interpolation function to be specified and is more flexible:

...

Possible problems and solutions

...


cdo -R option does not work with GRIB-2 fields to convert to regular grid

...

However, if this does not work for any reason, a workaround is to temporarily change the edition number to 1 of the GRIB file. Use the grib_set command (from ecCodes or grib_api) to change the GRIB edition number for all messages:

...

Code Block
cdo vct in.grb > pv.txt

...


Parameter names are lost for GRIB-1 fields
ECMWF GRIB-1 use keys which may not be recognised by CDO because they are locally defined (e.g. the shortName key) and not defined in the WMO GRIB tables that CDO uses.  This can cause parameter names to be lost or not recognised when using the CDO commands..
A workaround is to use the -t ecmwf option.  This will make CDO use ECMWF parameter table 128 for the definitions of variables but 128 is only for GRIB 1. This will work for the surface fields which are encoded by IFS as GRIB 1, but not for the multi-level fields which are encoded in GRIB 2. Variable names become 'unknown' if -t ecmwf is used with the multi-level fields. It is best to omit it completely, even for surface fields, and only use it for GRIB 1 data if you get missing variable names after CDO has converted the data A workaround is to use the -t ecmwf option.  The -t option tells CDO to use the predefined ECMWF parameter tables (see CDO documentation for more details).
...
 Warning
Do not use this option for GRIB-2 fields. It sets the GRIB table default to be specific to the GRIB-1 ECMWF tables. If problems persist, we recommend using grib_to_netcdf to convert to netCDF.
...

Acknowledgements
Thanks to Paul Dando of User Support for help with the contents of this page.
 
 


 HTML
<script type="text/javascript" src="https://softwarejira.ecmwf.int/issues/s/en_UKet2vtj/787/12/1.2.5/_/download/batch/com.atlassian.jira.collector.plugin.jira-issue-collector-plugin:issuecollector/com.atlassian.jira.collector.plugin.jira-issue-collector-plugin:issuecollector.js?collectorId=5fd84ec6"></script>


...