GRIB files

The instructions in this article show how to convert GRIB to netCDF for use in other analysis and visualisation software.

GRIB files are the standard format used at ECMWF. Files retrieved from the MARS archive, or from the public data server for reanalysis products such as ERA-Interim are all GRIB format.GRIB is a WMO standard.

The OpenIFS/IFS models output fields in GRIB format. These files are a mix of GRIB 1 & GRIB 2 format messages, the multi-level fields are encoded as GRIB-2, whereas surface fields are GRIB-1. GRIB files can also contain multiple vertical coordinates: pressure levels, model levels, sub-surface levels etc. This can cause a problem with some 3rd party tools as the same GRIB variable code is used for each axis. The instructions below show how to work around this.

Considering the output from OpenIFS, the ICMGG files contain all the gridpoint fields. The ICMSH files are the spherical harmonics of the wind fields, pressure and temperature. See controlling the OpenIFS output for more details of output fields and options.

Grib-api tools

Please note that grib-api is being phased out in preference for ecCodes. All the grib_* commands shown below are still available with ecCodes.

grib_to_netcdf

This command will convert one or more GRIB files to netCDF and is available with the ECMWF ecCodes/grib-api software (versions above 1.11.0 are recommended. It is not available in versions less than 1.10.0).

For more details please see: grib_to_netcdf description.

grib_to_netcdf only works correctly when a single level coordinate is present in the GRIB file. Often the model output files have fields on multiple level types (ie. hybrid model levels and pressure levels).

grib_copy ICMGGftkm+001440 ICMGG_[typeOfLevel].grb

In this example, the GRIB model output file ICMGGftkm+001440 contains a number of different model level types. The special square bracket "[ ]" syntax is recognised by grib_copy (and other grib commands such as grib_filter) and can contain any valid GRIB key.

This example will copy the original file, separating the level types into their own file: ICMGG_hybrid.grb, ICMGG_isobaricInhPA.grb, ICMGG_surface ...and so on.

grib_to_netcdf can then be used on the individual files:

grib_to_netcdf ICMGG_hybrid.grb -o ICMGG_hybrid.nc

To convert from vorticity and divergence to wind u and v, please see CDO instructions below.

Note that grib_to_netcdf does not perform any regridding. If the fields use a reduced Gaussian latitude grid, they will not be converted to a regular grid. Use CDO instead to do this.

The MetView application is available for analysis and visualisation of OpenIFS output.
Please see 'Using Metview with OpenIFS' for more details.

NCAR command language (NCL)

NCL provides a tool to convert both GRIB1 & GRIB2 to netCDF called ncl_convert2nc.

NCL example scripts to convert to netCDF are also available.

CDO: Climate Data Operators

These instructions assume the use of a recent version of Climate Data Operators (CDO) (available from Max-Planck-Institut, Germany).

CDO supports GRIB2 but needs the GRIB_API library from ECMWF in order to work correctly with OpenIFS model output.

cdo --version
Climate Data Operators version 1.7.2 (http://mpimet.mpg.de/cdo)
...........
     CDI library version : 1.7.2 of Aug 10 2016 09:22:26
 CGRIBEX library version : 1.7.5 of Jun  3 2016 14:44:00
GRIB_API library version : 1.15.0
  NetCDF library version : 4.4.1 of Aug  3 2016 11:10:49 $
.........

If you find the CDO commands below do not work, you can either build CDO yourself, making sure a recent version of grib_api or ecCodes is used, or you can use the workaround below.

Interpolate from reduced to regular grid

CDO can be used to interpolate from the reduced Gaussian grid to a regular Gaussian grid (IFS/OpenIFS resolutions T21 & T42 use a regular Gaussian grid)

 cdo -R copy <input grib> <output grib>

If the -R option does not work, use 'cdo setgridtype,regular' instead.

Steps to convert GRIB to netCDF

Split z axis

Before converting to netCDF, separate the different vertical axes using the generic command: cdo splitzaxis <input file> <output file pattern>.

This will ensure the following steps work correctly.

cdo splitzaxis ICMSHg4a4+000000.grb ICMSHg4a4+000000_split
ls ICMSH*split*
ICMSHg4a4+000000_split01.grb
ICMSHg4a4+000000_split02.grb
ICMSHg4a4+000000_split03.grb
ICMSHg4a4+000000_split04.grb

In this example, the different files might contain '01' fields on pressure levels, '02' fields on model levels, '03' hybrid levels, '04' surface and so on. The number of files created depends on the number of different types of levels. Use the command 'grib_ls' to inspect the contents of each split file.

Note that the filename 'ICMSH' in this example, indicates it contains spectral and not gridpoint fields.

Another way to split the file would be using the grib_copy command from the grib-api (or ecCodes) library:

grib_copy ICMSHg4a4+000000.grb ICMSHg4a4+000000_[typeOfLevel].grb

In this case 'typeOfLevel' is a GRIB key. The square brackets is a special syntax to the  grib_copy command. This approach works with any GRIB key.

Convert spectral to gridpoint

CDO supports two options for converting spectral to gridpoint data:

Both produce a regular Gaussian grid. For more details please see the description of these operators on the CDO homepage.

Convert the resulting output files to netcdf.

cdo -f nc copy <input grib> <output netcdf>

If you prefer wind components u & v instead of vorticity and divergence, use the cdo command:

cdo dv2uvl <input file> <output file>

To interpolate to pressure levels instead of model levels, use this cdo command for example:

cdo ml2pl,92500,85000,50000,20000 <input file> <output file>

Viewing contents of GRIB files

There are various commands for inspecting the contents of a GRIB file. The GRIB_API installation as part of OpenIFS has useful commands like grib_ls and grib_dump.

grib_ls ICMSHfrq2+000000.grb

Here is another example using the cdo command:

$ cdo sinfo ICMSHfrq2+000000.grb 
   File format: GRIB
    -1 : Institut Source   Param       Time Typ  Grid Size Num  Levels Num
     1 : ECMWF    unknown  11.3        var  P16     65792   1      60   1
     2 : ECMWF    unknown  39.3        var  P16     65792   1      60   1
     3 : ECMWF    unknown  43.3        var  P16     65792   1      60   1
     4 : ECMWF    unknown  44.3        var  P16     65792   1      60   1
     5 : ECMWF    unknown  152.128     var  P16     65792   1       1   2
     6 : ECMWF    unknown  6.3         var  P16     65792   1       1   2
   Horizontal grids :
     1 : spectral     > size      : dim = 65792  truncation = 255  spc = 32896
                                    complexPacking = 1
   Vertical grids :
     1 : hybrid            level : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 
                                   20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 
                                   36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 
                                   52 53 54 55 56 57 58 59 60 
     2 : hybrid            level : 1 
   Time axis :  1 step
     RefTime =  1999-12-24 12:00:00  Units = hours  Calendar = PROLEPTIC
  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss
  1999-12-24 12:00:00
cdo sinfo: Processed 6 variables over 1 timestep. ( 0.04s )

Examples

Example 1 - Converting a single file

In this example a single GRIB file with model level data, on a regular Gaussian grid is converted to netCDF with vorticity & divergence replaced by u & v and model level data interpolated to pressure levels.

cdo -f nc copy file.grb file.nc
cdo dv2uvl file.nc file_uv.nc
cdo sp2gpl file_uv.nc file_uv_gg.nc
cdo ml2pl,92500,85000,50000,20000 file_uv_gg.nc file_uv_gg_pl.nc

Example 2 - Converting series of files

The following script shows how to loop over several output files using the steps above to convert to netCDF. Note the spectral parameter files are also converted but the extra command using the sp2gpl operator is first used to convert the variables from spectral to gridpoint.

In this example, we use the pipelining feature of CDO to do both the 'copy' and 'sp2gpl' steps in one command, keeping the entire operations in memory.

for i in 000 024 048 072
do

cdo -R copy ICMGGfpp4+000${i}.grb ICMGGfpp4+000${i}_R.grb
cdo splitzaxis ICMGGfpp4+000${i}_R.grb ICMGGfpp4+000${i}
cdo -f nc copy ICMGGfpp4+000${i}01.grb ICMGGfpp4+000${i}_PLGG.nc
cdo -f nc copy ICMGGfpp4+000${i}02.grb ICMGGfpp4+000${i}_MLGG.nc

cdo splitzaxis ICMSHfpp4+000${i}.grb ICMSHfpp4+000${i}
cdo -f nc copy -sp2gpl ICMSHfpp4+000${i}01.grb ICMSHfpp4+000${i}01_GP_PL.grb
cdo -f nc copy -sp2gpl ICMSHfpp4+000${i}03.grb ICMSHfpp4+000${i}03_GP_ML.grb
cdo -f nc copy -sp2gpl ICMSHfpp4+000${i}04.grb ICMSHfpp4+000${i}04_GP_surf.grb

done

 

Possible problems and solutions

 

Workaround if CDO does not understand GRIB2 messages

If you are using an old version of CDO, or one that does not use the grib-api or ecCodes libraries, Use the grib_set command (from the grib_api installation) to change the GRIB edition number for all messages.

grib_set -s editionNumber=1 <input grib2 file> <output grib1 file>

Note this will only work if you are using less than 128 model levels as it does not actually convert the message to GRIB1 if there are any GRIB2 specific elements in the messages.

 

Parameter names are lost

The -t option tells CDO to use the predefined ECMWF parameter tables (see CDO documentation for more details). Without this option parameter names are not preserved. If the - t option doesn't work this is usually because CDO is using an old version of the ECMWF GRIB_API software.

 

Acknowledgements

Thanks to Paul Dando of User Support for help with the contents of this page.

 

 

<script type="text/javascript" src="https://software.ecmwf.int/issues/s/en_UKet2vtj/787/12/1.2.5/_/download/batch/com.atlassian.jira.collector.plugin.jira-issue-collector-plugin:issuecollector/com.atlassian.jira.collector.plugin.jira-issue-collector-plugin:issuecollector.js?collectorId=5fd84ec6"></script>