Info |
---|
The ecCodes GRIB tools are documented at: There are examples of how to use them at: Other useful resources: |
Tip |
---|
If you have already followed the Introduction Tutorial on GRIB decoding with ecCodes GRIB tools: inspecting GRIB files there is no need to redo these preliminary steps. |
Work Login to ecs-login and work in your $SCRATCH:
Make a copy of the practicals directory in your $SCRATCH to Atos:
...
No Format |
---|
$> ls $SCRATCH/grib_practicals
inspect modify python |
For this tutorial, cd to the modify directory:
No Format |
---|
$> cd $SCRATCH/grib_practicals/modify |
Tip |
---|
Remember to load the ecmwf-toolbox ! No Format |
---|
$> module load ecmwf-toolbox |
|
...
...
The file tz_an_pl.grib1 contains parameters T and Z on six pressure levels.
How could you use grib_copy to create two files, one containing all of the pressure levels for parameter T, the other for parameter Z ? Check the content of the new files with grib_ls.
Expand |
---|
title | Click here to see solution... |
---|
|
To copy only those messages for parameter T from the GRIB file tz_an_pl.grib1 use grib_copy with the -w option to specify a shortName=t: No Format |
---|
$> grib_copy -w shortName=t tz_an_pl.grib1 t_an_pl.grib1 |
Similarly to copy only those messages for parameter Z, use: No Format |
---|
$> grib_copy -w shortName=z tz_an_pl.grib1 z_an_pl.grib1 |
Or, more simply, and because the file contains only parameters T and Z, one can use: No Format |
---|
$> grib_copy tz_an_pl.grib1 "[shortName]_an_pl.grib1" |
Using grib_ls of the two files confirms that the contents are correct. For example: No Format |
---|
$> grib_ls -p centre,paramId,typeOfLevel,level,date,dataType,time,shortName t_an_pl.grib1
t_an_pl.grib1
centre paramId typeOfLevel level date dataType time shortName
ecmf 130 isobaricInhPa 1000 20170219 an 1200 t
ecmf 130 isobaricInhPa 850 20170219 an 1200 t
ecmf 130 isobaricInhPa 700 20170219 an 1200 t
ecmf 130 isobaricInhPa 500 20170219 an 1200 t
ecmf 130 isobaricInhPa 400 20170219 an 1200 t
ecmf 130 isobaricInhPa 300 20170219 an 1200 t
6 of 6 messages in t_an_pl.grib1
6 of 6 total messages in 1 files |
You can take this further and split the file tz_an_pl.grib1 into separate files for each parameter/pressure level combination with: No Format |
---|
$> grib_copy tz_an_pl.grib1 "[shortName]_[level].grib[edition]"
$> ls ?_*.grib1
t_1000.grib1 t_400.grib1 t_700.grib1 z_1000.grib1 z_400.grib1 z_700.grib1
t_300.grib1 t_500.grib1 t_850.grib1 z_300.grib1 z_500.grib1 z_850.grib1 |
Each file contains one message only, e.g.: No Format |
---|
$> grib_ls -p centre,paramId,typeOfLevel,level,date,dataType,time,shortName t_1000.grib1
t_1000.grib1
centre paramId typeOfLevel level date dataType time shortName
ecmf 130 isobaricInhPa 1000 20170219 an 1200 t
1 of 1 messages in t_1000.grib1
1 of 1 total messages in 1 files |
|
...
The field in the file tp.grib is supposed to contain total precipitation, created using MARS compute to sum the convective and large-scale (stratiform) precipitation.
What is the parameter set to in the file tp.grib ? How could you use grib_set to change it is set correctly to total precipitation (shortName=tp, paramId=228) ?
Expand |
---|
title | Click here to see solution... |
---|
|
Use grib_ls to inspect the contents of the file tp.grib1, printing also the paramId with the –P option: No Format |
---|
$> grib_ls -P paramId tp.grib1
tp.grib1
paramId edition centre typeOfLevel level dataDate stepRange dataType shortName packingType gridType
142 1 ecmf surface 0 20170221 24 fc lsp grid_simple regular_ll
1 of 1 messages in tp.grib1
1 of 1 total messages in 1 files |
Alternatively, you could inspect the parameter namesapce with grib_ls: No Format |
---|
$> grib_ls -n parameter tp.grib1
tp.grib1
centre paramId units name shortName
ecmf 142 m Large-scale precipitation lsp
1 of 1 messages in tp.grib1
1 of 1 total messages in 1 files |
The shortName is set to lsp and paramId to 142, i.e., large-scale precipitation. Info |
---|
The field was created using MARS compute to sum the convective and large-scale (stratiform) precipitation. When using MARS compute, the output field maintains the information from the GRIB header of the first input field so, in this case, we see that the field has paramId 142 even though the message is now supposed to contain the total precipitation. |
To change the parameter to total precipitation, use grib_set as follows: No Format |
---|
$> grib_set -s shortName=tp tp.grib1 tp_new.grib1 |
Check with grib_ls: No Format |
---|
$> grib_ls -P paramId tp_new.grib1
tp_new.grib1
paramId edition centre typeOfLevel level dataDate stepRange dataType shortName packingType gridType
228 1 ecmf surface 0 20170221 24 fc tp grid_simple regular_ll
1 of 1 messages in tp_new.grib1
1 of 1 total messages in 1 files |
Tip |
---|
Note that changing the shortName has also changed the value of paramId. The same could be achieved by setting paramId=228: No Format |
---|
$> grib_set -s paramId=228 tp.grib tp_new.grib |
|
|
...
How could you use the grib_to_netcdf tool to convert the GRIB messages in file1.grib to netCDF and store the result in a file named out1.nc using the default NC_SHORT netCDF data type ?
Expand |
---|
title | Click here to see solution... |
---|
|
To convert the GRIB messages in file.grib1 to netCDF with data type NC_SHORT and store in a file named out1.nc use: No Format |
---|
$> grib_to_netcdf -o out1.nc file1.grib1
grib_to_netcdf: Version 2.30.2
grib_to_netcdf: Processing input file 'file1.grib1'.
grib_to_netcdf: Found 4 GRIB fields in 1 file.
grib_to_netcdf: Ignoring key(s): method, type, stream, refdate, hdate
grib_to_netcdf: Creating netCDF file 'out1.nc'
grib_to_netcdf: NetCDF library version: 4.9.1 of Feb 9 2023 13:54:09 $
grib_to_netcdf: Creating large (64 bit) file format.
grib_to_netcdf: Defining variable 't2m'.
grib_to_netcdf: Done. |
|
...
No Format |
---|
$> module load netcdf4
$> ncdump -v t2m out1.nc |
Expand |
---|
title | Click here to see solution... |
---|
|
Using ncdump to print the values for the variable t2m shows: No Format |
---|
$> module load netcdf4
$> ncdump -v t2m out1.nc
netcdf out1 {
...
data:
t2m =
20497, -809,
-16312, -32766,
32767, 7042,
-1312, -18041,
17168, -12207,
-4148, -28392,
19382, -10827,
-2506, -25406 ;
} |
Tip |
---|
The data values appear as integers because of the NC_SHORT data format. These values need to be unpacked using the scale_factor and add_offset NetCDF attributes. |
|
Expand |
---|
title | Click here to see solution... |
---|
|
To store the values in the netCDF file as floating point numbers, use the grib_to_netcdf tool with the data format to NC_FLOAT or NC_DOUBLE with the -D option. For example: No Format |
---|
$> grib_to_netcdf -D NC_FLOAT -o out2.nc file1.grib1
grib_to_netcdf: Version 2.30.2
grib_to_netcdf: Processing input file 'file1.grib1'.
grib_to_netcdf: Found 4 GRIB fields in 1 file.
grib_to_netcdf: Ignoring key(s): method, type, stream, refdate, hdate
grib_to_netcdf: Creating netCDF file 'out2.nc'
grib_to_netcdf: NetCDF library version: 4.9.1 of Feb 9 2023 13:54:09 $
grib_to_netcdf: Creating large (64 bit) file format.
grib_to_netcdf: Defining variable 't2m'.
grib_to_netcdf: Done. |
Checking with ncdump, we see the values are now stored as floating point values: No Format |
---|
ncdump -v t2m out2.nc
netcdf out2 {
...
data:
t2m =
281.1666, 277.9943,
275.6859, 273.236,
282.9937, 279.1632,
277.9194, 275.4285,
280.671, 276.2971,
277.4971, 273.8872,
281.0007, 276.5026,
277.7416, 274.3318 ;
} |
In this case, the data values do not need further processing. |
Expand |
---|
title | Click here to see solution... |
---|
|
Converting file2.grib to netCDF gives the following error: No Format |
---|
grib_to_netcdf -o out2.nc file2.grib1
grib_to_netcdf: Version 2.30.2
grib_to_netcdf: Processing input file 'file2.grib1'.
grib_to_netcdf: Found 1 GRIB field in 1 file.
grib_to_netcdf: Ignoring key(s): method, type, stream, refdate, hdate
grib_to_netcdf: Creating netCDF file 'out2.nc'
grib_to_netcdf: NetCDF library version: 4.9.1 of Feb 9 2023 13:54:09 $
grib_to_netcdf: Creating large (64 bit) file format.
ECCODES ERROR : Grid type = reduced_gg
ECCODES ERROR : First GRIB is not on a regular lat/lon grid or on a regular Gaussian grid. Exiting. |
The conversion fails because the GRIB data is represented on a reduced Gaussian grid (gridType=reduced_gg). Conversion to NetCDF is possible only for GRIB data with gridType=regular_ll or gridType=regular_gg. To fix this it would be necessary to re-retrieve the data on a regular grid or interpolate to a regular grid. |
...