Info |
---|
The ecCodes GRIB tools are documented at: There are examples of how to use them at: Other useful resources: |
Preliminaries
Login to ecs-login and work Work in your $SCRATCH:
Make a copy of the practicals directory in your $SCRATCH to Atos:
No Format |
---|
$> tar –xvf-xvf /home/trx/grib_practicals.tar |
This will create a directory in your $SCRATCH called grib_practicals containing the GRIB data files for all today’s practicalsthis tutorial.
There are sub-directories for each practical:
No Format |
---|
$> ls $SCRATCH/grib_practicals
inspect modify python |
For this tutorial cd to the inspect directory:
No Format |
---|
$> cd $SCRATCH/grib_practicals/inspect |
Tip |
---|
Remember to load the ecmwf-toolbox ! No Format |
---|
$> module load ecmwf-toolbox |
|
This tutorial covers the following topics:
...
Use grib_ls to inspect the content of the files t2m.grib1 and t2m.grib2
What fields do the GRIB messages contain ?
Expand |
---|
title | Click here to see solution... |
---|
|
To list the GRIB messages in t2m.grib1 No Format |
---|
$> grib_ls t2m.grib1
t2m.grib1
edition centre typeOfLevel level dataDate stepRange dataType shortName packingType gridType
1 ecmf surface 0 20170217 0 cf 2t grid_simple reduced_gg
1 ecmf surface 0 20170217 6 cf 2t grid_simple reduced_gg
1 ecmf surface 0 20170217 12 cf 2t grid_simple reduced_gg
1 ecmf surface 0 20170217 18 cf 2t grid_simple reduced_gg
1 ecmf surface 0 20170217 24 cf 2t grid_simple reduced_gg
5 of 5 grib messages in t2m.grib1
5 of 5 total grib messages in 1 file |
The file contains 2 metre temperature (shortName=2tl) from the ECMWF Ensemble control forecast (dataType=cf) for 6-hourly steps (stepRange) from 0 to 24 hours for dataDate=20170217. Similarly, for t2m.grib2: No Format |
---|
$> grib_ls t2m.grib2
t2m.grib2
edition centre date dataType gridType stepRange typeOfLevel level shortName packingType
2 ecmf 20170217 cf reduced_gg 0 heightAboveGround 2 2t grid_simple
2 ecmf 20170217 cf reduced_gg 6 heightAboveGround 2 2t grid_simple
2 ecmf 20170217 cf reduced_gg 12 heightAboveGround 2 2t grid_simple
2 ecmf 20170217 cf reduced_gg 18 heightAboveGround 2 2t grid_simple
2 ecmf 20170217 cf reduced_gg 24 heightAboveGround 2 2t grid_simple
5 of 5 grib messages in t2m.grib1
5 of 5 total grib messages in 1 files |
Again, the file contains 2 metre temperature (shortName=2t) from the ECMWF Ensemble control forecast (dataType=cf) for 6-hourly steps (stepRange) from 0 to 24 hours for dataDate=20170217. In fact, they are the same fields as in t2m.grib1 but encoded in GRIB edition 2 and obtained from the TIGGE archive. |
Expand |
---|
title | Click here to see solution... |
---|
|
No, grib_ls prints the same set of default keys for the both GRIB edition 1 and GRIB edition2. However, the order of the columns output is different. The only differences seen with grib_ls is for the value of the typeOfLevel and level keys which are - typeOfLevel=surface and level=0 inthe GRIB1 encoding; and
- typeOfLevel=heightAboveGround and level=2 for the GRIB2 encoding.
|
Expand |
---|
title | Click here to see solution... |
---|
|
To print the mars keys, use either grib_ls -m or grib_ls -n mars. This is the output for the file t2m.grib1: No Format |
---|
$> grib_ls -m t2m.grib1
t2m.grib1
domain levtype date time step param class type stream expver
g sfc 20170217 0000 0 167.128 od cf enfo 0001
g sfc 20170217 0000 6 167.128 od cf enfo 0001
g sfc 20170217 0000 12 167.128 od cf enfo 0001
g sfc 20170217 0000 18 167.128 od cf enfo 0001
g sfc 20170217 0000 24 167.128 od cf enfo 0001
5 of 5 grib messages in t2m.grib1
5 of 5 total grib messages in 1 files |
For t2m.grib2, the same command gives: No Format |
---|
$> grib_ls -m t2m.grib2
t2m.grib2
origin date time step levtype number param expver class model type stream
ecmf 20170217 0000 0 sfc 0 167 prod ti glob cf enfo
ecmf 20170217 0000 6 sfc 0 167 prod ti glob cf enfo
ecmf 20170217 0000 12 sfc 0 167 prod ti glob cf enfo
ecmf 20170217 0000 18 sfc 0 167 prod ti glob cf enfo
ecmf 20170217 0000 24 sfc 0 167 prod ti glob cf enfo
5 of 5 grib messages in t2m.grib2
5 of 5 total grib messages in 1 files |
|
Expand |
---|
title | Click here to see solution... |
---|
|
To print the shortName along with the list of mars keys use the –P option e.g.: No Format |
---|
$> grib_ls -m -P shortName t2m.grib1
t2m.grib1
shortName domain levtype date time step param class type stream expver
2t g sfc 20170217 0000 0 167.128 od cf enfo 0001
2t g sfc 20170217 0000 6 167.128 od cf enfo 0001
2t g sfc 20170217 0000 12 167.128 od cf enfo 0001
2t g sfc 20170217 0000 18 167.128 od cf enfo 0001
2t g sfc 20170217 0000 24 167.128 od cf enfo 0001
5 of 5 grib messages in t2m.grib1
5 of 5 total grib messages in 1 files |
|
How do you change the previous command so that the output is in descending step order ?
Tip |
---|
Hint: think about stings strings and integers |
Expand |
---|
title | Click here to see the solution... |
---|
|
To order the output in descending step order use the –B option: No Format |
---|
$> grib_ls -B "step:i desc" t2m.grib1
edition centre typeOfLevel level dataDate stepRange dataType shortName packingType gridType
1 ecmf surface 0 20170217 24 cf 2t grid_simple reduced_gg
1 ecmf surface 0 20170217 18 cf 2t grid_simple reduced_gg
1 ecmf surface 0 20170217 12 cf 2t grid_simple reduced_gg
1 ecmf surface 0 20170217 6 cf 2t grid_simple reduced_gg
1 ecmf surface 0 20170217 0 cf 2t grid_simple reduced_gg
5 of 5 grib messages in t2m.grib1
5 of 5 total grib messages in 1 files |
Tip |
---|
Note that you need to specify “step:i” so that step is treated as an integer rather than a character string. |
|
Expand |
---|
title | Click here to see solution... |
---|
|
To specify a set of keys to print, use the -p option: No Format |
---|
$> grib_ls -p centre,dataDate,stepRange,typeOfLevel,shortName 2tm.grib1
t2m.grib1
centre dataDate stepRange typeOfLevel shortName
ecmf 20170217 0 surface t2m
ecmf 20170217 6 surface t2m
ecmf 20170217 12 surface t2m
ecmf 20170217 18 surface t2m
ecmf 20170217 24 surface t2m
5 of 5 grib messages in t2m.grib1
5 of 5 total grib messages in 1 files |
|
Expand |
---|
title | Click here to see solution... |
---|
|
To print the keys for forecast step 6 only, use the -p option with the -w option: No Format |
---|
$> grib_ls -w stepRange=6 -p centre,dataDate,stepRange,typeOfLevel,shortName t2m.grib1
t2m.grib1
centre date stepRange levelType shortName
ecmf 20170217 6 surface t2m
1 of 5 grib messages in t2m.grib1
1 of 5 total grib messages in 1 files |
Using the same commands with the file t2m.grib2 gives similar output. |
Expand |
---|
title | Click here to see solution... |
---|
|
Specifying the key as "centre:i" and “centre:s” prints the centre both as an integer and a string: No Format |
---|
$> grib_ls -w stepRange=6 -p centre:li,centre:s,dataDate,stepRange,typeOfLevel,shortName t2m.grib1
t2m.grib1
centre centre dataDate stepRange typeOfLevel shortName
98 ecmf 20170217 6 surface t2m
1 of 5 grib messages in t2m.grib1
1 of 5 total grib messages in 1 files |
As before, using the same commands with the file t2m.grib2 gives similar output. |
Expand |
---|
title | Click here to see solution... |
---|
|
To use grib_dump to inspect the contents of the fourth message in t2m.grib1 and t2m.grib2 use the -w option with "count=4" (the count key counts the number of messages starting form count=1 for the first message. To print the output in WMO Octet mode use the -O option: No Format |
---|
$> grib_dump –w count=4 -O t2m.grib1
...
$> grib_dump –w count=4 -O t2m.grib2
... |
|
...
Expand |
---|
title | Click here to see solution... |
---|
|
To use grib_dump to all of the keys for the fourth message in JSON mode, use the -j option: No Format |
---|
$> grib_dump -j -w count=4 t2m.grib1 > t2m.grib1.output
$> head -20 t2m.grib1.output
{ "messages" : [
[
{
"key" : "editionNumber",
"value" : 1
},
{
"key" : "table2Version",
"value" : 128
},
{
"key" : "centre",
"value" : 98
},
{
"key" : "generatingProcessIdentifier",
"value" : 147
},
{ |
Similarly, for t2m.grib2: No Format |
---|
$> grib_dump -j -w count=4 t2m.grib2 > t2m.grib2.output
$> head -20 t2m.grib1.output
{ "messages" : [
[
{
"key" : "discipline",
"value" : 0
},
{
"key" : "editionNumber",
"value" : 2
},
{
"key" : "centre",
"value" : 98
},
{
"key" : "subCentre",
"value" : 0
},
{ |
|
...
The file t2m.grib1 contains the 2m temperature from the ENS control forecast at 6-hourly time steps for the first 24 hours on the O640 octahedral reduced Gaussian grid. The file lsm.grib1 contains the ENS land-sea mask on the O640 grid.
Tip |
---|
Be careful to specify the longitude correctly ! |
...
Expand |
---|
title | Click here to see solution... |
---|
|
To list the nearest points to Lat 51.42° N, Lon 0.95° W use: No Format |
---|
$> grib_ls -l 51.42,-0.95 t2m.grib1
t2m.grib1
...
5 of 5 grib messages in t2m.grib1
5 of 5 total grib messages in 1 files
Input Point: latitude=51.42 longitude=-0.95
Grid Point chosen #2 index=156197 latitude=51.38 longitude=359.03 distance=4.79 (Km)
Other grid Points
- 1 - index=156198 latitude=51.38 longitude=359.35 distance=21.64 (Km)
- 2 - index=156197 latitude=51.38 longitude=359.03 distance=4.79 (Km)
- 3 - index=155082 latitude=51.52 longitude=359.35 distance=23.65 (Km)
- 4 - index=155081 latitude=51.52 longitude=359.03 distance=11.09 (Km) |
Tip |
---|
Note a longitude of 0.95° W must be specified as -0.95 ! |
Info |
---|
Note that ecCodes converts all longitude values so that they lie in the interval 0° to 360°. In GRIB 2 all longitude values must be positive ! |
|
Expand |
---|
title | Click here to see solution... |
---|
|
The nearest grid point is at latitude=51.38° longitude=359.03° (0.77° W) and is 4.79 km from ECMWF. |
Expand |
---|
To output only the forecast step and 2 metre temperature value at the nearest grid point set the MODE option to 1 and specify that the stepRange key only should be printed using the -p option: No Format |
---|
$> grib_ls -l 51.42,-0.95,1 -p stepRange t2m.grib1
t2m.grib1
stepRange value
0 279.889
6 278.54
12 282.242
18 278.133
24 276.823
5 of 5 messages in t2m.grib1
5 of 5 total messages in 1 files
Input Point: latitude=51.42 longitude=-0.95
Grid Point chosen #2 index=156197 latitude=51.38 longitude=359.03 distance=4.79 (Km)
Other grid Points
- 1 - index=156198 latitude=51.38 longitude=359.35 distance=21.64 (Km)
- 2 - index=156197 latitude=51.38 longitude=359.03 distance=4.79 (Km)
- 3 - index=155082 latitude=51.52 longitude=359.35 distance=23.65 (Km)
- 4 - index=155081 latitude=51.52 longitude=359.03 distance=11.09 (Km)
|
|
Expand |
---|
The output format of the values is specified with the -F option. Here, -F"%.2f" is used to specify decimal format with 2 decimal places. To specify the land-sea mask, pass the name of the file containing the land-sea mask as the FILE option: No Format |
---|
% grib_ls -F"%.2f" -l 51.42,-0.95,1,lsm.grib1 -p step t2m.grib1
t2m.grib1
step value
0 279.89
6 278.54
12 282.24
18 278.13
24 276.82
5 of 5 messages in t2m.grib1
5 of 5 total messages in 1 files
Input Point: latitude=51.42 longitude=-0.95
Grid Point chosen #2 index=156197 latitude=51.38 longitude=359.03 distance=4.79 (Km)
Mask values:
- 1 - index=156198 latitude=51.38 longitude=359.35 distance=21.64 (Km) value=1.00
- 2 - index=156197 latitude=51.38 longitude=359.03 distance=4.79 (Km) value=0.99
- 3 - index=155082 latitude=51.52 longitude=359.35 distance=23.65 (Km) value=0.99
- 4 - index=155081 latitude=51.52 longitude=359.03 distance=11.09 (Km) value=0.99 |
Here the value=1.00 (or 0.99) at the end of the line showing the four nearest grid points indicates which are land points (mask=1.0). Land points have mask ≥ 0.5; sea points have mask<0.5. Tip |
---|
Once the index of the nearest grid point is known, its value can be obtained directly with the -i option: No Format |
---|
$> grib_ls -F"%.2f" -i 156197 -p step t2m.grib1
t2m.grib1
step value(156197)
0 279.89
6 278.54
12 282.24
18 278.13
24 276.82
5 of 5 messages in t2m.grib1
5 of 5 total messages in 1 files |
|
|
...
How do you print the latitude, longitude and values for the field in surface1.grib with the values output in decimal format with 5 decimal places ?
Expand |
---|
title | Click here to see solution... |
---|
|
To print the data values for surface.grib1 in decimal format with 6 decimal places use grib_get_data with the option -F"%.6f": No Format |
---|
$> grib_get_data -F"%.6f" surface.grib1
Latitude Longitude Value
2.500 -20.000 301.848755
2.500 -17.500 302.186279
2.500 -15.000 301.810547
2.500 -12.500 301.721924
... |
|
Expand |
---|
title | Click here to see solution... |
---|
|
To output also the latitude and longitude values with 6 decimal places, we need to use the -L option: No Format |
---|
grib_get_data -L"%.6f %.6f" -F"%.6f" surface.grib1
Latitude Longitude Value
2.500000 -20.000000 301.848755
2.500000 -17.500000 302.186279
2.500000 -15.000000 301.810547
2.500000 -12.500000 301.721924
... |
Tip |
---|
Note that we need to specify the format for both latitude and longitude separately: -L"%.6f %.6f". |
|
Expand |
---|
title | Click here to see solution... |
---|
|
Missing values can be identified by using grib_get_data with the –m option, for example: No Format |
---|
$> grib_get_data –m MISSING surface.grib1
Latitude, Longitude, Value
...
2.500 7.500 3.0176782227e+02
2.500 10.000 MISSING
0.000 -20.000 3.0134069824e+02
... |
Tip |
---|
You can also print the key numberOfMissing with the grib_get or grib_ls tools: No Format |
---|
$> grib_get -p numberOfMissing surface.grib1
3 |
So there are 3 missing values in total. Other keys also provide the numberOfPoints and numberOfValues: No Format |
---|
grib_get -p numberOfPoints,numberOfValues,numberOfMissing surface.grib1
143 140 3 |
So that: No Format |
---|
numberOfPoints = numberOfValues + numberOfMissing |
|
|
...