GRIB API: Library and Tools - Training Course 2015

Tuesday 24 February

09:00 Welcome - Umberto Modigliani

• Lecturers
• Students
  

09:15 GRIB API Introduction - Shahram Najm

• GRIB editions 1 and 2: structure and differences
  
• GRIB API design and concepts

10:30 Coffee

11:00 GRIB API: Tools - Carsten Maass

• Introduction to the grib_api command-line tools
• Obtaining information about the grib_api installation with grib_info

• Inspecting the content of GRIB files with grib_count,  grib_ls and grib_dump

  Practical solution

  Solution to grib_dump and grib_keys practical
  
  1.  To view the keys available for GRIB1 and GRIB2 have a look at GRIB API or http://old.ecmwf.int/publications/manuals/grib_api/
  
  2.  To use grib_dump to inspect the contents of file1.grib1:
  
    % grib_dump file1.grib1
  
  By using grib_dump on its own and with the various options, (-O, -a, etc), you should find:
  
  •    the message is encoded using GRIB edition 1 (editionNumber=1);
  •    it contains the 2 metre temperature field (shortName=2t, marsParam=167.128);
  •    the dataDate= 20110225, the dataTime=12 and the forecast step=12;
  •    the data is on a reduced Gaussian grid (typeOfGrid=reduced_gg) with N=128 (Nj=256);
  
  Alternatively, with grib_dump inspect some templates in the “GRIB_SAMPLES_PATH” directory.
   
  3.  For the file file2.grib2
  
     % grib_dump file1.grib2
  
  By using grib_dump on its own and with various options, (-O, -a etc), you should find:
  
  •    the message is encoded using GRIB edition 2 - grib_edition=2;
  •    it contains the temperature field on a specified hight of 2 m above ground (shortName=2t, paramId = 167, discipline = 0, parameterCategory = 0, parameterNumber = 0, typeOfFirstFixedSurface = 103);
  •    the dataDate=20110225, the dataTime=12 and the forecast step=12;
  •    the data is on a reduced Gaussian grid (typeOfGrid=reduced_gg) with N=128 (Nj=256);
  
  4. To quickly find the maximum, minimum and average values of the field, look for the computed keys max, min, and average.  
  
  For both files you should find:
  
  •    maximum = 314.204
  •    minimum = 222.44
  •    average = 285.261
   
  Solution to grib_ls practical
  
  
  1.  To list the GRIB messages in msl.grib1

     %  grib_ls msl.grib1

     msl.grib1

  edition      centre       typeOfLevel  level        dataDate     stepRange    dataType     shortName    packingType  gridType

  1            ecmf         surface      0            20110225     0            cf           msl          grid_simple  regular_gg

  1            ecmf         surface      0            20110225     6            cf           msl          grid_simple  regular_gg

  1            ecmf         surface      0            20110225     12           cf           msl          grid_simple  regular_gg

  1            ecmf         surface      0            20110225     18           cf           msl          grid_simple  regular_gg

  1            ecmf         surface      0            20110225     24           cf           msl          grid_simple  regular_gg

  5 of 5 grib messages in msl.grib1

  5 of 5 total grib messages in 1 files

   

  
  

  The file contains mean sea-level pressure (shortName=msl) from the ECMWF EPS control forecast (dataType=cf) for 6-hourly steps from 0 to 24 hours for dataDate=20110225.  The messages are encoded in GRIB edition 1.
  
  
  
  2.  To list the GRIB messages in msl.grib2
  
  %  grib_ls msl.grib2
  msl.grib2
  edition      centre       date         dataType     gridType     stepRange    typeOfLevel  level        shortName    packingType
  2            ecmf         20110225     cf           regular_gg   0            meanSea      0            msl          grid_simple
  2            ecmf         20110225     cf           regular_gg   6            meanSea      0            msl          grid_simple
  2            ecmf         20110225     cf           regular_gg   12           meanSea      0            msl          grid_simple
  2            ecmf         20110225     cf           regular_gg   18           meanSea      0            msl          grid_simple
  2            ecmf         20110225     cf           regular_gg   24           meanSea      0            msl          grid_simple
  5 of 5 grib messages in msl.grib2
  
  5 of 5 total grib messages in 1 files
  
  The file also contains mean sea-level pressure (shortName=msl) from the ECMWF EPS control forecast (dataType=cf) for 6-hourly steps from 0 to 24 hours for dataDate=20110225.  In this file the messages are encoded in GRIB edition 2.
  
  The lists show the default keys printed by grib_ls for GRIB 1 and GRIB 2.
  
  
  
  3.  To print the mars keys, use grib_ls -m:
  
  
     % grib_ls -m msl.grib1
     msl.grib1
     domain      levtype     date        time        step        param       class       type        stream      expver
     g           sfc         20110225    0000        0           151.128     od          cf          enfo        0001
     g           sfc         20110225    0000        6           151.128     od          cf          enfo        0001
     g           sfc         20110225    0000        12          151.128     od          cf          enfo        0001
     g           sfc         20110225    0000        18          151.128     od          cf          enfo        0001
     g           sfc         20110225    0000        24          151.128     od          cf          enfo        0001
    5 of 5 grib messages in msl.grib1
  
    5 of 5 total grib messages in 1 files
  
     % grib_ls -m msl.grib1
     msl.grib2
     domain      date        time        expver      class       type        stream      step        levtype     levelist    param
     g           20110225    0000        0001        od          cf          enfo        0           sfc         0           151
     g           20110225    0000        0001        od          cf          enfo        6           sfc         0           151
     g           20110225    0000        0001        od          cf          enfo        12          sfc         0           151
     g           20110225    0000        0001        od          cf          enfo        18          sfc         0           151
     g           20110225    0000        0001        od          cf          enfo        24          sfc         0           151
     5 of 5 grib messages in msl.grib2
  
     5 of 5 total grib messages in 1 files
  
  
  
  4.  Output can be printed in descending step order using grib_ls with the -B option.  Note it is necessary to use “step:l” here as step has a default type of ‘string’.
  
     % grib_ls -B "step:l desc" msl.grib1
  
     edition      centre       typeOfLevel  level        dataDate     stepRange    dataType     shortName    packingType  gridType
     1            ecmf         surface      0            20110225     24           cf           msl          grid_simple  regular_gg
     1            ecmf         surface      0            20110225     18           cf           msl          grid_simple  regular_gg
     1            ecmf         surface      0            20110225     12           cf           msl          grid_simple  regular_gg
     1            ecmf         surface      0            20110225     6            cf           msl          grid_simple  regular_gg
     1            ecmf         surface      0            20110225     0            cf           msl          grid_simple  regular_gg
  
     % grib_ls -B "step:l desc" msl.grib2
     edition      centre       date         dataType     gridType     stepRange    typeOfLevel  level        shortName    packingType
     2            ecmf         20110225     cf           regular_gg   24           meanSea      0            msl          grid_simple
     2            ecmf         20110225     cf           regular_gg   18           meanSea      0            msl          grid_simple
     2            ecmf         20110225     cf           regular_gg   12           meanSea      0            msl          grid_simple
     2            ecmf         20110225     cf           regular_gg   6            meanSea      0            msl          grid_simple
     2            ecmf         20110225     cf           regular_gg   0            meanSea      0            msl          grid_simple
  
  
  5.  To specify a set of keys to print, use the -p option:
  
    % grib_ls -p centre,dataDate,stepRange,typeOfLevel,shortName  msl.grib1
  msl.grib1
  centre       dataDate     stepRange     typeOfLevel  shortName
  ecmf         20110225     0             surface      msl
  ecmf         20110225     6             surface      msl
  ecmf         20110225     12            surface      msl
  ecmf         20110225     18            surface      msl
  ecmf         20110225     24            surface      msl
  5 of 5 grib messages in msl.grib1
  
  5 of 5 total grib messages in 1 files
  
  Using the "key:l" for centre and levelType prints these keys as their integer values rather than strings:
  
    % grib_ls -p centre:l,dataDate,stepRange,levelType:l,shortName,paramId msl.grib1
  msl.grib1
  centre       dataDate     stepRange    levelType   shortName    paramId
  98           20110225     0            1            msl          151
  98           20110225     6            1            msl          151
  98           20110225     12           1            msl          151
  98           20110225     18           1            msl          151
  98           20110225     24           1            msl          151
  5 of 5 grib messages in msl.grib1
  
  5 of 5 total grib messages in 1 files
  
  Similarly for msl.grib2:
  
    % grib_ls -p centre:l,dataDate,stepRange,levelType:l,shortName,paramId msl.grib2
  msl.grib2
  centre       dataDate     stepRange    levelType  shortName    paramId
  98           20110225     0            101          msl          151
  98           20110225     6            101          msl          151
  98           20110225     12           101          msl          151
  98           20110225     18           101          msl          151
  98           20110225     24           101          msl          151
  5 of 5 grib messages in msl.grib2
  
  5 of 5 total grib messages in 1 files
  
  
  
   
  Solution to grib_ls -l  practical
  
  
  1.  To list the nearest points to ECMWF (Lat 51.42 N, Lon 0.95 W) use
  
     % grib_ls -l 51.42,-0.95 msl.grib1
     …
  
     5 of 5 total grib messages in 1 files
     Input Point: latitude=51.42  longitude=-0.95
     Grid Point chosen #3 index=17199 latitude=51.62 longitude=359.10 distance=22.57 (Km)
     Other grid Points
     - 1 - index=17599 latitude=50.72 longitude=359.10 distance=77.54 (Km)
     - 2 - index=17598 latitude=50.72 longitude=358.20 distance=97.59 (Km)
     - 3 - index=17199 latitude=51.62 longitude=359.10 distance=22.57 (Km)
     - 4 - index=17198 latitude=51.62 longitude=358.20 distance=62.87 (Km)
     
     Note we specify the longitude of 0.95 W with -0.95 !
  
     The nearest grid point is at latitude=51.62 longitude=359.10 (=0.9 W) and is 22.57 km from ECMWF.
  
  
  2. To output only the forecast step and MSLP value at the nearest grid point use:
  
  % grib_ls -l 51.42,-0.95,1 -p stepRange msl.grib1
     msl.grib1
     stepRange    value
     0           102527
     6           102365
     12          102456
     18          102289
     24          101862
     5 of 5 grib messages in msl.grib1
  
     5 of 5 total grib messages in 1 files
     Input Point: latitude=51.42  longitude=-0.95
     Grid Point chosen #3 index=17199 latitude=51.62 longitude=359.10 distance=22.57 (Km)
     Other grid Points
     - 1 - index=17599 latitude=50.72 longitude=359.10 distance=77.54 (Km)
     - 2 - index=17598 latitude=50.72 longitude=358.20 distance=97.59 (Km)
     - 3 - index=17199 latitude=51.62 longitude=359.10 distance=22.57 (Km)
     - 4 - index=17198 latitude=51.62 longitude=358.20 distance=62.87 (Km)
   
  Note it is necessary to specify MODE=1 in order to print the value of the nearest grid point only.  
  
  To output the values at the four grid points nearest to ECMWF, use MODE=4 which is the default:
  
  % grib_ls -l 51.42,-0.95 -p stepRange msl.grib1
     msl.grib1
     stepRange          value1  value2  value3  value4
     0           102674 102663 102527 102501
     6           102532 102516 102365 102333
     12          102633 102585 102456 102426
     18          102444 102390 102289 102259
     24          102057 101959 101862 101760
     5 of 5 grib messages in msl.grib1
  
     5 of 5 total grib messages in 1 files
     Input Point: latitude=51.42  longitude=-0.95
     Grid Point chosen #3 index=17199 latitude=51.62 longitude=359.10 distance=22.57 (Km)
     Other grid Points
     - 1 - index=17599 latitude=50.72 longitude=359.10 distance=77.54 (Km)
     - 2 - index=17598 latitude=50.72 longitude=358.20 distance=97.59 (Km)
     - 3 - index=17199 latitude=51.62 longitude=359.10 distance=22.57 (Km)
     - 4 - index=17198 latitude=51.62 longitude=358.20 distance=62.87 (Km)
   
  3. To specify the land-sea mask, use:
  
  % grib_ls -l 51.42,-0.95,1,lsm.grib1 -p stepRange msl.grib1
  msl.grib1
  stepRange    value
  0           102527
  6           102365
  12          102456
  18          102289
  24          101862
  5 of 5 grib messages in msl.grib1
  
  5 of 5 total grib messages in 1 files
  Input Point: latitude=51.42  longitude=-0.95
  Grid Point chosen #3 index=17199 latitude=51.62 longitude=359.10 distance=22.57 (Km)
  Mask values:
  - 1 - index=17599 latitude=50.72 longitude=359.10 distance=77.54 (Km) value=0.00
  - 2 - index=17598 latitude=50.72 longitude=358.20 distance=97.59 (Km) value=1.00
  - 3 - index=17199 latitude=51.62 longitude=359.10 distance=22.57 (Km) value=1.00
  - 4 - index=17198 latitude=51.62 longitude=358.20 distance=62.87 (Km) value=1.00
  
  Here the mask value at the end of the line showing the four nearest grid points indicates which are land points: Land points have mask ≥ 0.5; sea points have mask<0.5.

12:15 GRIB API: Keys - Shahram Najm

• Coded and computed keys
• Namespaces
• Examples of keys

13:00 Lunch

14:00 GRIB API: Tools - Paul Dando

• Comparing the content of GRIB messages with grib_compare
• Printing data values with grib_get_data
• Using grib_get

15:30 Coffee

16:00 GRIB API: Tools - Paul Dando

• Copying GRIB messages with grib_copy
• Making simple modifications to the GRIB header information with grib_set

16:55 Questionnaire/Feedback

17:00 Cocktails

Wednesday 25 February

09:00 GRIB API: Fortran, C, Python APIs - Basics - Dominique Lucas, Xavi Abellan, Cristian Simarro

• Overview of the API
• Using the API functions for decoding with Fortran, C and Python
•  Practicals  
  

10:30 Coffee

11:00 GRIB API: Fortran, C, Python APIs (continued) - Dominique Lucas, Xavi Abellan, Cristian Simarro

• Practicals  

13:00 Lunch

14:00 GRIB API: Advanced tools - Paul Dando

• Basic features of grib_filter
• Conversion from GRIB to NetCDF with grib_to_netcdf

15:30 Coffee

16:00 GRIB API: General practicals

16:55 Questionnaire/Feedback

17:00 End

Thursday 26 February

09:00 GRIB API: Fortran, C, python APIs - Advanced features - Dominique Lucas, Xavi Abellan, Cristian Simarro

• Use of the indexing routines
• Encoding GRIB messages and using GRIB samples
  
• Practicals 
  

10:30 Coffee

11:00 GRIB API: Fortran, C, python APIs - Advanced features (continued) - Dominique Lucas, Xavi Abellan, Cristian Simarro

• Practicals 
  

13:00 Lunch

14:00 GRIB API: Python interface - Xavi Abellan

• A short introduction to Python and its use at ECMWF
• The Python interface to grib_api for decoding, encoding and indexed access to GRIB messages

15:30 Coffee

16:00 GRIB API: Advanced topics I - Shahram Najm

• Simple packing
• Constant fields
• Bitmap fields
• Multi fields

16:55 Questionnaire/Feedback

17:00 End

Friday 27 February

09:00 GRIB API: Advanced topics II - Shahram Najm

• Parameter database
• Conversion from GRIB 1 to GRIB 2

10:30 Coffee

11:00 GRIB API: Advanced topics III - Shahram Najm

• Local adaptation - how to define your own local parameters

12:15 Quiz

12:55 Questionnaire/Feedback

13:00 End of module COM GRIB