Using the GRIB tools
The GRIB tools are part of the ECMWF GRIB API Library. They are a set of command line programs for interactive and batch decoding and processing of GRIB data
They provide ready and tested solutions to the most common processing of GRIB data and work with both GRIB edition 1 and GRIB edition 2.
All of the tools use a common syntax:
There are tools for getting information about the GRIB API installation and the keys available
- grib_info, grib_keys
There are tools to inspect the content of and compare GRIB messages
- grib_ls, grib_dump, grib_get, grib_get_data, grib_compare
There are tools for counting and copying some messages
- grib_count, grib_copy
There is a tool for making changes to the content of a GRIB message
There is an advanced tool with a macro language provided via a rules file which allows for more sophisticated manipulation of GRIB files
There is a tool for converting from GRIB to netCDF
You will find some example data in $SCRATCH/ectrain/trx/openifs or you can use output from one of your own experiments.
Let's play !
Getting help with the tools
Running any of the GRIB API tools without options or input files will produce a UNIX 'man page' style output describing the usage. Try:
Listing the contents of GRIB files
An inventory of the contents of GRIB files can be obtained with the grib_ls command.
For a single file, try:
Counting the number of messages in a file
Customising the output of grib_ls
Specifying the information to display:
Specifying the type of key (s: string, i: integer). To output the centre as a string (default):
or as an integer:
Listing specific namespace information. Try:
Other namespaces to try are time, vertical and geography.
Order the output. Try:
Compare and contrast with:
The where option: IS
Specifying the step unit. Compare the output from:
The stepUnits can be set to s,m,h,3h,6h,12h,D,M,Y,10Y,30Y or C. The result must be expressible as an integer (try setting stepUnits=D above). This is a temporary change for display purposes only !
grib_ls is a tool meant mainly for interactive use. If you want to use the tools in a shell script then take a look at grib_get instead. It works in much the same way as grib_ls but is less verbose and also fails if an error occurs.
Obtaining detailed information about a GRIB message with grib_dump
To print all of the keys available for a particular GRIB message:
To obtain a listing which is more like the WMO tables:
To obtain even more detail:
Printing the data values
For grid point data, the grib_get_data tool prints a list of latitude, longitude and data values from one or more GRIB files.
The format of the data values can be controlled with a C-style format statement. Try:
Other information can be printed along with the lat-lon-values triplet:
If you need the values in a program then do not be tempted to use grib_get_data to output the information to a file and then read the file in your program. It is more efficient to use the appropriate Fortran 90, C or Python interface to GRIB API to access the data values directly.
Using the 'smart' copy
The structure of GRIB allows files to be simply copied and concatenated with the UNIX 'cat' command. However, it's sometimes desirable to do something more sophisticated. The grib_copy tool provides a way to copy only selected content from GRIB files.
To copy all of the pressure level data at 1000 hPa from a number of separate files and store the output in a single file, try:
Remember that you can check the output file with grib_ls to see the result !
To do the same with only temperature field and to store in descending forecast step order, try:
Now let's gather all of the temperature fields and store in order of decreasing level and decreasing step:
It is important when using the "order by" (-B) option to specify the key type with, e.g., "level:i" to avoid unexpected (unwanted ?) results. Remember that the key values are strings by default. Usually, we want a numeric order.
Key names can be used to specify the output file names. Try:
This provides a very convenient way to filter GRIB messages into separate files.
Setting key/value pairs
The grib_set tool allows key values to be set or modified. It is also able to make simple global changes to the data values themselves.
Suppose we want to use the soil temperature on level 1 to provide a sea-surface temperature. We can change the header information for this parameter and set it to sea-surface temperature
By default, all parameters are copied to the output file. The 'strict' option (-S) writes only those messages that are changed to the output file. To see the difference, try running grib_set again but without the -S option !
This is still not encoded like a true sea-surface temperature. To do this, we need also to change the typeOfLevel:
An offset can be added to all data values:
Remember: use "grib_ls -n statistics" to make a quick check of the values in output file.
The values can be multiplied by a constant factor:
Some values are read only and cannot be changed. Try:
The units key cannot be set as it is read only. the only way to change the units key is by changing the parameter (either the shortName or paramId), assuming there is a suitable parameter defined to express Total column water in units of kg/cm2.
All data values can be set to a constant
The resulting file is small as it contains only the reference value. This is the method used to create the ifs_sample files used to provide 'template' GRIBs for model output.
Key names can be used to specify the output file names. This works in the same way as for grib_copy. Try:
grib_set is very powerful but it cannot be used to:
- make transformations to the data representation
- transform data from one grid representation to another
- select sub-areas of data
Resources and further reading
- The WMO FM 92 GRIB Manuals can be obtained from www.wmo.int/pages/prog/www/WMOCodes.html
- The ECMWF GRIB API manual is available at https://software.ecmwf.int/wiki/display/GRIB/Home
- ECMWF parameter database: http://apps.ecmwf.int/codes/grib/param-db/
- GRIB 1 keyshttp://apps.ecmwf.int/codes/grib/format/grib1/
- GRIB 2 keyshttp://apps.ecmwf.int/codes/grib/format/grib2/
- Edition independent keys http://apps.ecmwf.int/codes/grib/format/edition-independent/
- MARS specific keys http://apps.ecmwf.int/codes/grib/format/mars/
- The GRIB API Tools are documented at GRIB tools
- Details of the GRIB API Fortran 90 interface:Fortran package grib_api
- Example Python scripts and Fortran 90 and C programs can be found at GRIB API examples
- ECMWF GRIB API training course material: GRIB API: Library and Tools