suppose you have two large GRIB files and their messages are not in order, how do you compare them? One technique is to use the "-r" key for grib_compare. This will compute the md5 hash value of each message (meta-data only) and sort both files (in memory) by that md5 value before doing the comparison. |
suppose you have two large GRIB files and their messages are not in order, how do you compare them? One technique is to use the "-r" key for grib_compare. This will compute the md5 hash value of each message (meta-data only) and sort both files (in memory) by that md5 value before doing the comparison.
Here is the version with "-r":
% grib_compare -r $input1 $input2 |
And now using grib_index_build with MARS keys:
#!/bin/sh grib_index_build -o idx1 $input1 >/dev/null grib_index_build -o idx2 $input2 >/dev/null grib_compare idx1 idx2 rm -f idx1 idx2 # Clean up |
By default grib_index_build uses the MARS keys (those in the "mars" namespace). Otherwise the user can pass his/her own desired keys via the "-k" option.
An even faster version of the 2nd script can be done if we launch the two grib_index_build processes in the background (run them concurrently):
#!/bin/sh grib_index_build -o idx1 $input1 >/dev/null & grib_index_build -o idx2 $input2 >/dev/null & wait # For the background tasks for finish grib_compare idx1 idx2 rm -f idx1 idx # Clean up |
Related articles appear here based on the labels you select. Click to edit the macro and add or change labels.