...
By default, Metview can only see files under the $HOME/metview
directory. You can select a different home directory for Metview (-u
option on the Metview startup command line), but you can also copy files into this directory (or a sub-directory of it) or create links to external data files - this can be useful if you have large data files which would exceed the quota in your home directory.
One way is to do this from a shell command line (ln
command), but Metview can also create these links for you.
...
Format overview
Geopoints is the ASCII format used by Metview to handle spatially irregular data (e.g. observations). There are a number of variations on the format, but the default one is a 6-column layout. The columns do not have to be aligned, but there must be at least one whitespace character between each entry.
This example shows a geopoints file containing dry bulb temperature at 2m (PARAMETER = 12004).
#GEO PARAMETER = 12004 lat long level date time value #DATA 36.15 -5.35 0 19970810 1200 300.9 34.58 32.98 0 19970810 1200 301.6 41.97 21.65 0 19970810 1200 299.4 45.03 7.73 0 19970810 1200 294 45.67 9.7 0 19970810 1200 302.2 44.43 9.93 0 19970810 1200 293.4 |
If you have observation data which you wish to import into Metview, Geopoints is probably the best format because:
- it is easy to write data into this format
- Metview has lots of functions to manipulate data in this format
Variants of the format allow 2-dimensional variables to be stored (e.g. U/V or speed/direction wind components), and another variant stores only lat, lon and value for a more compact file.
Examining geopoints
Examine the supplied geopoints.gpt icon to confirm the contents of the file. The columns are sortable. You may wish to open the file in an external text editor to see exactly what it looks like.
Visualising geopoints
Visualise the icon. The visdef used for geopoints is Symbol Plotting, and its default behaviour is to plot the actual numbers on the map. This can become cluttered, and text rendering can be slow. Create a new Symbol Plotting icon and set the following parameters:
Legend | On |
Symbol Type | Marker |
Symbol Table Mode | Advanced |
Symbol Advanced Table Max Level Colour | Red |
Symbol Advanced Table Min Level Colour | Blue |
Symbol Advanced Table Max Colour Direction | Clockwise |
Rename the icon to symb_auto and drop it into the Display Window to see the points coloured according to their value.
Computing some statistics in Macro
First, we will print some information about our geopoints data. Create a new Macro icon, type this code and run it:
gp = read( 'geopoints.gpt' ) print ( 'Num points: ' , count(gp)) print ( 'Min value: ' , minvalue(gp)) print ( 'Max value: ' , maxvalue(gp)) |
Perform a simple data manipulation and return the result to Metview's user interface:
return gp * 100 |
Save the macro and see its result by right-clicking on its icon and choosing examine or visualise. We could also have put a write()
command into the macro to write the result to a geopoints file.
Finding geopoints points within 100km of a given location
As a more complex example, we will combine two functions in order to find the locations of the points within a certain distance of a given location. We will use the same geopoints file as before.
The distance()
function returns a new geopoints variable based on its input geopoints, where each point's value has been replaced by the distance of that point from the given location. The description of this function follows:
Panel |
---|
Returns geopoints with the value of each point being the distance in metres from the given geographical location. The location may be specified by supplying either two numbers (latitude and longitude respectively) or a 2-element list containing latitude and longitude in that order. The location should be specified in degrees. |
Choose a location and use this function to compute the distances of the data points from it. Assign the result to a variable called distances
and return it to the user interface to examine the numbers. The distances are in metres.
Now we will see a boolean operator in action. The expression distances < 100000
(one hundred thousand) will return a new geopoints variable where, for each point, if the input value was less than 100000, the resulting value will be 1; otherwise the resulting value will be zero. So the resulting geopoints will have a collection of ones and zeros. Confirm that this is the case.
The filter()
function, from the documentation:
Panel |
---|
A filter function to extract a subset of its geopoints input using a second geopoints as criteria. The two input geopoints must have the same number of values. The resulting output geopoints contains the values of the first geopoints where the value of the second geopoints is non-zero. It is usefully employed in conjunction with the comparison operators :
The variable |
Use this in combination with what you have already done to produce a geopoints variable consisting only of the points within 100km of your chosen location. Plot the result to confirm it.
Saving geopoints data
Geopoints variables can be saved to disk using the write()
command:
write( 'my_computed_data.gpt' , points) |
It is also possible to convert between geopoints and GRIB
This format - this will be covered in Processing Data.
Observation Data in BUFR files
Much observation data is received in BUFR format. BUFR is a complex format, capable of storing almost anything; BUFR files can vary widely, but there are some conventions which can help software to interpret them. We will have a brief overview of Metview's BUFR-handling capabilities here; for more information, see the dedicated tutorial on the Tutorials page.
Examining BUFR Meta-data
Right-click on the supplied synop.bufr BUFR icon and select examine from the icon menu. This will start the BUFR examiner application. The right-hand panel displays data for the message selected in the left-hand panel. This can be an easy way to find the correct descriptor for a given parameter such as Relative Humidity.
Plotting BUFR Data
Metview is able to plot certain BUFR data directly, mainly some WMO conventional observation types including SYNOP and TEMP.
Right-click and visualise the synop.bufr BUFR icon. This will bring up the Display Window using the default visualisation assigned to observation plotting. What we see here is a spatially thinned set of SYNOP observations plotted on the map by using the official WMO-style. If you zoom into a smaller area you will see more observations but the thinning is still kept so that the plot should not seem cluttered.
Filtering Observation Data
BUFR files can contain a lot of information, but we often want to extract just one or two parameters.
The Observation Filter icon extracts a single scalar or vector value from each message in a BUFR file. It is able to perform filtering according to message type, date, time, level, area, location and custom descriptors. Examine the BUFR file and find the descriptor for wind speed at 10m (look in the blue right-hand panel) - make a note of it.
Create a new Observation Filter icon, rename it to wind_speed and edit it. Drop the BUFR icon into the Data field and set the following to extract the wind speed values in geopoints format:
Output | Geographical Points |
Parameter | 11012 |
Visualise the icon - the filtering will take place, then the result is plotted using the default Symbol Plotting definition, which is to plot the data as numbers. Drop your symb_auto icon into the Display Window for a nicer plot.
Notice that there is a point which claims a wind speed of 80m/s! Reliability can be a big issue with observational data, and this point claims winds of 288km/h! We can filter out data that we consider unrealistic - add the following parameters to your wind_speed icon:
Custom Filter | Filter By Range |
Custom Parameter | 11012 |
Custom Values | 0/50 |
This ensures that we only extract points whose wind speed is between 0 and 50 (m/s). Having a smaller range of values also allows the automatic colour range to spread more evenly through the data. There is still a point with a large value, which you can also filter out if desired.
Notice that the values in the colour scale change as you zoom in and out of different areas - this is computed according to the data currently visible. Try the supplied icon symb_wind_speed_fixed, which has a fixed value/colour mapping.
Extracting vector values from BUFR
We can extract the wind direction too, and plot the wind as arrows (or flags).
Make a copy of your wind_speed filter icon and call it wind_speed_and_direction. Find out which descriptor provides wind direction, then change the following parameters in your new filter icon:
Output | Geographical Polar Vectors |
Parameter | 11012/????? |
where ????? is replaced by the wind direction descriptor number.
When you visualise the icon, you will see numbers as before, but if you drop a newly created Wind Plotting icon into the Display Window you will see wind arrows. Try the supplied coloured_wind_arrows icon too. Try changing it to plot wind flags instead of arrows.
You may wish to customise a Coastlines icon to provide a darker background for the plot.
Extra Tasks
Write a macro to plot the wind arrows from BUFR
Use the icons you created to filter and plot the wind arrows from BUFR data to write a macro which produces the same plot. Extract the 'magic numbers' such as the filtering threshold and the wind parameter descriptors into variables at the top of the macro, and use these variables in the macro rather than the raw numbers.
Investigate different grids
GRIB fields are often not as simple as regular lat/lon! ECMWF also produces data in "reduced Gaussian grids", two of which are included in your folder. Visualise them with your grid_1x1 icon to see how the points are spaced around the globe. Use both a cylindrical and a polar stereographic projection to look at them (Geographic View icon).
Try the search facilities in the data examiners
Examine the GRIB file and the BUFR file; press CTRL-F to initiate the search. Look carefully at the options!
Create your own GRIB Examiner key profile
When you examine a GRIB file, a list of 'keys' is used to display the GRIB messages - one key per column. These columns are configurable - a 'key profile' is a set of keys, and you can create as many of them as you want. It can be very useful to have different key profiles for different tasks. From the user interface in the GRIB Examiner, create a new key profile; starting either from scratch, or else from a duplicate of the default profile. Note that the Display Window also operates on the same principles, and you can share key profiles between the two.
Observation filtering
Extract 2m temperature values below the freezing point from synop.bufr.
Hints:
- use geopoints output
- use custom filter
- temperature values are given in K