Versions Compared

Old Version 2

changes.mady.by.user Martin Janousek

Saved on

compared with

New Version Current

changes.mady.by.user Martin Janousek

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

At the WMO Commission for Basic Systems (CBS) ET-OWFPS Meeting on 22-24 October 2014 in Geneva, a draft document on standardized surface verification of deterministic NWP products was agreed. The document specifies the content of the future exchange of surface scores between global centres and is available here: Standardized Surface Verification for Deterministic NWP.

ECMWF will be collecting and archiving the scores. Data will be exchanged in an ASCII-based format which is simple but flexible and is similar to the format used WMO exchange of upper-air scores.

Data format

Data file consists of score values and corresponding metadata in an ASCII format.

...

A record has the following format:

centre=centre, model=model_id, d=yyyymm, t=time, s=forecast_step, st=station_id, lat=latitude, lon=longitude, lam=model_grid_latitude, lom=model_grid_longitude, se=station_elevation, me=model_orography_elevation, par=parameter, sc=score, th=event_thresholds, n=sample_size, v=
mean
score_value

If the value of any key is unknown it is encoded as "na". However, every record has to have a valid meanscore_value (the "v" key); if the value is not known such record should not be included in the data file.

Example

Tip

For information on how these reports are assembled at ECMWF please refer to ECMWF implementation notes.

Code Block
centre=ecmf,model=hr_0001,d=201602,t=000,s=0,st=97146,lat=-4.1,lon=122.43,lam=-4.147,lom=122.484,se=50,me=163,par=tcc,sc=ct,th=2/6,v=0/0/0/0/0/7/0/0/21
t=3,s=3,v=0/0/0/0/0/4/0/0/24
t=6,s=6,v=0/0/0/0/0/2/0/0/26
t=12,s=12,v=0/0/0/0/0/6/0/0/22
t=15,s=15,v=0/0/0/0/0/3/0/0/25
t=18,s=18,v=0/0/0/0/0/4/0/0/24
t=0,s=00,24,v=0/0/0/0/0/5/0/0/23
s=0,sc=mae,th=na,v=60.92
t=3,s=3,v=59.92
t=6,s=6,v=62.01
t=12,s=12,v=60.59
t=15,s=15,v=59.81
t=18,s=18,v=62.08
t=0,s=00,24,v=65.01
s=0,sc=me,v=-60.92
t=3,s=3,v=-59.92
t=6,s=6,v=-62.01
t=9,s=9,n=26,v=-66.37
...

Values of keys

  • centre (4-characters string) is the WMO identifier of the originating centre (ammc, cwao, ecmf, edzw, egrr, kwbc, lfpw, rjtd, rksl, rums etc);

  • model_id (a string not containing a comma or vertical bar) is free model identifier assigned by the originating centre (to distinguish between potentially different models provided by the centre);

  • yyyymm is the month of the mean, where yyyy is the year and mm is the month (01-12);
  • time is the validity time (in hours UTC) of the forecasts verified;
  • forecast_step is the length of the forecast (in hours);
  • station_id (a number) is the WMO ID of the observation station verifying the forecasts;
  • latitude is the latitude of the observation station verifying the forecasts;
  • longitude is the longitude of the observation station verifying the forecasts;
  • model_grid_latitude is the latitude of the model grid point used to extract the forecast at the observation location (*);
  • model_grid_longitude is the longitude of the model grid point used to extract the forecast at the observation location (*);
  • station_elevation is the elevation of the observation station above the mean sea level in meters;
  • model_orography_elevation is the elevation of the model orography at the observation location ; if the model orography changed during the month (due to a model upgrade etc) the value corresponding to the orography of the last forecast in the month should be given here (*);
  • parameter is the verified model output parameter:

...

  • Include Page

...

  • List of parameters
    List of parameters

...

  • score is the name of the verification score or statistic:
scoredescription 
me

mean error (bias)

 
maemean absolute error 
rmseroot mean square error 
ct

contingency table values,
scanned from bottom up, then left to right

The rank of a contingency table is defined by number of values in the key event_thresholds
(rank_of_ct = len(event_thresholds)+1)

 

NB. Please note how those contingency tables are constructed.
For an event like "10m wind speed >15m/s" a corresponding contingency
table would traditionally be

 obsnon-obs
fcHF
non-fcMR

but following the definition of 2x2 contingency table on the left the table should be

 ob<1515≤ob
fc<15RM
15≤fcFH
2x2ob<th1th1≤ob
fc<th1AB
th1≤fcCD

v=C/A/D/B

3x3ob<th1th1≤ob<th2th2≤ob
ob<th1ABC
th1≤ob<th2DEF
th2≤obGHI

v=G/D/A/H/E/B/I/F/C

4x4ob<th1th1≤ob<th2th2≤ob<th3th3≤ob
ob<th1ABCD
th1≤ob<th2EFGH
th2≤ob<th3IJKL
th3≤obMNOP

v=M/I/E/A/N/J/F/B/O/K/G/C/P/L/H/D

 

  • event_thresholds is a value or values of threshold(s) defining events for contingency tables; number of values in event_thresholds defines the rank of the contingency table; multiple values are separated by a forward slash /
event_thresholddescription
15threshold for a 2x2 contingency table, e.g. if par=ff10m this is an event of 10m wind speed
5/10/15thresholds for a 4x4 contingency table for 10m wind speed

 

  • sample_size is number of observations used to compute the monthly mean at the given station;
  • meanscore_value is the value or values of the score mean computed based on the forecast initiating at time time UTC, verifying in the month yyyymm, for the forecast length forecast_step hours; in case of nxn contingency table these are the n2 values delimited by forward slashes (see the score table above); if possible, the value should be printed using the printf string format specifier "g" (or equivalent)

Procedures for exchange of data

  • .

Remarks

Anchor
model_change
model_change
(*) If the model grid or orography changed during the reported month (due to a model upgrade etc) the values of lam, lom and me should be those of the latest model runComing soon...