Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

range to pass the binomial test with a significance level of 95% (see e.g. http://www.fuellenbach-online.de/fh/pdf/binomialverteilung.pdf, pp. 26).


n

kmin,95%

3

2

6

5

9

7

10

8

12

10

15

13

3.3 Concluding remarks

We drop the initially foreseen KPI stability for ICDR checks completely since the stability itself, as defined in Eq. (5), is already included in existing validation reports for brokered CDRs, and will be included in non-brokered datasets in the C3S validation documents. The main reason for doing so is that the ICDR periods are expected to be too short for detecting trends in noisy (or rather highly variable) data. The above approach for the KPI accuracy/bias will also enable us to detect a significant trend in the ICDR bias, because eventually the ICDR bias will deviate significantly from the CDR bias if it grows consistently smaller or larger over time. Consequently, we propose to investigate possible trends in the bias once the binomial test for KPI accuracy/bias indicates the need for action.

...

Anchor
table2
table2
Table 2: Required KPI values for CDRs


ECV name

Product name

KPI 1: accuracy

Explanations/Comments

KPI 2: stability

Explanations/Comments

Precipitation








Precipitation

Global Precipitation Climatology Project (GPCP) v1.3 daily

PRECIP_1DD_GPCP_TCDR

0.3 mm/d

Numbers are given for global means, taken from CM SAF target and threshold requirements, achieved when comparing HOAPS and GPCP

0.034 mm/d/dec

Taken from CM SAF threshold requirements, based on global averages, achieved when comparing HOAPS and GPCP


Precipitation

Global Precipitation Climatology Project (GPCP) v2.3 monthly

PRECIP_2.5DM_GPCP_TCDR

0.3 mm/d


0.034 mm/d/dec



Precipitation

Copernicus micrOwave-based gloBal pRecipitAtion (COBRA) daily

PRECIP_1DD_MW_TCDR

0.3 mm/d
0.034 mm/d/dec

Precipitation

Copernicus micrOwave-based gloBal pRecipitAtion (COBRA) monthly

PRECIP_1DM_MW_TCDR

0.3 mm/d
0.034 mm/d/dec

Surface Radiation Budget



 

 

 

 


Surface Radiation Budget

SIS_MM_AVHRR_CLARA

10 W/m²

Numbers are given based on CM SAF Product Requirement Document PRD-2. Reference dataset is BSRN surface stations, parameter in the validation report - mean absolute bias (MAB)

1.5 W/m²/dec

Numbers are given for trend in the bias vs. BSRN surface stations, based on CM SAF requirements


Surface Radiation Budget

SIS_DM_AVHRR_CLARA

20 W/m²


1.5 W/m²/dec



Surface Radiation Budget

SDL_MM_AVHRR_CLARA

10 W/m²


0.5 W/m²/dec



Surface Radiation Budget

SOL_MM_AVHRR_CLARA

15 W/m²


0.5 W/m²/dec



Surface Radiation Budget

SRS_MM_AVHRR

None

None. Datasets are verified using the propagation of uncertainties method

None

None. Datasets are verified using the propagation of uncertainties method


Surface Radiation Budget

SNS_MM_AVHRR

None


None



Surface Radiation Budget

SNL_MM_AVHRR

None


None



Surface Radiation Budget

SRB_MM_AVHRR

None


None


Water vapour



 

 

 

 


Water Vapour

GRM-29-L3-H-R1

3%

According to current ROM SAF specs. Numbers are given for monthly means within5-degree latitude bins.

N / A

According to current ROM SAF specs.A stability criterion is planned to be intro-duced in the future.


Upper Tropospheric Humidity

UTH_MW

5%

Set according to the GCOS Implementation Plan (GCOS-200, The Global Observing System for Climate: Implementation Needs (2016)). The validation is performed within +/-60 degrees latitude on a daily basis.

1 %/dec

Adapted GCOS requirement. Verified for each satellite (mainly for NOAA-18 and MetOp-A which span ~10 years) within +/-60 degrees latitude.

 

Water Vapour

TCWV_GV_TCDR

1.4 kg/m²

Numbers are given for global means, taken from CM SAF target requirements (Table 5-2 Requirement Review 3.4 document HOAPS 5.0, Ohring etal 2005 & ESA DUE GlobVapour - Saunders etal 2010)

0.2 kg/m²/dec

Taken from CM SAF target requirements defined in RR (Requirement Review 3.4 document HOAPS 5.0 Table 5-2), based on global averages (Ohring etal 2005 & ESA DUE GlobVapour - Saunders etal 2010)

 

Water Vapour

TCWV_SSMI/SSMIS_TCDR

1 kg/m²

Numbers are given for global means, taken from CM SAF VR HOAPS 4.0 document (Table 6-6) of optimal requirements, achieved when comparing HOAPS 4.0 and REMSS

0.2 kg/m²/dec

Taken from CM SAF VR HOAPS 4.0 document (Table 6-6) of target requirement, based on global averages, achieved when comparing HOAPS 4.0 and REMSS

Cloud properties



 

 

 

 


Cloud Properties


CFC_MM_AVHRR_CLARA

5%

Numbers correspond to CM SAF target requirements (see EUMETSAT CM SAF CDOP-2 Product Requirements Document [SAF/CM/DWD/PRD, version 2.9], tables in Chapter 8 concerning requirements for CM-11011, CM11031, CM 11051, CM-11061). They refer to global means.

2%/dec

Numbers correspond to CM SAF target requirements (see EUMETSAT CM SAF CDOP-2 Product Requirements Document [SAF/CM/DWD/PRD, version 2.9], tables in Chapter 8 concerning requirements for CM-11011, CM11031, CM 11051, CM-11061). They refer to global means.


Cloud Properties

CFC_DM_AVHRR_CLARA

15%


2%/dec



Cloud Properties

CTO_MM_AVHRR_CLARA

50 hPa


20 hPa/dec



Cloud Properties

CTO_DM_AVHRR_CLARA

50 hPa


20 hPa/dec



Cloud Properties

LWP_MM_AVHRR_CLARA

10 g/m²


3 g/m²/dec



Cloud Properties

LWP_DM_AVHRR_CLARA

None


3 g/m²/dec



Cloud Properties

IWP_MM_AVHRR_CLARA

20 g/m²


6 g/m²/dec



Cloud Properties

IWP_DM_AVHRR_CLARA

None


6 g/m²/dec


TOA radiation



 

 

 

 


Earth Radiation Budget

OLR_ERB_CERES

LW up: RMS 2.5 W/m²

Set according to results achieved in CERES_EBAF_Ed4.0 Data Quality Summary (January 12, 2018, available at https://ceres.larc.nasa.gov/documents/DQ_summaries/CERES_EBAF_Ed4.0_DQS.pdf ).

LW up: < 0.2 W/m²/dec

Set according to the GCOS Implementation Plan (GCOS-200, The Global Observing System for Climate: Implementation Needs (2016)).


Earth Radiation Budget

RSF_ERB_CERES

SW up: RMS 2.5 W/m²


SW up: < 0.3 W/m²/dec



Earth Radiation Budget

OLR_HIRS

LW up: RMS 2.5 W/m²


LW up: < 0.2 W/m²/dec


5. Application of the KPI evaluation approach to ICDRs

...

Anchor
table3
table3
Table 3: Required KPI values for ICDRs

 

ECV name

Product name

KPI 1: accuracy

Explanations/ Comments

Precipitation






Precipitation

Global Precipitation Climatology Project (GPCP) v1.3 daily

PRECIP_1DD_GPCP_ICDR


p2.5 = -0.314 mm/d
p97.5 = 0.341 mm/d
(constrained to +/-50° latitude)

CDR percentiles when compared to TRMM TMPA 3B42 v7 inside +/-50° latitude.


p2.5 = -0.528 mm/d
p97.5 = +0.013 mm/d

CDR percentiles when compared to ERA5 (global). These quantiles have been used for daily GPCP data since Jan. 2020.


Precipitation

Global Precipitation Climatology Project (GPCP) v2.3 monthly

PRECIP_2.5DM_GPCP_ICDR


p2.5 = -0.169 mm/d,
p97.5 = 0.185 mm/d
(constrained to +/-50° latitude)

CDR percentiles when compared to TRMM TMPA 3B43 v7 inside +/-50° latitude.


p2.5 = -0.355 mm/d
p97.5 = -0.073 mm/d

CDR percentiles when compared to ERA5 (global). These quantiles have been used for monthly GPCP data since Jan. 2020.

Surface Radiation Budget



 

 


Surface Radiation Budget

SIS_MM_AVHRR_CLARA


p2.5 = -17.4 W/m²
p97.5 = 35 W/m²

The CDR is compared to the surface measurements from the Baseline Surface Radiation Network (BSRN). The monthly averages are calculated from the high-resolution BSRN data. The collocations are selected using the nearest neighbour technique. Percentiles are calculated for the mean absolute bias. Satellite data with less than 20 observations per month are excluded.


Surface Radiation Budget

SIS_DM_AVHRR_CLARA


p2.5 = -36.8 W/m²
p97.5 = 74 W/m²



Surface Radiation Budget

SDL_MM_AVHRR_CLARA


p2.5 = -10.9 W/m²
p97.5 = 26.7 W/m²



Surface Radiation Budget

SOL_MM_AVHRR_CLARA


p2.5 = -22.5 W/m²
p97.5 = 49.9 W/m²



Surface Radiation Budget

SDL_MM_AVHRR


None

None. Datasets are verified using the propagation of uncertainties method. The dataset fulfills the KPIs if the input data fulfills the KPIs.


Surface Radiation Budget

SOL_MM_AVHRR


None



Surface Radiation Budget

SRS_MM_AVHRR


None



Surface Radiation Budget

SNS_MM_AVHRR


None



Surface Radiation Budget

SNL_MM_AVHRR


None



Surface Radiation Budget

SRB_MM_AVHRR


None


Water vapour



 

 


Water Vapour

GRM-29-L3-H-I1

04 km: 

p2.5 = -2.00% 
p97.5 = -0.93%

The observed water vapour data are retrieved from Metop RO data, and the reference data consist of co-located ERA-Interim, similarly averaged as the observations.We compute relative differences between monthly means of observed data and reference data on a global latitude-height grid. These relative differences are globally averaged (properly area weighted) and vertically averaged (in 0-4 km, 4-8 km, and 8-12 km layers). No attempt is made to subtract an annual cycle in the observation-reference differences.For each vertical layer, we find the 2.5% and 97.5% percentiles of the distribution of the 121 months in the TCDR difference time series.

48 km:

p2.5 = -1.04%
p97.5 = +0.82%

812 km:

p2.5 = -0.66%
p97.5 = +2.13%


Upper Tropospheric Humidity

UTH_MW


p2.5 = -0.47%,
p97.5 = 0.55%
(constrained to +/-60° latitude)

The values result from the comparison of the three UTH products from MetOp-A, MetOp-B, and NOAA-18 against ERA-Interim. The time periods considered are 2006-2015 for MetOp-A, 2008-2015 for MetOp-B, and 2013-2015 for NOAA-18. The values depict the (lowest of the three) 2.5 % and the (highest of the three) 97.5 % percentiles of the differences in daily global means. A cosine latitude weighted average was applied for the calculation of the means in each 1° x 1° grid box. The comparison excludes the polar regions.

Cloud properties



 

 


Cloud Properties

CFC_MM_AVHRR_CLARA


p2.5 = -0.60%
p97.5 = 0.48%

The values result from a comparison of the CLARA-A2 monthly CDR with the monthly TCDR of MODIS Collection 6.1 (time period considered: May 2003 to December 2015). They depict the 2.5 % and 97.5 % percentiles of the annual-cycle corrected differences in global means. Therefore at first, the time series of the differences in the global means between CLARA-A2 and MODIS was calculated. For the calculation of the global means, a cosine latitude weighted average was applied. Afterwards the mean annual cycle, i.e. the temporal mean of all Januaries, Februaries, etc., was subtracted. Finally, the 2.5 % and 97.5 % percentiles were calculated. Thus, to compare the new ICDR values to the declared percentile values, the mean annual cycle must be subtracted, too. For LWP and IWP the comparison excludes the polar regions (due to only part-time coverage during the year and challenging retrieval conditions over snow/ice). Only the monthly dataset is considered. As the monthly dataset originates from the daily dataset, one can assume that the results for the monthly dataset are also representative for the daily dataset.


Cloud Properties

CFC_DM_AVHRR_CLARA


None



Cloud Properties

CTO_MM_AVHRR_CLARA


p2.5 = -6.32.97 hPa
p97.5 = 3.74 hPa



Cloud Properties

CTO_DM_AVHRR_CLARA


None



Cloud Properties

LWP_MM_AVHRR_CLARA


p2.5 = -1.97 g/m²
p97.5 = 2.23 g/m²
(constrained to +/- 50° latitude)



Cloud Properties

LWP_DM_AVHRR_CLARA


None



Cloud Properties

IWP_MM_AVHRR_CLARA


p2.5 = -1.74 g/m²
p97.5 = 1.91 g/m²
(constrained to +/- 50° latitude)



Cloud Properties

IWP_DM_AVHRR_CLARA


None


TOA radiation



 

 


Earth Radiation Budget

RSF_ERB_CERES_FF


p2.5 = 0.4 W/m²
p97.5 = 2.3 W/m²

The values result from a comparison of the CERES monthly CDR with ERA 5 (time period considered: February 2002 to October 2018). They depict the 2.5 % and 97.5 % percentiles of the differences in global means.


Earth Radiation Budget

OLR_ERB_CERES_FF


p2.5 = -2.4W/m²
p97.5 = -1.3 W/m²


Anchor
references
references
References

...