...
range to pass the binomial test with a significance level of 95% (see e.g. http://www.fuellenbach-online.de/fh/pdf/binomialverteilung.pdf, pp. 2–6).
n | kmin,95% |
3 | 2 |
6 | 5 |
9 | 7 |
10 | 8 |
12 | 10 |
15 | 13 |
3.3 Concluding remarks
We drop the initially foreseen KPI stability for ICDR checks completely since the stability itself, as defined in Eq. (5), is already included in existing validation reports for brokered CDRs, and will be included in non-brokered datasets in the C3S validation documents. The main reason for doing so is that the ICDR periods are expected to be too short for detecting trends in noisy (or rather highly variable) data. The above approach for the KPI accuracy/bias will also enable us to detect a significant trend in the ICDR bias, because eventually the ICDR bias will deviate significantly from the CDR bias if it grows consistently smaller or larger over time. Consequently, we propose to investigate possible trends in the bias once the binomial test for KPI accuracy/bias indicates the need for action.
...
Anchor | ||||
---|---|---|---|---|
|
ECV name | Product name | KPI 1: accuracy | Explanations/Comments | KPI 2: stability | Explanations/Comments | |
Precipitation | ||||||
Precipitation | PRECIP_1DD_GPCP_TCDR | 0.3 mm/d | Numbers are given for global means, taken from CM SAF target and threshold requirements, achieved when comparing HOAPS and GPCP | 0.034 mm/d/dec | Taken from CM SAF threshold requirements, based on global averages, achieved when comparing HOAPS and GPCP | |
Precipitation | Global Precipitation Climatology Project (GPCP) v2.3 monthly PRECIP_2.5DM_GPCP_TCDR | 0.3 mm/d | 0.034 mm/d/dec | |||
Precipitation | 0.3 mm/d | 0.034 mm/d/dec | ||||
Precipitation | 0.3 mm/d | 0.034 mm/d/dec | ||||
Surface Radiation Budget |
|
|
|
| ||
Surface Radiation Budget | SIS_MM_AVHRR_CLARA | 10 W/m² | Numbers are given based on CM SAF Product Requirement Document PRD-2. Reference dataset is BSRN surface stations, parameter in the validation report - mean absolute bias (MAB) | 1.5 W/m²/dec | Numbers are given for trend in the bias vs. BSRN surface stations, based on CM SAF requirements | |
Surface Radiation Budget | SIS_DM_AVHRR_CLARA | 20 W/m² | 1.5 W/m²/dec | |||
Surface Radiation Budget | SDL_MM_AVHRR_CLARA | 10 W/m² | 0.5 W/m²/dec | |||
Surface Radiation Budget | SOL_MM_AVHRR_CLARA | 15 W/m² | 0.5 W/m²/dec | |||
Surface Radiation Budget | SRS_MM_AVHRR | None | None. Datasets are verified using the propagation of uncertainties method | None | None. Datasets are verified using the propagation of uncertainties method | |
Surface Radiation Budget | SNS_MM_AVHRR | None | None | |||
Surface Radiation Budget | SNL_MM_AVHRR | None | None | |||
Surface Radiation Budget | SRB_MM_AVHRR | None | None | |||
Water vapour |
|
|
|
| ||
Water Vapour | GRM-29-L3-H-R1 | 3% | According to current ROM SAF specs. Numbers are given for monthly means within5-degree latitude bins. | N / A | According to current ROM SAF specs.A stability criterion is planned to be intro-duced in the future. | |
Upper Tropospheric Humidity | UTH_MW | 5% | Set according to the GCOS Implementation Plan (GCOS-200, The Global Observing System for Climate: Implementation Needs (2016)). The validation is performed within +/-60 degrees latitude on a daily basis. | 1 %/dec | Adapted GCOS requirement. Verified for each satellite (mainly for NOAA-18 and MetOp-A which span ~10 years) within +/-60 degrees latitude. | |
| Water Vapour | TCWV_GV_TCDR | 1.4 kg/m² | Numbers are given for global means, taken from CM SAF target requirements (Table 5-2 Requirement Review 3.4 document HOAPS 5.0, Ohring etal 2005 & ESA DUE GlobVapour - Saunders etal 2010) | 0.2 kg/m²/dec | Taken from CM SAF target requirements defined in RR (Requirement Review 3.4 document HOAPS 5.0 Table 5-2), based on global averages (Ohring etal 2005 & ESA DUE GlobVapour - Saunders etal 2010) |
| Water Vapour | TCWV_SSMI/SSMIS_TCDR | 1 kg/m² | Numbers are given for global means, taken from CM SAF VR HOAPS 4.0 document (Table 6-6) of optimal requirements, achieved when comparing HOAPS 4.0 and REMSS | 0.2 kg/m²/dec | Taken from CM SAF VR HOAPS 4.0 document (Table 6-6) of target requirement, based on global averages, achieved when comparing HOAPS 4.0 and REMSS |
Cloud properties |
|
|
|
| ||
Cloud Properties | CFC_MM_AVHRR_CLARA | 5% | Numbers correspond to CM SAF target requirements (see EUMETSAT CM SAF CDOP-2 Product Requirements Document [SAF/CM/DWD/PRD, version 2.9], tables in Chapter 8 concerning requirements for CM-11011, CM11031, CM 11051, CM-11061). They refer to global means. | 2%/dec | Numbers correspond to CM SAF target requirements (see EUMETSAT CM SAF CDOP-2 Product Requirements Document [SAF/CM/DWD/PRD, version 2.9], tables in Chapter 8 concerning requirements for CM-11011, CM11031, CM 11051, CM-11061). They refer to global means. | |
Cloud Properties | CFC_DM_AVHRR_CLARA | 15% | 2%/dec | |||
Cloud Properties | CTO_MM_AVHRR_CLARA | 50 hPa | 20 hPa/dec | |||
Cloud Properties | CTO_DM_AVHRR_CLARA | 50 hPa | 20 hPa/dec | |||
Cloud Properties | LWP_MM_AVHRR_CLARA | 10 g/m² | 3 g/m²/dec | |||
Cloud Properties | LWP_DM_AVHRR_CLARA | None | 3 g/m²/dec | |||
Cloud Properties | IWP_MM_AVHRR_CLARA | 20 g/m² | 6 g/m²/dec | |||
Cloud Properties | IWP_DM_AVHRR_CLARA | None | 6 g/m²/dec | |||
TOA radiation |
|
|
|
| ||
Earth Radiation Budget | OLR_ERB_CERES | LW up: RMS 2.5 W/m² | Set according to results achieved in CERES_EBAF_Ed4.0 Data Quality Summary (January 12, 2018, available at https://ceres.larc.nasa.gov/documents/DQ_summaries/CERES_EBAF_Ed4.0_DQS.pdf ). | LW up: < 0.2 W/m²/dec | Set according to the GCOS Implementation Plan (GCOS-200, The Global Observing System for Climate: Implementation Needs (2016)). | |
Earth Radiation Budget | RSF_ERB_CERES | SW up: RMS 2.5 W/m² | SW up: < 0.3 W/m²/dec | |||
Earth Radiation Budget | OLR_HIRS | LW up: RMS 2.5 W/m² | LW up: < 0.2 W/m²/dec |
5. Application of the KPI evaluation approach to ICDRs
...
Table 3: Required KPI values for ICDRs Anchor table3 table3
| ECV name | Product name | KPI 1: accuracy | Explanations/ Comments | |
Precipitation | |||||
Precipitation | Global Precipitation Climatology Project (GPCP) v1.3 daily PRECIP_1DD_GPCP_ICDR | p2.5 = -0.314 mm/d | CDR percentiles when compared to TRMM TMPA 3B42 v7 inside +/-50° latitude. | ||
p2.5 = -0.528 mm/d | CDR percentiles when compared to ERA5 (global). These quantiles have been used for daily GPCP data since Jan. 2020. | ||||
Precipitation | Global Precipitation Climatology Project (GPCP) v2.3 monthly PRECIP_2.5DM_GPCP_ICDR | p2.5 = -0.169 mm/d, | CDR percentiles when compared to TRMM TMPA 3B43 v7 inside +/-50° latitude. | ||
p2.5 = -0.355 mm/d | CDR percentiles when compared to ERA5 (global). These quantiles have been used for monthly GPCP data since Jan. 2020. | ||||
Surface Radiation Budget |
|
| |||
Surface Radiation Budget | SIS_MM_AVHRR_CLARA | p2.5 = -17.4 W/m² | The CDR is compared to the surface measurements from the Baseline Surface Radiation Network (BSRN). The monthly averages are calculated from the high-resolution BSRN data. The collocations are selected using the nearest neighbour technique. Percentiles are calculated for the mean absolute bias. Satellite data with less than 20 observations per month are excluded. | ||
Surface Radiation Budget | SIS_DM_AVHRR_CLARA | p2.5 = -36.8 W/m² | |||
Surface Radiation Budget | SDL_MM_AVHRR_CLARA | p2.5 = -10.9 W/m² | |||
Surface Radiation Budget | SOL_MM_AVHRR_CLARA | p2.5 = -22.5 W/m² | |||
Surface Radiation Budget | SDL_MM_AVHRR | None | None. Datasets are verified using the propagation of uncertainties method. The dataset fulfills the KPIs if the input data fulfills the KPIs. | ||
Surface Radiation Budget | SOL_MM_AVHRR | None | |||
Surface Radiation Budget | SRS_MM_AVHRR | None | |||
Surface Radiation Budget | SNS_MM_AVHRR | None | |||
Surface Radiation Budget | SNL_MM_AVHRR | None | |||
Surface Radiation Budget | SRB_MM_AVHRR | None | |||
Water vapour |
|
| |||
Water Vapour | GRM-29-L3-H-I1 | 0–4 km: | p2.5 = -2.00% | The observed water vapour data are retrieved from Metop RO data, and the reference data consist of co-located ERA-Interim, similarly averaged as the observations.We compute relative differences between monthly means of observed data and reference data on a global latitude-height grid. These relative differences are globally averaged (properly area weighted) and vertically averaged (in 0-4 km, 4-8 km, and 8-12 km layers). No attempt is made to subtract an annual cycle in the observation-reference differences.For each vertical layer, we find the 2.5% and 97.5% percentiles of the distribution of the 121 months in the TCDR difference time series. | |
4–8 km: | p2.5 = -1.04% | ||||
8–12 km: | p2.5 = -0.66% | ||||
Upper Tropospheric Humidity | UTH_MW | p2.5 = -0.47%, | The values result from the comparison of the three UTH products from MetOp-A, MetOp-B, and NOAA-18 against ERA-Interim. The time periods considered are 2006-2015 for MetOp-A, 2008-2015 for MetOp-B, and 2013-2015 for NOAA-18. The values depict the (lowest of the three) 2.5 % and the (highest of the three) 97.5 % percentiles of the differences in daily global means. A cosine latitude weighted average was applied for the calculation of the means in each 1° x 1° grid box. The comparison excludes the polar regions. | ||
Cloud properties |
|
| |||
Cloud Properties | CFC_MM_AVHRR_CLARA | p2.5 = -0.60% | The values result from a comparison of the CLARA-A2 monthly CDR with the monthly TCDR of MODIS Collection 6.1 (time period considered: May 2003 to December 2015). They depict the 2.5 % and 97.5 % percentiles of the annual-cycle corrected differences in global means. Therefore at first, the time series of the differences in the global means between CLARA-A2 and MODIS was calculated. For the calculation of the global means, a cosine latitude weighted average was applied. Afterwards the mean annual cycle, i.e. the temporal mean of all Januaries, Februaries, etc., was subtracted. Finally, the 2.5 % and 97.5 % percentiles were calculated. Thus, to compare the new ICDR values to the declared percentile values, the mean annual cycle must be subtracted, too. For LWP and IWP the comparison excludes the polar regions (due to only part-time coverage during the year and challenging retrieval conditions over snow/ice). Only the monthly dataset is considered. As the monthly dataset originates from the daily dataset, one can assume that the results for the monthly dataset are also representative for the daily dataset. | ||
Cloud Properties | CFC_DM_AVHRR_CLARA | None | |||
Cloud Properties | CTO_MM_AVHRR_CLARA | p2.5 = -6.32.97 hPa | |||
Cloud Properties | CTO_DM_AVHRR_CLARA | None | |||
Cloud Properties | LWP_MM_AVHRR_CLARA | p2.5 = -1.97 g/m² | |||
Cloud Properties | LWP_DM_AVHRR_CLARA | None | |||
Cloud Properties | IWP_MM_AVHRR_CLARA | p2.5 = -1.74 g/m² | |||
Cloud Properties | IWP_DM_AVHRR_CLARA | None | |||
TOA radiation |
|
| |||
Earth Radiation Budget | RSF_ERB_CERES_FF | p2.5 = 0.4 W/m² | The values result from a comparison of the CERES monthly CDR with ERA 5 (time period considered: February 2002 to October 2018). They depict the 2.5 % and 97.5 % percentiles of the differences in global means. | ||
Earth Radiation Budget | OLR_ERB_CERES_FF | p2.5 = -2.4W/m² |
Anchorreferences references
References
references | |
references |
...