Contributors: N. Clerbaux (Royal Meteorological Institute of Belgium (RMIB)), A. Velazquez Blazquez (RMIB), E. Baudrez (RMIB), S. Dewitte (RMIB), S. Nevens (RMIB)
Issued by: RMIB
Date: 18/12/2020
Ref: C3S_D312b_Lot1.2.5.9-v2.1_202101_PQAR_ECVEarthRadiationBudget_v1.0
Official reference number service contract: 2018/C3S_312b_Lot1_DWD/SC1
History of modifications
List of datasets covered by this document
Related documents
Acronyms
List of tables
List of figures
Scope of the document
This Product Quality Assessment Report (PQAR) provides validation results for the second version of the Climate Data Records (TCDR and ICDR) of the Total Solar Irradiance (TSI) part of the Essential Climate Variable (ECV) Earth Radiation Budget (ERB). These records are generated in the frame of C3S 312b-lot1 project as a composite of various TSI records.
The scope of this PQAR document is limited to the presentation of the validation results for the TCDR and ICDR. The methodology is detailed in the Product Quality Assurance Document (PQAD, [D6]).
Executive summary
The Total Solar Irradiance (TSI) quantifies the amount of solar energy that is received by the Earth. The TSI is defined as the amount of solar power that reaches the Earth per unit surface perpendicular to the Sun–Earth direction, reduced to the mean Earth-Sun distance (1 Astronomical unit).
The Total Solar Irradiance (TSI) is a fundamental variable governing the climate system, and is recognized as an ECV by the GCOS. Within the C3S, a long composite TSI record is constructed from measurements of an ensemble of space instruments. The measurements of the individual instruments are first put on a common radiometric scale, and their quality is assessed by intercomparison. Then, the composite time series is constructed as the average of the available measurements, on a daily basis. The full processing is described in the ATBD [D3] and also in two journal papers [D1] and [D2].
In its version 2.0, the C3S daily TSI composite covers the time period from 1st January 1979 to 31st December 2018. The record is regularly updated as ICDR with increasing version number v2.x. This PQAR document includes ICDR data v2.2 up to 31st of October 2020.
The TSI timeseries can be accessed via the Copernicus Climate Data Store (CDS) at https://cds.climate.copernicus.eu.
1. Product validation methodology
There are no reference observations that can be used for a direct validation of the Total Solar Irradiance (TSI). Instead, the accuracy is estimated as an intercomparison with TSI records derived by other teams based, when possible, on other input data.
The validation methodology is fully described in the Product Quality Assurance Document [D6].
In short, the validation involves :
- the evaluation of the individual instrument timeseries with the composite CDR,
- the intercomparison of the composite CDR with the NOAA NRL TSI v2 daily TSI CDR (Coddington et al, 2015),
- the intercomparison of the composite CDR with the SATIRE-S (Yeo et al, 2014a and 2014b) record (available at http://www2.mps.mpg.de/projects/sun-climate/data.html)
2. Validation results
2.1 Individual timeseries evaluations
A total of 12 different TSI instruments have been compiled to create the C3S TCDR/ICDR as detailed in the ATBD [D3]. To put them on the same radiometric level, a scaling factor is applied for each instrument as detailed in [D6]. The resulting timeseries are said to be “adjusted”.
In Figure 1, Figure 2 and Figure 3, each of the adjusted individual timeseries (described in [D3]) is evaluated by comparison with the C3S and the NRL TSI v2 composites. On these Figures, the red parts of the timeseries are the parts that have been discarded according to Dewitte and Nevens (2016) [D1]. Once the red parts are discarded, the 12 instruments show close agreement with the 2 composites. Table 2-1 provides the bias, the RMS and the bias-corrected RMS (bc-RMS) between the adjusted individual timeseries and the resulting C3S composite. The last column indicates if the data are used in the TCDR v2.0 and/or the ICDR v2.x. The (adjusted) instrument bias remains below 0.1 W/m² and the bc-RMS improves in time from about 0.17 W/m² down to about 0.05 W/m² with the recent TIM instruments. As an exception, the short SOVAP timeseries (used over 19 Nov. 2010 – 3 Nov. 2013) exhibits higher variability (bc-RMS of 0.190 W/m²).
Table 2-1: Evaluation of the adjusted individual timeseries with respect to C3S daily TSI composite.
Instrument | Adjustment | Number of daily values | Bias | RMS | bc-RMS | Used in TCDR v2.0 (T) and/or ICDR v2.x (I) |
ERB | 0.993204 | 3287 | 0.042 | 0.184 | 0.179 | T |
ACRIM1 | 0.996232 | 3374 | -0.042 | 0.176 | 0.171 | T |
ERBS | 0.997864 | 5775 | -0.035 | 0.098 | 0.092 | T |
ACRIM2 | 0.998587 | 3502 | 0.041 | 0.141 | 0.135 | T |
DIARAD/VIRGO | 0.997241 | 8370 | -0.029 | 0.095 | 0.091 | T+I |
PMO06/VIRGO | 0. 997609 | 7566 | -0.021 | 0.090 | 0.087 | T |
ACRIM3 | 1.000938 | 3900 | 0.045 | 0.090 | 0.079 | T |
TIM/SORCE | 1.001216 | 6000 | -0.001 | 0.041 | 0.040 | T+I |
PREMOS | 1.001085 | 1120 | 0.010 | 0.118 | 0.117 | T |
SOVAP | 1.000518 | 1165 | 0.075 | 0.203 | 0.189 | T |
TIM/TCTE | 1.000633 | 1477 | 0.085 | 0.102 | 0.057 | T+I |
TIM/TSIS1 | 1.000450 | 1066 | 0.075 | 0.093 | 0.056 | T+I |
Figure 2-1: Evaluation of the individual daily TSI timeseries for ERB, ACRIM-1, ERBS and ACRIM-2. The green curves show the data used in the C3S composite while the red parts show data discarded from the C3S composite. The blue and orange curves show the 'running mean' (121 days time period) of the individual instruments, respectively for the period kept/discarded from the composite. Black and brown curves show the C3S and NRL TSI v2 composites after '121 days running mean'.
Figure 2-2: Evaluation of the individual daily TSI timeseries for DIARAD/VIRGO, PMO06/VIRGO, ACRIM3 and TIM/SORCE. The green curves show the data used in the C3S composite while the red parts show data discarded from the C3S composite. The blue and orange curves show the 'running mean' (121 days time period) of the individual instruments, respectively for the period kept/discarded from the composite. Black and brown curves show the C3S and NRL TSI v2 composites after '121 days running mean'.
Figure 2-3: Evaluation of the individual daily TSI timeseries for PREMOS,SOVAP, TIM/TCTE, and TIM/TSIS1. The green curves show the data used in the C3S composite while the red parts show data discarded from the C3S composite. The blue and orange curves show the 'running mean' (121 days time period) of the individual instruments, respectively for the period kept/discarded from the composite. Black and brown curves show the C3S and NRL TSI v2 composites after '121 days running mean'.
2.2 TCDR and ICDR comparison with NRL TSI v2 composite
Figure 4 shows the comparison of the C3S and NRL TSI v2 (see PQAD [D6]) daily TSI composites. The two composites are in close agreement, although the period before 1984 shows a higher level of daily TSI difference.
Table 2-2 provides the bias, the RMS and the bias-corrected RMS (bc-RMS) between the two timeseries. A significant bias of 1.4 W/m² exists. Once corrected for the bias, the daily TSI values are in close agreement, with bc-RMS of only 0.253 W/m² over the TCDR period. Over the ICDR period, the bc-RMS goes down to 0.061 W/m². The better consistency during the ICDR period could be attributed to the lower variability of the TSI at the solar minimum.
Table 2-2: Bias and RMS difference between the C3S and NRL v2 daily TSI composites.
CDR | Number of daily values | Bias | RMS | bc-RMS |
TCDR | 14610 | 1.403 | 1.426 | 0.253 |
ICDR | 639 | 1.411 | 1.412 | 0.061 |
Figure 2-4: Top: C3S daily TSI composite (green) and NRL TSI v2 composite (red). Bottom : difference between the C3S and NRL TSI v2 daily TSI composites (green). The 121-days running mean curves are also shown (black).
2.3 TCDR and ICDR comparison with SATIRE-S
Figure 5 shows the comparison of the C3S and SATIRE-S reconstruction (see PQAD [D6]) daily TSI composites. The 2 composites are in close agreement. At the very beginning of the TCDR, before 16 Feb. 1980, the TCDR is mostly based on SATIRE which makes the comparison not relevant. This period has been discarded in the present analysis.
Table 2-3 provides the bias, the RMS and the bias-corrected RMS (bc-RMS) between the 2 timeseries. The bias shows more temporal variability than in the comparison of C3S and NRL TSI-v2. A significant bias of 1.3 W/m² exists. Once corrected for the bias, the daily TSI values are in close agreement, with bc-RMS of 0.287 W/m² over the TCDR period. Over the ICDR period, the bc-RMS decreases to 0.052 W/m². This improvement could be attributed to the lower variability of the TSI at the solar minimum.
Table 2-3: Bias and RMS difference between the C3S and SATIRE-S composites.
CDR | Number of daily values | Bias | RMS | bc-RMS |
TCDR | 14199 | 1.340 | 1.371 | 0.287 |
ICDR | 502 | 1.546 | 1.547 | 0.052 |
Figure 2-5: Top: C3S daily TSI composite (green) and SATIRE-S reconstruction (red). Bottom : difference between the C3S and SATIRE-S daily TSI composites (green). The 121-days running mean curves are also shown (black).
3. Application(s) specific assessments
In addition to the extensive product validation (see chapter 2 for results and chapter 2/3 in [D6] for validation methodology) a second assessment is introduced to evaluate the Interim Climate Data Record (ICDR) against the Thematic Climate Data Record (TCDR) in terms of consistency. Since frequent ICDR deliveries make detailed validation not feasible, a consistency check against the deeply validated TCDR is used as an indication of quality. This is done by a comparison of the following two evaluations:
- TCDR against a stable, long-term and independent reference dataset
- ICDR against the same stable, long-term and independent reference dataset
The evaluation method is generated to detect differences in the ICDR performance in a quantitative, binary way with so called Key Performance Indicators. The general method is outlined in [D7] chapter 3. The same difference between TCDR/ICDR and the reference dataset would lead to the conclusion that TCDR and ICDR have the same quality (key performance is "good"). Variations or trends in the differences (TCDR/ICDR against reference) would require a further investigation to analyze the reasons. The key performance would be marked as "bad". The binary decision whether the key performance is good or bad is made in a statistical way by a hypotheses test (binomial test). Based on the TCDR/reference comparison (global means, monthly or daily means) a range is defined with 95% of the differences are within. This range (2.5 and 97.5 percentile) is used for the ICDR/reference comparison to check whether the values are in or out of the range. The results could be the following:
- All or a sufficient high number of ICDR/reference differences lies within the range defined by the TCDR/reference comparison: Key performance of the ICDR is "good"
- A smaller number of ICDR/reference differences is within the pre-defined range: Key performance of the ICDR is "bad"
3.1 Results
The results of the KPI test are summarized in Table 3-1.
Table 3-1: Summary of KPI results with 2.5 and 97.5 percentiles and number of ICDR days within the range. Colors green or red mark the results of the binomial tests as good or bad, respectively.
Total Solar Irradiance v2 | Total Solar Irradiance v3 | ||
---|---|---|---|
Percentiles | p2.5 p97.5 | 0.87 W/m² 1.83 W/m² | 0.130 W/m² 0.562 W/m² |
31.10.2020 - 31.03.2021 | 152/152 | ||
01.04.2021 - 28.02.2022 | 332/334 | ||
01.03.2022 - 31.03.2022 | 31/31 | ||
01.04.2022 - 30.06.2022 | 86/91 | ||
01.07.2022 - 30.09.2022 | 91/92 |
Percentiles were calculated based on the comparison of the TCDR against NRL TSI v2 composite as reference dataset for the variable Total Solar Irradiance (TSI). The ICDR daily means remain in general with in the 2.5 and 97.5 percentiles.
4. Compliance with user requirements
The C3S daily TSI record is about 1.4 W/m² higher than SATIRE-S and NRL TSI v2 records, which is out of the 1 W/m² accuracy requirement from GCOS. This (positive) bias is largely dependent on the choice of the reference instrument(s) used to homogenize the composite timeseries.
The validation results reported in this document provide evidence that, if the overall bias is not considered, the accuracy is much better than 1 W/m² for the daily mean TSI. Once bias-corrected, the RMS differences with the SATIRE-S and NRL TSI v2 records are both at about 0.25 W/m².
From the results presented in this report, it is not possible to conclude that the stability of the C3S TSI record (i.e. the maximum change of the systematic error over a running 10-years time period) complies with the 0.3 W/m²/decade GCOS requirements. The most significant decadal variation of the bias is 0.9 W/m²/decade. This change is observed with respect to the SATIRE-S between 1980 (bias about 1.6 W/m²) and 1984 (bias about 0.7 W/m²). With respect to NRL TSI v2 the variation of the bias is limited to 0.8 W/m² over the entire period covered by the TCDR.
References
Dewitte, S. and Nevens, S. (2016). The total solar irradiance climate data record. The Astrophysical Journal, 830(1), p.25.
Dewitte, S. and Clerbaux, N., (2017). Measurement of the earth radiation budget at the top of the atmosphere—a review. Remote Sensing, 9(11), p.1143.
Odele Coddington, Judith L. Lean, Doug Lindholm, Peter Pilewskie, Martin Snow, and NOAA CDR Program (2015): NOAA Climate Data Record (CDR) of Total Solar Irradiance (TSI), NRLTSI Version 2. [1984-2020]. NOAA National Centers for Environmental Information. doi:10.7289/V55B00C1
Available at: https://data.nodc.noaa.gov/cgi-bin/iso?id=gov.noaa.ncdc:C00899
Yeo K.L., Krivova N.A., Solanki S.K., Glassmeier K.H. (2014a).Reconstruction of total and spectral solar irradiance from 1974 to 2013 based on KPVT, SoHO/MDI and SDO/HMI observations. Astron. Astrophys. 570, A85 (2014). DOI: 10.1051/0004-6361/201423628
Yeo K.L., Krivova N.A., Solanki S.K. (2014b) Solar cycle variation in solar irradiance. Space Sci. Rev. (2014). DOI 10.1007/s11214-014-0061-7
SATIRE-S TIS dataset: http://www2.mps.mpg.de/projects/sun-climate/data/SATIRE-S_TSI_20190621.txt