G5.07

G5.07     Incomplete development and/or application and/or documentation of an unbroken traceability chain of Cal/Val data manipulations for atmospheric ECV validation systems prevents progress in the characterization of satellite products.

Gap detailed description

In the context of sustainable Earth Observation data services such as those in development for the Copernicus Climate Change Service (C3S) and Atmospheric Monitoring Service (CAMS), Quality Assurance (QA) and geophysical validation play a key role in enabling users to assess the fitness of available data sets for their purpose. User requirements, e.g., those formulated for the Global Climate Observing System (GCOS), have to be identified and translated into QA and validation requirements; in turn, QA and validation results must be formulated in the form of appropriate Quality Indicators (QI) to check and document the compliance of the data with the user requirements. Metrology practices recommend the development and implementation of traceable end-to-end QA chains, based on Système International d’Unités (SI) and community-agreed standards (as identified for instance in the GEO-CEOS QA4EO framework).

Generic guidelines for such QA systems applicable virtually to all atmospheric and land ECVs are being developed within the EU FP7 QA4ECV project (2014-2018), while more specific guidelines developed in projects like ESA’s CCI and dedicated to atmospheric ECVs are being published (e.g., Keppens et al., 2015a). Generic and specific QA systems and guidelines established in those recent projects are not sufficiently well recognized or understood in the global community, where validation purposes, methodologies and results can differ significantly from one report to another. Harmonized practices should now be advertised and applied more universally across the community.

The impact of not adopting a traceable end-to-end validation approach is diverse. Firstly, important quality indicators may be missing in the analysis, e.g. information on spatio-temporal coverage, resolution, dependences of the data quality on particular physical parameters (e.g. solar zenith angle, could cover, thermal contrast) etc.  Secondly, results may be incoherent between several validation exercises on the same data set, and the origin of the discrepancies unclear due to insufficient traceability. Thirdly, methodological uncertainties in, e.g., geographical mapping, in the use of vertically averaging kernels, or in unit conversions using auxiliary data, may lead to unreliable results. Finally, all this may imply sub-optimal use of the true validation capabilities of the ground-based reference network.

Activities within GAIA-CLIM related to this gap

The GAIA-CLIM project adds to other EU projects with respect to more ECVs and disseminates results via the Virtual Observatory.

Gap remedy(s)

Development of a generic end-to-end QA and validation chain is ongoing for atmospheric ECVs in the EU FP7 QA4ECV project, with application to 3 pilot ECV precursors in QA4ECV and to ozone in ESA’s CCI Ozone project. This work needs to be extended to other ECVs, and the implementation of these QA4EO compliant practices must be illustrated (e.g. in the GAIA-CLIM VO) and operationalized (ongoing in the Multi-TASTE Cal/Val system operated at BIRA-IASB, e.g. Keppens et al. 2015b).

Remedy #1

Specific remedy proposed

The science for the end-to-end QA and validation chain is developed in several WPs in GAIA-CLIM. The Virtual Observatory can serve as a tool making the practises visible to users of the comparison data and results. In addition to remedies proposed for the gaps G5.03 and G5.06 described above the Virtual Observatory can present the traceability chains derived and also provide tutorials describing the methods used.

Measurable outcome of success

Full documentation available in the Virtual Observatory; and demonstrated use by VO users.

Achievable outcomes

Technological and organizational viability: High

Indicative cost estimate: Low (<1 million)

Relevance

The proposed remedy is fully in line with procedures established by QA4EO and provides a step forward to the currently available documentation.

Timebound

The remedy proposed here is a key focus and deliverable of GAIA-CLIM WP5 due for delivery (D5.3, D5.4 and D5.5) in months 24, 30 and 33, respectively.

Gap risks to non-resolution