Remedy 1: Propagation and adoption of metrological best practices in sustained validation activities
In the context of sustainable Earth Observation data services, such as those in development for the Copernicus Climate Change Service (C3S) and Atmospheric Monitoring Service (CAMS), Quality Assurance (QA) and geophysical validation play a key role in enabling users to assess the fitness of available data sets for their purpose. User requirements, e.g., those formulated for the Global Climate Observing System (GCOS), have to be identified and translated into QA and validation requirements. In turn, QA and validation results must be formulated in the form of appropriate Quality Indicators (QI) to check and document the compliance of the data with the user requirements. Metrology practices recommend the development and implementation of traceable end-to-end QA chains, based on Système International d’Unités (SI) and community-agreed standards (as identified for instance in the GEO-CEOS QA4EO framework).
Generic guidelines for such QA systems applicable virtually to all atmospheric and land ECVs are being developed within the EU FP7 QA4ECV project (2014-2018), while more specific guidelines developed in projects like ESA’s CCI and dedicated to atmospheric ECVs are being published (e.g., Keppens et al., 2015a). Generic and specific QA systems and guidelines established in those recent projects are not sufficiently well recognized or understood in the global community, where validation purposes, methodologies, and results can differ significantly from one report to another. Harmonized practices should now be advertised and applied more universally across the community.
The impacts of not adopting a traceable end-to-end validation approach are diverse. Firstly, important quality indicators may be missing in the analysis, e.g. information on spatio-temporal coverage, resolution, dependences of the data quality on particular physical parameters (e.g. solar zenith angle, cloud cover, thermal contrast) etc. Secondly, results may be incoherent between several validation exercises on the same data set and the origin of the discrepancies unclear due to insufficient traceability. Thirdly, methodological uncertainties in, e.g., geographical mapping, in the use of vertically averaging kernels, or in unit conversions using auxiliary data, may lead to unreliable results. Finally, all this may imply sub-optimal use of the true validation capabilities of the ground-based reference network
The GAIA-CLIM project adds to other EU projects with respect to more ECVs and disseminates results via the "Virtual Observatory" facility but does not close the gap.
The tools to be developed to address G5.06 in the context of validation work should be based on the traceability principles and Cal/Val best practices referred to in G5.07. In this sense, G5.06 should be addressed first, as it represents a contribution to the remedy for G5.07 (see G5.07 gap remedy #1).
Recently established quality assurance and validation guidelines and systems are not sufficiently well recognised or understood in the global community, where validation purposes, methodologies, and results can differ significantly from one report to another. Harmonised practices should now be advertised and applied more universally across the community to avoid (1) missing quality indicators, (2) incoherent results between different validation exercises, and (3) unreliable results or additional methodological uncertainties due to sub-optimal data manipulations. Moreover, there is room for further improvement in validation methodologies, taking advantage of the ever-increasing breadth of measurement, modelling, and data analysis techniques.