Remedy 1: Use of Observing System Simulation Experiments (OSSEs)
Remedy 2: Statistical estimation of typical co-location mismatch effects
Ideally, every validation study based on comparisons with ground-based reference data should investigate whether the comparison statistics (bias or mean difference, spread on the differences, drift, etc.) are compatible with the reported random and systematic measurement uncertainties, while taking into account the additional uncertainties due to spatiotemporal sampling and smoothing differences, i.e. non-perfect co-location of the airmasses sensed by both instruments. Indeed, it is only in a few particular cases possible to adopt co-location criteria that result in a sufficiently large number of co-located pairs, while at the same time keeping the impact of atmospheric variability on the comparisons (due to spatiotemporal mismatches) well below the measurement uncertainties. In all other cases, the discrepancy between two data sets will contain non-negligible terms arising from sampling and smoothing differences, which need to be taken into account. In fact, such an analysis is essential to fully assess the data quality and its fitness-for-purpose, but in practice, it is rarely performed, as this co-location mismatch is hard to quantify reliably. Some pioneering work was published by Cortesi et al. (2007) on uncertainty budget closure for MIPAS/ENVISAT ozone profile validation, by Ridolfi et al. (2007) for the case of MIPAS/ENVISAT temperature profiles validation, by Fasso et al. (2013) in the context of radiosonde intercomparisons, by Lambert et al. (2012) on water vapour comparisons, and by Verhoelst et al. (2015) for GOME-2/MetOp-A total ozone column validation. However, no such studies have hitherto been performed for most other ECVs and/or instruments. This gap therefore concerns the need for (1) further research dealing with methods to quantify co-location mismatch, and (2) governance initiatives to include in the common practices among validation teams dedicated efforts to construct full uncertainty budgets, and use these in the consistency checks.
Dedicated studies within GAIA-CLIM aimed for full error (or uncertainty) budget decomposition for representative comparison exercises, involving all non-satellite measurement types targeted by GAIA-CLIM and several current satellite sounders. Moreover, some of these results were transferred into the Virtual Observatory to allow end users to also decompose the uncertainty budget of their comparisons. Nevertheless, further work is required to quantify comparison error budgets in many cases, to operationalise comparison error budget calculations in operational satellite validation and production of higher level services, and to increase awareness in the community of the need for comparison error budget closure.
G3.01. To be addressed before G3.06
Argument: To quantify the additional errors and uncertainties in a comparison due to co-location mismatch, it is advantageous to have external information on the atmospheric variability on the scale of the co-location mismatch.
G3.04. To be addressed before G3.06
Argument: To quantify the additional errors and uncertainties in a comparison due to co-location mismatch, it is important to know the smoothing and sampling properties of the individual instruments
-
Cortesi et al., “Geophysical validation of MIPAS-ENVISAT operational ozone data”, ACP v7, 2007
-
Fassò et al., “Statistical modelling of collocation uncertainty in atmospheric thermodynamic profiles”, AMT v7, 2014
-
Lambert, J.-C., et al., “Comparing and merging water vapour observations: A multi-dimensional perspective on smoothing and sampling issues”, in “Monitoring Atmospheric Water Vapour: Ground-Based Remote Sensing and In-situ Methods”, N. Kämpfer (Ed.), ISSI Scientific Report Series, Vol. 10, Edition 1, 326 pp., ISBN: 978-1-4614-3908-0, DOI 10.1007/978-1-4614-3909-7_2, © Springer New York 2012
-
Ridolfi et al., “Geophysical validation of temperature retrieved by the ESA processor from MIPAS/ENVISAT atmospheric limb-emission measurements”, ACP v7, 2007
-
Verhoelst et al., “Metrology of ground-based satellite validation: Co-location mismatch and smoothing issues of total ozone comparisons”, AMT v8, 2015
A data validation study is meant to check the consistency of a given dataset with respect to a reference dataset within their reported uncertainties. As such, the uncertainty budget of the data comparison is crucial. Besides the measurement uncertainties on both data sets, the discrepancy between the two datasets will be increased by uncertainties associated with data harmonization manipulations (e.g. unit conversions requiring auxiliary data, interpolations for altitude regridding) and with co-location mismatch, i.e. differences in sampling and smoothing of the structured and variable atmospheric field. In particular, the latter term is hard to quantify and often missing in validation studies, resulting in incomplete uncertainty budgets and improper consistency checks.