Remedy 1: Systematic quantification of the impacts of different co-location criteria
Co-location criteria should represent an optimal compromise between the obtained number of co-located measurements (as large as possible to have robust statistical results) and the impact of natural variability on the comparisons (as low as possible to allow a meaningful comparison between measured differences and reported measurement uncertainties). Hitherto, only a few ground-based satellite validation studies explored the impact of the adopted co-location criteria on the comparison results (e.g. Wunch et al., 2011, and Dils et al., 2014, for CO2 , Verhoelst et al., 2015, for O3, Pappalardo et al., 2010, for aerosols, Lambert et al. 2012, for water vapour, Van Malderen et al. 2014, for integrated water vapour). Still, atmospheric variability is often assumed –or even known- to impact the comparisons, but without detailed testing of several co-location criteria (or by extensive model-based simulations), this impact is hard to quantify. Besides the need for dedicated studies, this gap also concerns the “community practices” regarding co-location approaches, which are neither consistent across different studies, nor optimal as they often rely on historical co-location criteria, which are not necessarily fit-for-purpose for the accuracy and spatiotemporal sampling properties of current measurement systems. Consequently, to ensure reliable and traceable validation results, as required in an operational context, community-agreed standards for co-location criteria should be developed, published, and adopted.
Two activities within GAIA-CLIM targeted this gap to some extent:
Within GAIA-CLIM, a dedicated task (T3.2 in WP3) dealt with data intercomparison studies, focusing on a closure of the comparison uncertainty budget and including an exploration of different co-location criteria, see for instance the results on total ozone columns published by Verhoelst et al. (2015, their Fig. 11).
The Virtual Observatory developed within GAIA-CLIM offers the user the possibility to adjust co-location criteria and to visualize the resulting impact on the comparison results.
However, no attempt has been made within GAIA-CLIM to produce an authoritative analysis and resulting documentation on this matter.
G3.04. To be addressed before G3.02
Argument: Ideally, co-location criteria take into account the smoothing and sampling properties of the measurements. Consequently, studies on co-location criteria can benefit from a proper characterization of these smoothing and sampling properties.
G3.06. To be addressed before G3.02
Argument: The merit of certain co-location criteria can best be assessed when the uncertainty budget of the comparisons is decomposed in measurement and co-location mismatch uncertainties.
G6.03. To be addressed after G3.02
Argument: Deciding on the best time and location for targeted reference observations should be informed by information on the optimal co-location criteria
- Dils et al., “The Greenhouse Gas Climate Change Initiative (GHG-CCI): comparative validation of GHG-CCI SCIAMACHY/ENVISAT and TANSO-FTS/GOSAT CO2 and CH4 retrieval algorithm products with measurements from the TCCON”, AMT v7, 2014
- Lambert, J.-C., et al., “Comparing and merging water vapour observations: A multi-dimensional perspective on smoothing and sampling issues”, in “Monitoring Atmospheric Water Vapour: Ground-Based Remote Sensing and In-situ Methods”, N. Kämpfer (Ed.), ISSI Scientific Report Series, Vol. 10, Edition 1, 326 pp., ISBN: 978-1-4614-3908-0, DOI 10.1007/978-1-4614-3909-7_2, © Springer New York 2012
- Pappalardo et al., “EARLINET correlative measurements for CALIPSO: First intercomparison results”, J.G.R.: Atmospheres v115, 2010
- Van Malderen, R. et al., “A multi-site intercomparison of integrated water vapour observations for climate change analysis”, AMT v7, 2014
- Verhoelst et al., “Metrology of ground-based satellite validation: Co-location mismatch and smoothing issues of total ozone comparisons”, AMT v8, 2015
- Wunch et al., “A method for evaluating bias in global measurements of CO2 total columns from space”, ACP v11, 2011
The impact of a particular choice of co-location criterion is only rarely studied in the scientific literature reporting on satellite validation results. However, without some quantification of the impact of the co-location criterion that was adopted, it is virtually impossible to assess the contribution of natural variability to the total error budget of the data comparisons. As such, this gap impacts significantly the potential interpretation of the data comparison result in terms of data quality. Some in-depth studies do exist, but testing multiple criteria, or using criteria based on the latest results of such exploratory work, is far from common (indeed, often arbitrary) practice(s). This gap thus concerns the need for more awareness among validation teams, for more detailed studies comparing the (dis-)advantages of various co-location criteria, and for community-agreed standards on co-location criteria that are broadly adopted in the context of operational services.