G5.03

G5.03    No common source for co-located data exists which prevents use of reference data to validate reference measurements to each other and to evaluate satellite data.

Gap detailed description

Several sources for co-located data sets exist but most of them are specialized to compare mapped fields, e.g., obs4mips ESG data provisions, reference and other non-satellite data against models or satellite data, e.g., the NORS project. But most of these are not fully utilizing the potentially available information on uncertainty or including uncertainty arising from spatiotemporal mismatch of the compared data streams. Some of the existing datasets are publically available via the internet, while others are run internally to organizations like EUMETSAT to monitor data quality in real time. The effect of this gap is that many validation activities are performed, but do not use the available uncertainty information in an optimal way which has general effects on the quality of the research and the robustness of any conclusions drawn from such validation exercises.

A common source that integrates several reference data networks with satellite data considering traceable uncertainty does not exist but is needed according to the GAIA-CLIM user survey.

Activities within GAIA-CLIM related to this gap

WP5 develops a Virtual Observatory that addresses this gap.

Gap remedy(s)

The Virtual Observatory shall be developed to demonstrate the use of non-satellite reference data and NWP model data for the characterisation of satellite data. The Virtual Observatory must integrate the different measurements, their metadata, quantified uncertainty for the measurements, and the uncertainty arising from the comparison process. The major part of the VO is a co-location data base that enables various scenarios for comparison of the satellite and non-satellite data.

Remedy #1

Specific remedy proposed

As a major part of the VO a co-location data base must be developed to establish a foundation for the remedy of this gap. The first step is to identify all pertinent satellite and non-satellite reference datasets that are of interest for a comparison to a given satellite sensor data. This could either be via a forward modeling approach to derive an estimate of the satellite sensor data or a comparison to geophysical variables derived from the satellite data. The provided data need to be complemented by as complete as possible meta-data and traceable uncertainty information, including comparison mismatch uncertainties that need to be derived from the comparison setting and the variability of the geophysical variable to be compared.

For satellite measurements, suitable spatial extent around reference measurement sites must be specified. Similarly, for non-satellite measurements, pertinent time range around satellite overpass time must be specified to allow for a meaningful comparison.

Many satellite measurements contain information from different geophysical variables, e.g., the separation of temperature and humidity signals in a retrieval scheme working in the infrared spectral range is very complex. Instead, the co-located reference measurements, for this example containing temperature and humidity profiles, can be used to simulate the expected satellite measurement which allows a more meaningful comparison. Such a ‘processor’ is addressed in the remedy of gap G4.01 in WP4.

For some variables, further work may be required to make them consistent across satellite sensors or observation sites. For instance, for aerosol products, retrievals may be available at different wavelengths requiring further calculation to derive a reference variable: e.g. AOD (Aerosol Optical Depth) at 550 nm. Similarly, some retrieval may provide fine mode AOD and coarse mode AOD, others may provide total AOD and fine mode fraction.

A technical constraint for a co-location data base is its volume that needs to of reasonable size to remain manageable.

Measurable outcome of success

Established co-location database that is accessible for extraction, analysis, visualisation tools and allows for interrogation by users via a user interface.

Achievable outcomes

Technological viability: High

Indicative cost estimate: High (>5 million) for a database covering ECVs beyond the GAIA-CLIM ECV set; low (<1 million) for demonstrating the general capability as planned in GAIA-CLIM.

Relevance

The remedy proposed here is in full agreement with the results of the GAIA-CLIM User Survey and the results of the 1^st GAIA-CLIM User Workshop.

Timebound

The remedy proposed here is a key focus, and deliverable of GAIA-CLIM WP5 due for delivery (D5.3, D5.4) in months 24 and 30, respectively.

Gap risks to non-resolution