G5.06

G5.06    Extraction, analysis and visualization tools to exploit the potential of reference measurements are currently only rudimentary. This in particular includes tools to display uncertainty of the comparison results due to differences in sampling and so called smoothing error.

Gap detailed description

Services that provide data extraction, analysis and visualization tools are currently only rudimentary. In particular, analysis capabilities that for instance allow analysis at different time or spatial scales are missing. The analysis capability provides a challenge to such services as no widely used tools for the comparison of satellite and non-satellite reference measurements exist presently. The GAIA-CLIM User Survey indicated a clear need for such a capability to be developed, but challenges remain because whatever analysis / visualization tool can be provided it will not necessarily match all individual needs. The GAIA-CLIM User Survey also indicated that the analysis of the co-locations provided by the Virtual Observatory may not solely be used to evaluate satellite measurements but also vice-versa the satellite measurements may be used to evaluate the quality of the reference measurements, e.g., their temporal consistency. Such a flexible tool does not exist to date.

While measurement uncertainties are at least displayed by some existing services, e.g., the FP7 NORS project, the visualization of uncertainty arising from differences in spatiotemporal sampling is generally not included, but is needed to fully understand the uncertainty budget of a specific comparison.

The impact of this gap is that a co-location data base without a comprehensive capability for data extraction, visualization and data analysis is of limited value to the scientific community and also for a potential later use in Copernicus Services.

Activities within GAIA-CLIM related to this gap

WP5 develops a Virtual Observatory that addresses this gap.

Gap remedy(s)

The Virtual Observatory shall demonstrate state-of-the-art approaches to the use of non-satellite reference data and NWP model data for the characterisation of satellite data by providing a comprehensive data extraction and visualisation capability together with an analysis capability for a number of static case studies.

Remedy #1

Specific remedy proposed

A data extraction capability can be developed that allows the export of data from the Virtual Observatory in user friendly formats. The GAIA-CLIM User Survey indicated a clear preference for the self-descriptive NetCDF format which also allows comparison data being amended by meta-data of the comparison, e.g., the used co-location criteria, etc. Such format also supports analysis of the data that may not be enabled, at least initially, in the Virtual Observatory. Data extraction tools also shall be capable of sub-setting each data source contained in the co-location data base by ECV, time and location, observing system and other boundary conditions such as surface type.

To exploit the co-location data base proposed as a remedy for gap G5.03, analysis tools must be developed to provide statistics and various indicators for a comparison that meet user needs as indicated by the GAIA-CLIM User Survey outcomes. These analysis tools must have some flexibility, such as interchanging the reference in a comparison and the ability to perform analysis at different time and eventually space scales.

Visualization tools need to be capable of displaying multiple collocated parameters to circumvent the complexity of comparing datasets of varying type and geometries, e.g. time series and instantaneous, spatially localized and large spatial extent observations, column-integrated observations and vertical profiles, etc. Special attention must be paid to the specification of graphical representation of individual parameters and various uncertainty measures, including the smoothing uncertainty.

Tool development shall benefit from existing elements and capabilities whenever possible. All developed tools need to be accessible via a graphical user interface that also needs to be developed.

Measurable outcome of success

Developed tools being accessible via a graphical user interface.

Achievable outcomes

Technological viability: High

Indicative cost estimate: High (>5 million) for a database covering ECVs beyond the GAIA-CLIM ECV set; low (<1 million) for demonstrating the general capability as planned in GAIA-CLIM.

Relevance

The remedy proposed here is in full agreement with the results of the GAIA-CLIM User Survey and the results of the 1^st GAIA-CLIM User Workshop.

Timebound

The remedy proposed here is a key focus and deliverable of GAIA-CLIM WP5 due for delivery (D5.3, D5.4) in months 24 and 30, respectively.

Gap risks to non-resolution