Non-satellite instrument techniques involved
Independent of instrument technique
Gap remedies
Detailed description

Metadata is an increasingly essential tool enabling large-scale, distributed management of resources. Recent years have seen a growth in interaction between previously relatively isolated communities across observing domains and techniques, driven by a need for interdisciplinary research and understanding. However, metadata standards have not been able to meet the needs of interoperability between these to date largely independent communities and networks. Observations without metadata are of very limited use: it is only when accompanied by adequate metadata (data describing the data) that the full potential of the observations can be realized. Format conversions always bring with them the danger of destroying information in the process, in particular in the accompanying metadata, which usually receives less attention.

Several efforts have been undertaken to improve the harmonization of metadata across numerous networks and international programs, but this is still not sufficient. Harmonization effort in the atmospheric science community is starting to be addressed by the emerging WIGOS standards, currently under development and subsequent implementation at the WMO, and by the ESA Climate Change Initiative (CCI), amongst others. Copernicus Climate Change Service Data Store activities are also highly relevant to this gap. There are also challenges that arise due to interoperability across observational domains (surface, atmospheric, oceanic, terrestrial etc.). 

Operational space missions or space instruments impacted
Independent of specific space mission or space instruments
Validation aspects addressed
Radiance (Level 1 product)
Geophysical product (Level 2 product)
Gridded product (Level 3)
Assimilated product (Level 4)
Time series and trends
Representativity (spatial, temporal)
Gap status after GAIA-CLIM
GAIA-CLIM explored and demonstrated potential solutions to close this gap in the future

GAIA-CLIM metadata standards and format harmonization have been carried out with aim to provide a model for facilitating the users access and the usability in-situ data. This exercise included the establishment and documentation of common metadata and data formats for a selected subset of networks that will contribute to the Virtual Observatory. The Virtual Observatory facility shall also support the remedy of this gap by providing data format conversion for various input data and a data extraction function that makes the outputs available in user friendly formats.

GAIA-CLIM activities will be followed up by the Copernicus Climate Change Service, where, for a selected number of networks reviewed within GAIA-CLIM, the harmonization of the data and metadata format and structure is ongoing. According to the requirements provided by the Copernicus end-users through the C3S Sectoral Information System (SIS) projects, this effort involves the implementation of a common data model compliant with the ECWMF Observational DataBase (ODB) and a data-management facility, which shall become part of the operational C3S services at the end of the above-mentioned contract. 

Dependencies

G1.03 and G1.04 should be addressed after G1.06

The resolution of G1.06 would bring invaluable benefits to support the resolution of G1.03 and G1.04 by facilitating the review of existing capabilities, starting from rich standardized information and enabling classification of measurement maturity in a more accurate way.

G5.01 should be addressed with this gap

Metadata harmonization across multiple data provides will also positively impact on the interoperability among different data repositories with clear benefits for addressing gap G5.01 

The need for extensive and accurate metadata is ever increasing in both research and operations, enabling large-scale, distributed management of resources. Recent years have seen a growth in interaction between previously relatively isolated communities, driven by a need for cross-domain collaboration and exchange of data and products. However, metadata standards have generally not been able to meet the needs of interoperability between independent standardization communities. Observations without useable metadata are of very limited use as the metadata provides key context such as the time, location, and modality of the measurements. Several efforts have been undertaken to improve the harmonization of metadata across the networks and international programs, but currently this is still insufficient.