This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 640276.
G1.05 Lack of integrated user tools showing all existing observing capabilities for measuring ECVs with respect to satellite spatial coverage
Submitted by Anna Mikalsen on 9 June, 2017 - 15:59
Gap abstract:
The availability of user tools able to jointly visualize the current satellite and non-satellite observing capabilities for measuring ECVs at the global scale has never been provided in the past. Several tools have been implemented for specific instruments or networks of the global observing system, but all of them have been designed on the basis of very specific needs, using different criteria/functionalities, and typically including just one or a few ECVs and only one or a small subset of the available networks at the global scale. They have often been designed without user consultation. This lack of integrated user tools serves to inhibit the uptake of non-satellite measurements to characterize satellite observations.
Part I Gap description
Primary gap type:
- Technical (missing tools, formats etc.)
ECVs impacted:
- Temperature,Water vapour, Ozone, Aerosols, Carbon Dioxide, Methane
User category/Application area impacted:
- Operational services and service development (meteorological services, environmental services, Copernicus Climate Change Service (C3S) and Atmospheric Monitoring Service (CAMS), operational data assimilation development, etc.)
Non-satellite instrument techniques involved:
- Independent of instrument technique
-
There is an interdependency between G1.03 and G1.04, and consequently with this gap, whereby the resolution of the former will aid resolution of G1.04 by providing an assessable basis with broad buy-in to classify individual contributing measurement systems
In order to allow EO providers and users to maximize the value of existing observations and implement a user-friendly mapping facility, a comprehensive review of the current observing capabilities, at both European and global scales, is needed for all the ECVs.
Detailed description:
Several independent tools to enable discovery-metadata visualisation and exploitation have been implemented for specific networks of the global observing system. However, their design is often driven on the basis of very specific and particular needs, using different criteria / tools, and typically including just one ECV and only one or a small subset of the available networks. Users therefore have limited access to user-friendly tools, which can be used to explore the full and comprehensive view of all the sub-orbital observing capabilities. Users thus currently have a cumbersome and time-consuming search process to complete, if they wish to understand and exploit non-satellite data to its full potential. What is required is a unified tool that provides access to all relevant discovery metadata and appropriate search functionalities to enable users to discover and access the appropriate subset of data for their needs.
One of the most apposite examples of such a tool is represented by the OSCAR (Observing Systems Capability Analysis and Review Tool) system of CEOS and WMO and in particular for the surface based capabilities, which is still under development. At its present state, this tool is focused on national operational services and does not include all the ECVs and all the existing networks. For example, many of the high quality observational facilities are not run by National Meteorological or Hydrological Services and thus are not currently catalogued via OSCAR. Moreover, satellite-observing capabilities are collected separately from in-situ under WMO. This inhibits co-exploration of satellite and non-satellite capabilities. An integrated tool able to show simultaneously all the existing non-satellite capabilities, along with the field of view of the satellite-based instruments would greatly help end-users in the design of new validation strategies and in the full exploitation of both satellite and non-satellite data. This would in turn help inform users on the available ECV measurements within different domains (atmosphere, land, and ocean) through a facilitated analysis of the geographical distribution of the full suite of networks at the global scale.
Operational space missions or space instruments impacted:
- Independent of specific space mission or space instruments
Validation aspects addressed:
- Representativity (spatial, temporal)
Gap status after GAIA-CLIM:
- GAIA-CLIM has partly closed this gap
The GAIA-CLIM Virtual Observatory allows users to jointly explore data and metadata from available non-satellite and satellite observing capabilities, providing information on in-situ surface, in-situ sounding, columnar and profiling observations.
Part II Benefits to resolution and risks to non-resolution
| Identified benefit | User category/Application area benefitted | Probability of benefit being realised | Impacts |
|---|---|---|---|
Users to be able to fully exploit the content of surface-based and sub-orbital data and metadata |
|
| To facilitate the use of non-satellite data and their selection for satellite cal/val To enhance the analysis of the degree of temporal sampling mismatch between satellite instruments and surface-based stations for a relevant subset of EO platforms at real or selected time |
To provide user-friendly open-source tools in support of a powerful strategy to interact with users and communicate science |
|
| Availability of an interactive graphical user interface to explore the existing observing capabilities strongly facilitates the dialogue with end users, the identification of their needs, and the interaction with any type of broader audience, including students, policy-makers, and citizens |
| Identified risk | User category/Application area at risk | Probability of risk being realised | Impacts |
|---|---|---|---|
Lack of tools to drive support in future investments for the EO |
|
| Lack of services in the frame of the EO programs enabling users to quickly access and assess the suitability of a number of fully traceable reference measurements for a given application to satellite characterisation |
Lack of open-source tools to develop a virtual community of scientist and ICT experts capable to improve data exploitation |
|
| Reduction in the number of experts involved in the development of tools for the data exploitation. Missing support to the implementation of robust downstream services |
Part III Gap remedies
Gap remedies:
Remedy 1: Provision of mapping tools to match satellite and non-satellite observations
Primary gap remedy type:
Technical
Proposed remedy description:
The GAIA-CLIM 3D-mapping software is able to visualize a comprehensive list of in-situ metadata along with the main related satellite instruments. The software has the capability to continuously update the metadata also in an automatic fashion depending on the availability of updated metadata from in-situ networks.
Future potential work might include an extension of the current software capabilities to visualize also the observational data for a few instruments (e.g. the radiosonde flying from the launch station) and the capability to perform queries for a few existing data archives to check the data availability on-line. This work might be offered to the community also to encourage a joint effort amongst global stakeholders like GCOS, GEOSS, GAW to foster the design of further relevant tools.
In a broader context, the implementation of a unified tool that provides users with access to all metadata and data should be cognizant of a global community already sensitized to open-source software that can be easily accessed. Therefore, efforts should be made to implement an efficient, useful, platform-independent and open-source based service.
The work should consider:
- Use of open-source codes: examples include the Python ARM Radar Toolkit (Py-ART; https://github.com/ARM-DOE/pyart) and the GAIA-CLIM Virtual Observatory.
- Provide a detailed documentation of the codes, installation instructions, frequently asked questions, and other help facilities for users;
- Support enabling the users to program macros and small applications for a range of hardware platforms and compilers;
- Allocate resources to strengthen cooperation programmes between research institutes and global stakeholders to efficiently implement joint initiatives, which could offer a number of opportunities to the users and facilitate the implementation of downstream services.
For the last two items listed above, the forthcoming Copernicus Climate Change Service Data Store toolbox shall offer a first example of the direction to follow over the coming decade.
Relevance:
The GAIA-CLIM 3D-mapping software is a flexible open-source solution to visualize and quickly identify geographical gaps and, therefore, the starting point for any scientific assessment within the GAIA-CLIM project, but also going forward to support stakeholder’s data visualisation. It also offers some potential opportunities to work a use case of C3S and to support the development of downstream services.
Expected viability for the outcome of success:
- High
Scale of work:
- Programmatic multi-year, multi-institution activity
Time bound to remedy:
- Less than 3 years
Indicative cost estimate (investment):
- Low cost (< 1 million)
Indicative cost estimate (exploitation):
- No
Potential actors:
- Copernicus funding
- WMO
- ESA, EUMETSAT or other space agency