This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 640276.
G1.10 Relative paucity and geographical concentration of reference-quality measurements, with limited understanding of uncertainty in remaining measurements, limits ability to formally close satellite to non-satellite comparisons
Submitted by Anna Mikalsen on 9 June, 2017 - 16:07
Gap abstract:
Limited availability of traceable uncertainty estimates limits the direct applicability of the majority of existing data to high-quality applications, such as satellite-data characterisation, model validation, and reanalysis. While a vast amount of data are available, the uncertainty of these data is - in a metrological sense - often only insufficiently specified, estimated, or even unknown. The reference-quality measurements that exist, tend to be geographically concentrated in the Northern Hemisphere mid-latitudes. In order to achieve progress, it is critical to have sufficient global coverage of reference quality data records that are stable over time, across the various methods of measurement, uniformly processed, and based on traceable references. This will allow to establish the robust scientific basis for using such data as a transfer standard in satellite-dataset characterization and other activities, such as trend analysis, and for assessing the cost-effectiveness of potential observing system enhancements. It is also essential to identify the scope for baseline and comprehensive networks to leverage expertise from reference networks, including adopting elements of best practice, and/or facilitating reprocessing that iteratively improves dataset quality.
Part I Gap description
Primary gap type:
- Knowledge of uncertainty budget and calibration
ECVs impacted:
- Temperature,Water vapour, Ozone, Aerosols, Carbon Dioxide, Methane
User category/Application area impacted:
- Operational services and service development (meteorological services, environmental services, Copernicus Climate Change Service (C3S) and Atmospheric Monitoring Service (CAMS), operational data assimilation development, etc.)
- Climate research (research groups working on development, validation and improvement of ECV Climate Data Records)
Non-satellite instrument techniques involved:
- Independent of instrument technique
-
This family of gaps collectively being addressed would substantively increase the pool of reference qualified techniques and instrument assets available globally to undertake measurements suitable for satellite Cal/Val.
Detailed description:
Presently, limited availability of traceable uncertainty estimates for non-satellite measurement techniques propagates to other applications, such as satellite characterisation. Such applications would be significantly improved were traceable uncertainty estimates more broadly available on the comparator measurements. The development work of the GAIA-CLIM Virtual Observatory has been addressing the selection of reference data, provision of measurement and co-location uncertainty estimates, and the provision of match-ups with satellite data to be characterized. This work has highlighted the relative geographical paucity of reference quality qualified measurement systems and their concentration in certain regions, principally Northern Hemisphere mid-latitudes. It can be expected that for other ECVs in atmospheric, but also oceanic and terrestrial domains, similar issues exist.
The issue of uneven geographical distribution of high-quality observation sites pervades many observational networks. In earlier versions of the GAID, a number of gaps pertaining to weaknesses in individual networks were identified. On further reflection, these gaps are sufficiently similar that the underlying challenges, and therefore solutions, were better addressed collectively through a recognition that this uneven sampling is a generic cross-cutting issue requiring a holistic, rather than per network consideration from the perspective of end-users, such as satellite calibration and validation activities. Compounding that is a lack of work that extends that knowledge to enable utilisation of remaining observations with requisite confidence.
While a vast amount of data are potentially available, unfortunately, the uncertainty of these data is all too often - in a metrological sense - insufficiently specified, estimated or even unknown, which frequently limits the applicability of the measurements to uses such as satellite characterisation. In order to achieve progress, it is critical to have data records that are stable over time, metrologically traceable to the method of measurement, uniformly processed worldwide (and thus comparable), and based on traceable references. This will allow us to establish the robust scientific basis for using such data as a transfer standard in satellite-dataset characterization and other activities, and for assessing the cost-effectiveness of potential observing system enhancements.
Thorne et al. (2017) provide the rationale behind and defining characteristics of a system-of-systems approach of “reference”, “baseline” and “comprehensive” networks. In that work, it is recognised that datasets from baseline and comprehensive networks provide valuable spatiotemporal coverage, but lack the metrological characteristics needed to facilitate traceable uncertainty estimates. It is therefore essential to identify the scope for baseline and comprehensive networks to leverage expertise from reference networks, including adopting elements of best practice from reference networks, and/or facilitating reprocessing that iteratively improves dataset quality. Such work may increase their utility for a range of applications, including satellite characterisation.
Operational space missions or space instruments impacted:
- Independent of specific space mission or space instruments
Validation aspects addressed:
- Geophysical product (Level 2 product)
- Time series and trends
- Calibration (relative, absolute)
- Spectroscopy
Gap status after GAIA-CLIM:
- GAIA-CLIM explored and demonstrated potential solutions to close this gap in the future
GAIA-CLIM participants have undertaken work on this issue on both a network and product level by working to improve mapping of current capabilities and addressing shortcomings of traceable uncertainty estimates. However, these activities have not completely addressed the issues arisen in this gap.
Part II Benefits to resolution and risks to non-resolution
| Identified benefit | User category/Application area benefitted | Probability of benefit being realised | Impacts |
|---|---|---|---|
Improved metrological characterisation of measurements |
|
| Clear improvement in the accuracy of climate data records Improved instrumentation arising from better understanding. |
Increased pool of reference quality measurements for satellite characterisation |
|
| Clear improvement in the capability to reliably validate satellite-data products. |
Better propagation of innovations across complementary observing systems |
|
| Improved quality and qualification of baseline- and comprehensive-network data suitable for satellite characterisation. |
Maturity matrix assessment provided by GAIA-CLIM project allows classification of observations into appropriate tiers |
|
| Better use of observations arising from better understanding of suitability for given applications |
| Identified risk | User category/Application area at risk | Probability of risk being realised | Impacts |
|---|---|---|---|
Limited impact of reference measurements on the observations provided by baseline and comprehensive networks for climate studies and satellite Cal/Val |
|
| Poor or lack of calibration procedures and data quality/traceability from baseline and comprehensive networks critically impacts on all those applications requiring high-quality measurements in time and space (i.e. satellite Cal/Val). |
Limited or neutral improvement of assimilation-based measurements. |
|
| Products lacking metrological traceability provide limited improvement in the characterization of model-based & assimilation-based uncertainties. |
Restricted set of reference quality observations persists |
|
| Continued uncertainty about the quality of satellite products for many ECVs used in service relevant applications. |
Part III Gap remedies
Gap remedies:
Remedy 1: Improved characterisation of high quality instrumentation to increase the pool of reference quality observing techniques without necessitating new observational deployments
Primary gap remedy type:
Research
Proposed remedy description:
Work to substantially improve the breadth of existing measurement techniques and programs that can be considered truly reference quality measurement systems. Building upon foundational work in existing EU H2020 projects such as QA4ECV, GAIA-CLIM, and FIDUCEO and by other international activities such as METEOMET, GRUAN, NDACC, GAW, the ESA Fiducial Reference Measurements program, etc. Undertake to improve the metrological characterisation of present and planned non-satellite measurement techniques for a broad range of atmospheric, oceanic, and terrestrial ECVs. Necessary steps include:
- Full characterisation of the processing chain for each individual measurement technique considered;
- Establishing traceability to SI or community standards;
- Quantifying the uncertainty in each processing step with metrological rigor;
- Ensuring comparability through necessary standardisation of techniques;
- Documentation of final product via the peer-reviewed literature and associated documentation.
This work shall require the involvement of instrument experts, metrologists, and potential end-users. The remedy should involve those measurement networks, which may deploy the developed measurement techniques as key partners to ensure uptake of the newly developed measurement streams in the field.
Relevance:
Directly addresses the paucity of reference-quality instrumentation by developing improved metrological understanding for a broad range of instrumentation that is either currently in the field or could be deployed.
Measurable outcome of success:
Improved number of reference qualified measurement techniques and increase in number of data streams available to end-users as a result.
Expected viability for the outcome of success:
- Medium
- High
Scale of work:
- Single institution
- Consortium
Time bound to remedy:
- Less than 3 years
Indicative cost estimate (investment):
- Medium cost (< 5 million)
Indicative cost estimate (exploitation):
- No
Potential actors:
- EU H2020 funding
- Copernicus funding
- National funding agencies
- WMO
- ESA, EUMETSAT or other space agency
- Academia, individual research institutes
Remedy 2: Take steps to better realise the benefits of a system-of-systems approach to observing strategies
Primary gap remedy type:
Research
Proposed remedy description:
Current observational networks are treated as distinct entities, all too frequently meaning that synergies resulting from a system-of-systems approach to observing are not realised. Without means to propagate innovations, practices, and know-how, the benefits of improved understanding from high-quality reference networks are limited. Work is required to develop tools and approaches that allow the effective flow of information from reference quality measurement networks to baseline and comprehensive observing networks, so that the benefits of that improved understanding can be realised. In the first instance, a case study based approach may be advisable that considers a well-defined problem set and allows testing of various approaches, following which are more substantial roll-out would be possible. An obvious candidate may be atmospheric temperature and humidity measurements for which several reference quality measurement techniques exist or are in the advanced stages of preparation and for which assimilation models and other techniques are similarly advanced. Work may include (but not be limited to) aspects such as:
- Use of reference sites to qualify uncertainties in techniques used in remaining networks via intercomparison campaigns. This may benefit from improved management of holdings if the new Copernicus Climate Change Service C3S 311a Lot 3 (access to observations from baseline and reference networks) activity is successfully executed.
- Enhancing observational practices in non-reference networks by taking realisable aspects of best practices from reference techniques. For example, the use of 100%-RH checks on radiosondes to characterise hysteresis effects more explicitly.
- Using data assimilation and statistical techniques to propagate information from reference sites to surrounding locales.
The work would need to involve operators of both reference and baseline / comprehensive networks to be effective and to recognise the realities involved in measurement programs. Cost-effective solutions that were technically and financially achievable should be developed that more effectively integrate information across networks and improve the quality of all observations.
Relevance:
Better propagating information across observing networks increases the value of all measurement programs to a range of applications, including satellite characterisation.
Measurable outcome of success:
Improved data quality leading to new and / or improved applications.
Expected viability for the outcome of success:
- Medium
- High
Scale of work:
- Single institution
- Consortium
Time bound to remedy:
- Less than 3 years
Indicative cost estimate (investment):
- Medium cost (< 5 million)
Indicative cost estimate (exploitation):
- No
Potential actors:
- EU H2020 funding
- Copernicus funding
- National funding agencies
- WMO
- ESA, EUMETSAT or other space agency
- Academia, individual research institutes
Remedy 3: Improving quantification of the impacts of geographical gaps on ability to undertake user-driven activities such as to characterize satellite data
Primary gap remedy type:
Research
Secondary gap remedy type:
Technical
Proposed remedy description:
Robust assessments of the impacts of geographical spatial and temporal gaps in the availability of reference quality measurement systems are required. GAIA-CLIM has developed studies based on global chemistry models, as well as on advanced statistical techniques, to evaluate these issues for a restricted subset of networks and ECVs (aerosol, ozone, trace gases, temperature and humidity). Similarly, other assessments have been undertaken elsewhere. But, historically, these have variously considered a subset of ECVs and / or networks and undertaken distinct methodological approaches which serve to inhibit their synthesis. Therefore, there is no clear and definitive set of analyses which unambiguously points to where additional observational assets would add most value. As evidenced by the interest in programs like Copernicus and the Fiducial Reference Measurements (FRM) program of European Space Agency (ESA), users are generally interested in the totality of capabilities and not a per network approach. Therefore, what is required is a holistic assessment approach that considers the issue across the full range of both reference-quality networks and ECVs.
In assessing against competing stakeholder needs, a robust means to quantify the cost-benefit trade-offs of different measurement capability expansion options (including both locations and scheduling of measurement strategies) that considered the problem more holistically (across ECVs and networks) would lead to more optimal configurations (or reconfigurations) of networks, recognising that there exists an ecosystem of synergistic and complementary networks. A substantive program that holistically assessed current capabilities and potential expansions / reconfigurations would require the participation of experts in modelling (climate, chemistry, weather), dynamics, statistics, and field measurement techniques. It would also require the engagement of the numerous stakeholders (end-users) of these data and the assessed networks.
Relevance:
A more robust scientific basis to assessing the impacts of current gaps would greatly aid decision makers in deciding how and where to expand reference-network capabilities
Measurable outcome of success:
Availability of a quantified basis to support decision-making.
Expected viability for the outcome of success:
- Medium
Scale of work:
- Consortium
Time bound to remedy:
- Less than 5 years
Indicative cost estimate (investment):
- High cost (> 5 million)
Indicative cost estimate (exploitation):
- No
Potential actors:
- EU H2020 funding
- Copernicus funding
- National Meteorological Services
- WMO
- ESA, EUMETSAT or other space agency
- Academia, individual research institutes
- SMEs/industry
- National measurement institutes
References:
- Thorne, P. W., Madonna, F., Schulz, J., Oakley, T., Ingleby, B., Rosoldi, M., Tramutola, E., Arola, A., Buschmann, M., Mikalsen, A. C., Davy, R., Voces, C., Kreher, K., De Maziere, M., and Pappalardo, G. (2017): "Making better sense of the mosaic of environmental measurement networks: a system-of-systems approach and quantitative assessment", Geosci. Instrum. Method. Data Syst., 6, 453-472, https://doi.org/10.5194/gi-6-453-2017, 2017.