Public Content

This content is available for all users and guests of this site.

Smoothing and co-location uncertainty libraries

GAIA-CLIM is supporting the European Commission’s Copernicus Programme by assessing and improving the fitness-for-purpose of sub-orbital (ground- and balloon-based) reference measurements for the validation of observational data sets from satellites. Amongst others, the project aims at improved traceability and uncertainty characterization of the individual sub-orbital measurement systems, and of the comparison with satellite data.  The latter implies the need for a rigorous treatment of so-called co-location mismatch uncertainties. These are the result of (differences in) smoothing and sampling properties of the various ground-based and satellite observing systems. These differences matter because the atmosphere is inhomogeneous and variable on the scale of individual measurements and on the scale of a typical co-location. Consequently, sampling and smoothing differences yield real physical and chemical differences, which have nothing to do with the measurement performance per se. Two perfect measurements measuring a slightly different volume and time integral are a priori expected to differ.   

Within GAIA-CLIM, several methods have been developed for - and applied to - the quantification of smoothing and sampling issues in a range of atmospheric ground-based measurement techniques, and to estimate the uncertainties that need to be taken into account when comparing non-perfectly co-located ground-based and satellite measurements with different spatio-temporal smoothing and sampling properties. The motivation for these developments and the principles of the different methods has been described in deliverable D3.2 (“Generic metrology aspects of an atmospheric composition measurement and of data comparisons”), and a wide variety of applications were performed and reported on in D3.4 (“Measurement mismatch studies and their impact on data comparisons”).

To translate these developments and results into a practical, usable resource for Copernicuas Climate Change Services users, two avenues were foreseen within GAIA-CLIM: (1) an integration of (derived) tools into the Virtual Observatory (VO), where they serve to estimate smoothing, sampling, and co-location uncertainties for the data ingested into the VO and for the satellite-ground co-locations performed by that system, and (2) publication of libraries (i.e. guiding material and data files) of such uncertainties for key ground-based measurement systems and for key comparisons. The former avenue constitutes the material for deliverables D3.5 ("Tools for quantification and smoothing uncertainties for integration in development of VO") and D3.7 ("Final version of tools for quantification of collocation mismatch and smoothing uncertainties and associated documentation for integration in the virtual observatory"), while the latter is the topic of the deliverable D3.6 ("Library of (1) smoothing/sampling error estimates for key atmospheric composition measurement systems, and (2) smoothing/sampling error estimates for key data comparisons").

The actual data files, are available for download (by anonymous ftp) at:

ftp://ftp-ae.oma.be/dist/GAIA-CLIM/D3_6/

Test

Test

Recommendations

The Recommendations document attempts to distill the findings from the gap analysis performed within GAIA-CLIM as described in the Gaps Assessment and Impacts Document (GAID) and associated online Catalogue of Gaps into a set of high-level actionable priority recommended activities. It is intended to be a high-level accessible document supported by the rich information on gaps collected throughout the project.

Comments and feedback are welcome via Feedback

Table: High-level recommendation titles and thematic clustering.

Education and training
Maintain and further develop a workforce competent in EO data characterisation and downstream applications to support Copernicus activities
Non-satellite data quality and availability
Improve the metrological characterisation of non-satellite measurement techniques: Striving for traceable, reference quality, fiducial measurement series
Augment and consolidate existing geographical coverage of reference quality observational networks to be more globally representative, including a range of surface types and climate zones
Improve time scheduling coherency of satellite and non-satellite measurements to minimise co-location uncertainty effects and ensure timely exchange of match-ups
Instigate and sustain timely access to a comprehensive set of harmonised reference data and metadata holdings under a common data model and open data policy that enables interoperability for applications
Observational network governance
Take steps to reassess, rationalise, and improve coordination of high quality observing networks
Conversion of non-satellite measures to TOA radiance-equivalents and their use
Improve knowledge of fundamental spectroscopy and undertake associated innovations in radiative-transfer modelling
Improve quantification of the effects of surface properties to reduce uncertainties in satellite data assimilation and satellite to non-satellite data comparisons
Develop and provide tools that convert non-satellite reference quality measurements to TOA radiance equivalents with associated rigorously quantified uncertainties
Understanding and quantifying irreducible co-location mismatch effects
Improve the basis for assigning co-locations and quantifying rigorously the associated uncertainties, including steps towards operational provision of co-location uncertainties
Provision of user tools that enable exploitation
Operationalise a co-location match-ups, visualisation and extraction tool, such as the GAIA-CLIM Virtual Observatory, to facilitate user access to satellite to non-satellite match-ups

27 November, 2017 - 08:30 General Assembly

  • A final General Assembly will be held at EUMETSAT in Darmstadt, Germany, on 27 and 28 November 2017 (all day). - Please save the date!

2017 General Assembly presentations

Reference documents: 

Gap reference list

Gap reference list

This is the full list of gaps identified in GAIA-CLIM to date, with a SMART analysis of each gap. Details of how and when such SMART remedies may be addressed for each gap are included in the full descriptions. The gaps each have unique gap identifier numbers of the form: X.XX, and full descriptions can be found at gaia-clim.eu/wiki/gX.XX; or by following the links in the Gap Identifiers below. We encourage user-input on the gaps, so if you can update our knowledge on existing gaps, or you would like to suggest the inclusion of additional gaps, please contact us using the "feedback" link next to the relevant gap, or the feedback button in the given gap description. We encourage you to use our template for external input which can be found here. You can also provide feedback using the contact form, using the 'GAID feedback' category.

Below search function allows filtering the Catalogue of Gaps along different cross-section, e.g. the type of gap and sugested remedies, instrument technicque, ECVs, or the potential actors addressed.

 

Gap identifier

Gap name

Feedback

G1.03

Lack of internationally recognised framework for assessment of fundamental observation capabilities

Feedback

G1.04

Lack of a comprehensive review of current non-satellite observing capabilities for the study of ECVs in atmospheric, ocean and land domains

Feedback

G1.05

Lack of integrated user tools showing all the existing observing capabilities for measuring ECVs with respect to satellite spatial coverage

Feedback

G1.06

Currently heterogeneous metadata standards negatively impact data discoverability and usability

Feedback

G1.10

Relative paucity and geographical concentration of reference quality measurements, with limited
understanding of uncertainty in remaining measurements, limits ability to formally close satellite to non-satellite comparisons

Feedback

G2.06

Poor spatial coverage of high-quality multi-wavelength lidar systems capable of characterising aerosols

Feedback

G2.07

Lack of uptake of lidar measurements in data assimilation

Feedback

G2.08

Lack of a metrological rigorous approach for ensuring continuous long-term water vapour measurements from Raman lidars in the troposphere and UT/LS

Feedback

G2.10

Tropospheric ozone profile data from non-satellite measurement sources is limited and improved capability is needed to characterise new satellite missions

Feedback

G2.11

Lack of rigorous tropospheric ozone lidar error budget availability

Feedback

G2.12

Lack of rigorous temperature lidar error budget availability limits utility for applications such as satellite characterisation

Feedback

G2.13

Missing microwave standards maintained by national/international measurement institutes

Feedback

G2.18
Better agreement needed on systematic and random part of the uncertainty in FTIR measurements and how to evaluate each part

Feedback

G2.22 FTIR cell measurements carried out to characterize ILS have their own uncertainties

Feedback

G2.24
Lack of calibrated in-situ vertical profiles of CH4, CO2 and, CO for improving the accuracy of FTIR column and profile measurements of CH4, CO2and CO

Feedback

G2.26
Poorly understood uncertainty in ozone cross-sections used in the spectral fit for DOAS, MAX-DOAS and Pandora data analysis

Feedback

G2.27
Lack of understanding of random uncertainties, AMF calculations and vertical averaging kernels in the total ozone column retrieved by UV-visible  spectroscopy

Feedback

G2.30 Incomplete uncertainty quantification for Pandora ozone measurements

Feedback

G2.31 Incomplete understanding of the different retrieval methods, information
content, and random and systematic uncertainties of MAX-DOAS tropospheric ozone measurements

Feedback

G2.34
Limit in traceability of GNSS-IPW ZTD estimates owing to dependency on 3rd party software

Feedback

G2.36 Lack of traceable uncertainties in MWR measurements and retrievals

Feedback

G2.37
Poorly quantified uncertainties in spectroscopic information

Feedback

G3.01
Incomplete knowledge of spatiotemporal atmospheric variability at the scale of the measurements and their co-location

Feedback

G3.02
Missing standards for, and evaluation of, co-location criteria

Feedback

G3.04
Limited characterization of the multi-dimensional (spatiotemporal) smoothing and sampling properties of atmospheric remote sensing systems, and of the resulting uncertainties

Feedback

G3.05
Representativeness uncertainty assessment missing for higher-level data based on averaging of individual measurements

Feedback

G3.06
Missing comparison (validation) uncertainty budget decomposition including uncertainty due to sampling and smoothing differences

Feedback

G4.01
Lack of traceable uncertainty estimates for NWP and reanalysis fields & equivalent TOA radiances – relating to temperature and humidity

Feedback

G4.08 Estimates of uncertainties in ocean surface microwave radiative transfer

Feedback

G4.09 Imperfect knowledge of estimates of uncertainties in land surface microwave radiative transfer

Feedback

G4.10
Incomplete estimates of uncertainties in land surface infrared emissivity atlases

Feedback

G4.12 Lack of reference quality data for temperature in the upper stratosphere and mesosphere

Feedback

G5.01
Plethora of data portals serving data under distinct data policies in multiple formats for reference quality data inhibits their discovery, access and usage for applications such as satellite Cal/Val

Feedback

G5.06
Extraction, analysis and visualization tools to exploit the potential of reference measurements are currently only rudimentary

Feedback

G5.07 Incomplete development and/or application and/or docuemntation of an unbroken traceability chain of data manipulations for atmosperic ECV validation systems

Feedback

G5.09
Need to propagate various reference quality geophysical measurements and uncertainties to TOA radiances and uncertainties to enable robust characterisation of satellite FCDRs

Feedback

G5.11
Non-operational provision of reference measurement data and some (L2) satellite products may prevent use in Copernicus operational product monitoring

Feedback

G6.01
Dispersed governance of high-quality measurement assets leading to gaps and redundancies in capabilities and methodological distinctions

Feedback

G6.02 Geographically dispersed observational assets reduce their utility for satellite Cal/Val

Feedback

G6.03 Lack of sustained dedicated observations to coincide with satellite overpass to minimise co-location effects

Feedback

G6.06
Requirement to make reference-quality measurements on a sustained and continuous basis, to maximise opportunities for the validation of satellite L1and derived
higher level products

Feedback

G6.07 Different data policies in different networks harm the use of complementary data from different networks

Feedback

G6.12
Under-capacity of workforce to exploit satellite data and satellite characterisation

Feedback

Pages

Subscribe to RSS - Public Content