This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 640276.
G4.01 Lack of traceable uncertainty estimates for NWP and reanalysis fields & equivalent TOA radiances – relating to temperature and humidity
Submitted by Anna Mikalsen on 12 June, 2017 - 12:15
Gap abstract:
Numerical Weather Prediction (NWP) models are already routinely used in the validation and characterisation of Earth Observation (EO) data. However, a lack of robust uncertainties associated with NWP model fields and related top-of-atmosphere (TOA) radiances prevent the use of these data for a complete and comprehensive validation of satellite EO data, including an assessment of absolute radiometric errors in new satellite instruments. Agencies and instrument teams, as well as key climate users, are sometimes slow (or reluctant) to react to the findings of NWP-based analyses of satellite data, due to the current lack of traceable uncertainties.
Part I Gap description
Primary gap type:
- Uncertainty in relation to comparator
Secondary gap type:
- Knowledge of uncertainty budget and calibration
ECVs impacted:
- Temperature
- Water vapour
User category/Application area impacted:
- Operational services and service development (meteorological services, environmental services, Copernicus Climate Change Service (C3S) and Atmospheric Monitoring Service (CAMS), operational data assimilation development, etc.)
- International (collaborative) frameworks and bodies (space agencies, EU institutions, WMO programmes/frameworks etc.)
Non-satellite instrument techniques involved:
- Radiosonde
G4.08 and G4.09 are concerned with uncertainties in microwave surface radiative transfer for respectively the ocean and land surfaces. This gap (G4.01), being concerned with modelled TOA radiances, is partially dependent on a knowledge of uncertainties in the surface microwave radiative transfer. G4.08 should be addressed with the current gap and G4.09 can be addressed independently
G4.10 is concerned with uncertainties in infrared land surface emissivity atlases. This gap (G4.01), being concerned with modelled TOA radiances, is partially dependent on a knowledge of surface emissivity uncertainties. G4.10 can be addressed independently of the current gap.
G4.12 is concerned with the lack of reference measurements for the higher atmosphere (pressures less than 40 hPa). This gap (G4.01) cannot be closed for this part of the atmosphere without first addressing G4.12
Detailed description:
Numerical Weather Prediction (NWP) models and reanalysis systems possess a number of key attributes for the comprehensive assessment of observational datasets. These models routinely ingest large volumes of observations within the framework of data assimilation and, combined with model data, produce optimal estimates of the global atmospheric state. The model fields are constrained to be physically consistent, and have continuous coverage in time and space. NWP fields exhibit sufficient accuracy in their representation of temperature and humidity fields to enable the characterisation of subtle biases in monitored satellite data. Examples include the evaluation of SSMIS (Bell et al., 2008), FY-3A sensors (Lu et al., 2011) and AMSU-A (Lupu et al., 2016).
However, robust uncertainty estimates for NWP fields are still lacking. Space agencies and instrument teams, as well as key climate users, are sometimes slow (or reluctant) to react to the findings of NWP-based analyses of satellite data due to the current lack of traceable uncertainties. Reliable estimates for the uncertainty of NWP fields, and modelled TOA radiances, would allow an assessment of absolute radiometric errors in satellite instruments. The aim is to assess uncertainties in NWP fields, through systematic monitoring, using reference-quality data,
The aim of GAIA-CLIM activities is to assess uncertainties in NWP fields through systematic monitoring, using data from the GCOS Reference Upper-air Network (GRUAN) radiosonde network. Difference statistics evaluated by Noh et al. (2016) for three institutes’ models indicated good agreement with GRUAN profiles for temperature (biases not exceeding 0.1-0.2 K throughout the troposphere, with root-mean-square (RMS) differences within 1 K). Models were found to be less skilful at representing relative humidity (RH) fields, with biases cf. GRUAN sondes of up to 5% RH and RMS differences up to 15% RH. This illustrates the particular need to quantify NWP humidity uncertainties, as a means of improving the assessment of satellite EO data, which are sensitive to atmospheric water vapour.
GAIA-CLIM has developed a ‘GRUAN processor’ as a software tool, which enables the routine comparisons of NWP fields with reference radiosonde data. Importantly, these comparisons can be conducted both in terms of geophysical variables (temperature, humidity) and TOA radiances or brightness temperatures. It is estimated that significant progress can be made in establishing this routine monitoring within the timescale of GAIA-CLIM, although maintenance of the processor is not guaranteed beyond the lifetime of the project.
The complexity of NWP and reanalysis systems is such that a complete error budget is unattainable. However, progress can be made in accounting for spatial, seasonal, diurnal, and weather regime factors that affect uncertainties. This can be achieved through comparisons with recognised reference measurements, such as GRUAN radiosondes, complemented by ‘near-reference’ measurements with greater global coverage.
Operational space missions or space instruments impacted:
- Meteosat Third Generation (MTG)
- MetOp
- MetOp-SG
- Polar orbiters
- Microwave nadir
- Infrared nadir
- Passive sensors
- Other, please specify:
US Joint Polar Satellite System (JPSS): ATMS, CrIS instruments
Chinese Fengyun (FY) weather satellites: MWTS, MWHS, MWRI instruments
Validation aspects addressed:
- Representativity (spatial, temporal)
- Calibration (relative, absolute)
Gap status after GAIA-CLIM:
- GAIA-CLIM has partly closed this gap
Significant progress has been made in the development of a ‘GRUAN processor’ for the routine comparison of NWP fields with reference data. It is likely that components of the uncertainty budget relating to the comparisons will need further investigations beyond GAIA-CLIM.
GAIA-CLIM has further established the value of NWP in the validation of microwave temperature sounding instruments (e.g. Meteor-M N2 MTVZA-GY), microwave humidity sounders (e.g. FY-3C MWHS-2) and microwave imagers (e.g. GCOM-W AMSR-2).
Part II Benefits to resolution and risks to non-resolution
| Identified benefit | User category/Application area benefitted | Probability of benefit being realised | Impacts |
|---|---|---|---|
Through lower cost, effective and timely validation of new microwave missions, of which there are >10 planned over the next 2 decades. |
|
| More timely integration of new, validated satellite data sets into reanalyses. |
Broader C3S user base |
|
| Improved confidence in, and established quantitative uncertainties for, ERA temperature and humidity analyses. Improved confidence in projected impacts. |
| Identified risk | User category/Application area at risk | Probability of risk being realised | Impacts |
|---|---|---|---|
Sub-optimal validation of EO data |
|
| Continued uncertainty on the value of NWP for the validation of (primarily temperature sounder and humidity sounder and imager) satellite data. Motivates more costly Cal/Val campaigns based on airborne measurements (a large and recurring cost for each new mission). Data users have less confidence in findings based on observational data of uncertain quality. Slower evolution of the community’s understanding of the quality of EO data sets, particularly for new missions. Failure to recognise defects in instruments and/or processing chains may result in sub-optimal satellite data being used in downstream applications (e.g. reanalyses or climate studies). |
Unknown uncertainties associated with NWP temperature and humidity fields |
|
| While model biases and uncertainties remain unquantified, NWP centres cannot respond by targeting model performance improvements. Users of NWP and reanalysis data want reliable uncertainty estimates rather than taking the data on trust. While uncertainties are lacking, this limits the confidence in, and societal impact of, NWP forecasts and reanalyses. |
Part III Gap remedies
Gap remedies:
Remedy 1: Development of tools to propagate geophysical profile data and attendant uncertainties to TOA radiances and uncertainties
Primary gap remedy type:
Technical
Technology Readiness Level (TRL) 6
Proposed remedy description:
Develop a ‘GRUAN processor’ as a software deliverable from GAIA-CLIM. The GRUAN processor consists of a platform that enables the visualisation and exploitation of co-locations between GRUAN observed profiles and NWP fields. The processor enables visualisation both in geophysical space and as TOA radiance equivalents for a range of temperature and humidity sensitive satellite sensors. GAIA-CLIM has produced the processor in a demonstration capability. Further efforts would be required to operationalise its availability and generalise the processor to include other reference-quality measurements from further non-satellite measurement techniques.
Relevance:
The software is open-source and enables users (by which we mean reasonably knowledgeable users) to compare NWP fields from both ECMWF and Met Office (in the first instance) with GRUAN data. This includes a comparison of temperature and humidity, as well as TOA brightness temperatures for all sensors supported by the (publicly available) RTTOV radiative transfer model.
Measurable outcome of success:
- Statistics available on the comparison, for all GRUAN sites, with respect to ECMWF and Met Office NWP fields.
- A web page displaying these statistics.
- An open-source GRUAN processor available to the wider community.
- Integration of the GRUAN processor into the GAIA-CLIM Virtual Observatory.
Expected viability for the outcome of success:
- High
Scale of work:
- Single institution
- Consortium
Time bound to remedy:
- Less than 3 years
Indicative cost estimate (investment):
- Low cost (< 1 million)
Indicative cost estimate (exploitation):
- Yes
Minor costs associated with hosting, upkeep and periodic reviews for updates
Potential actors:
- EU H2020 funding
- National Meteorological Services
Remedy 2: Evaluate quality of NWP and reanalysis fields through comparisons with reference data as a means of establishing direct traceability.
Primary gap remedy type:
Technical
TRL 6
Proposed remedy description:
The GRUAN processor developed for GAIA-CLIM offers the means of traceable evaluation of the quality of NWP fields at the GRUAN-site locations. Due to the scarcity of reference measurements for comprehensive evaluation of NWP data, it will be necessary to determine additional ‘near-reference’ measurements for which defensible uncertainty estimates can be provided. It is proposed to extend the assessment of NWP fields using other data of demonstrated quality, such as selected GUAN radiosondes and GNSS radio occultations, in order to sample a larger subspace of NWP regimes. Additionally, NWP and reanalysis systems now make use of ensembles (multiple forecasts to represent error growth from uncertain initial conditions and stochastic physics perturbations). Uncertainties as estimated from ensembles should be evaluated using available NWP minus reference-data differences. It is also desirable to extend the assessment to include atmospheric composition, for which reference composition measurements and their uncertainties are required.
Relevance:
NWP and reanalysis fields and products are very widely used for the validation and characterisation of EO data, although associated robust uncertainties are lacking. Traceable uncertainties will engender more confidence from users.
Measurable outcome of success:
Published uncertainties should be available for widely used NWP and reanalysis model fields such that the uncertainties and associated correlation structures are traceable to underlying reference data.
Expected viability for the outcome of success:
- High
Scale of work:
- Single institution
- Consortium
Time bound to remedy:
- Less than 5 years
Indicative cost estimate (investment):
- Low cost (< 1 million)
Indicative cost estimate (exploitation):
- No
Potential actors:
- EU H2020 funding
- National Meteorological Services
References:
- Bell, W., English, S. J., Candy, B., Hilton, F., Atkinson, N., Swadley, S., Baker, N., Bormann, N. and Kazumori, M. (2008). The assimilation of SSMIS radiances in numerical weather prediction models. IEEE Trans. Geosci. Remote Sensing 46: 884–900.
- Lu, Q., Bell, W., Bauer, P., Bormann, N. and Peubey, C. (2011), An evaluation of FY-3A satellite data for numerical weather prediction. Q.J.R. Meteorol. Soc., 137: 1298–1311. doi:10.1002/qj.834
- Lupu, C., Geer, A., Bormann, N. and English, S. (2016). An evaluation of radiative transfer modelling errors in AMSU-A data, ECMWF Technical Memorandum 770. Available from http://www.ecmwf.int/en/elibrary/technical-memoranda
- Noh, Y.-C., Sohn, B.-J., Kim, Y., Joo, S., and Bell, W. (2016). Evaluation of Temperature and Humidity Profiles of Unified Model and ECMWF Analyses Using GRUAN Radiosonde Observations. Atmosphere, 7, 94; doi:10.3390/atmos7070094