G2.27 Lack of understanding of random uncertainties, air mass factor calculations, and vertical averaging kernels in the total ozone column retrieved by UV-visible spectroscopy

Gap abstract: 

The uncertainties in the ozone slant columns retrieved with DOAS data analysis fitting procedures are predominantly caused by instrumental imperfections and by issues introduced within the analysis routines. Such uncertainties are often random and therefore can be estimated statistically from, e.g., the least-squares fit procedure. However, the fitting uncertainties derived from such analysis typically result in unrealistically small uncertainties and can lead to an underestimate by up to a factor of two. Further uncertainties are introduced during the calculation of air mass factors (AMFs) which are required to convert the measured ozone slant columns into vertical columns. The AMF uncertainties are dominated by errors in a priori profile shape effects with ozone and pressure/temperature a priori profiles being key input parameters for the AMF calculations. For further interpretation of the total column observations, averaging kernel information as part of the retrieval product plays an important role. However, currently vertical averaging kernels are only approximations of the real 3D averaging kernel and cannot fully account for the representativeness of the data.

Part I Gap description

Primary gap type: 
  • Knowledge of uncertainty budget and calibration
ECVs impacted: 
  • Ozone
User category/Application area impacted: 
  • Operational services and service development (meteorological services, environmental services, Copernicus Climate Change Service (C3S) and Atmospheric Monitoring Service (CAMS), operational data assimilation development, etc.)
  • Climate research (research groups working on development, validation and improvement of ECV Climate Data Records)
Non-satellite instrument techniques involved: 
  • UV/VIS zenith DOAS
  • UV/VIS MAXDOAS
  • Pandora
  • This gap represents the top-level coordination and harmonisation activity required across the general spectroscopic measurement field, therefore G2.27 should be addressed in parallel with G2.37

Detailed description: 

This gap addresses three of the major individual issues in our understanding of the analysis processing chain from the raw spectrum to the final total column ozone data product using the DOAS technique, and the interpretation of the actual final product.

The first aspect is the uncertainties in the ozone slant columns retrieved with the standard DOAS data analysis fitting procedures. They are to a large part caused by (1) instrumental imperfections such as detector noise, resolution change, etaloning (a fault that develops in thin charge-coupled devices when they behave as etalons) and other non-linearities of the detector, stray-light, and polarisation effects, as well as (2) by issues introduced within the analysis routine such as uncertainties in the Ring effect, unknown absorbers, and the wavelengths dependency of the AMF. Such uncertainties are mostly random in nature and therefore can be estimated statistically from the least-squares fit procedure. However, the fitting uncertainties derived from the least-squares analysis typically result in unrealistically small uncertainties and can lead to an underestimate of the measurement uncertainty by up to a factor of two. Results from intercomparison exercises (e.g. Van Roozendael et al., 1998, Vandaele et al., 2005, Roscoe et al., 2010) show that state-of-the-art instruments hardly ever agree to better than a few percent, even when standardised analysis procedures are used. This indicates that the actual accuracy in the ozone slant columns is at least to some degree limited by uncontrolled instrumental and/or analysis factors. And it leads to the question if something is not yet adequately addressed in the fitting procedures.

Further uncertainties are introduced during the calculation of air mass factors (AMFs) which are required to convert the measured ozone slant columns into vertical columns which means that the measured slant column density (SCD) is divided by the AMF to calculate the vertical column density (VCD) in molecules/cm2 which is then converted into Dobson Units. The NDACC UV-visible spectroscopy working group recommends the use of a generic look-up table of ozone AMFs which has been developed at BIRA-IASB (see NDACC UV-vis working group report) and accounts for the latitudinal and seasonal dependencies of the ozone vertical profiles. The NDACC recommendation is furthermore to average all retrieved vertical columns of ozone between 86° and 91° Solar Zenith Angle (SZA). The recommended approach is to apply a linear fit on vertical columns in the above SZA range and then derive the column value at the effective SZA (so far recommended to be 90° SZA). This range minimizes the measurement uncertainties arising during the fitting procedures and AMF calculation, and provides stratospheric ozone measurements with limited sensitivity to tropospheric ozone and clouds. Ozone and pressure/temperature a priori profiles are key input parameters for the AMF calculations, and AMF uncertainties for zenith-sky twilight ozone retrievals are dominated by uncertainties in a priori profile shape effects. Hendrick et al. (2011) found that the uncertainty in the calculated AMFs based on uncertainties in the ozone profiles is around 1%. However, there is a lack of an adequate database of tropospheric ozone in particular, and in regions where tropospheric or stratospheric ozone contents deviate from the climatological values, uncertainties of several percent can be introduced in total column ozone retrievals. Apart from uncertainties in the ozone a priori profiles, further sources of uncertainty are based on uncertainties in the aerosol and cloud information used. The typically small impact of clouds on zenith-sky ozone UV-vis measurements at twilight is due to the fact that the mean scattering layer is generally located at higher altitude than that of the clouds. However, AMFs calculated for cloudy conditions can be systematically larger than AMFs calculated for non-cloudy conditions.

The DOAS ozone total column retrieval is implicitly dependent on an a priori tracer profile. The radiative transfer calculation within the DOAS analysis accounts for the sensitivity of the measurement to tracer concentrations at all altitudes. These sensitivities are implicitly weighted with the assumed tracer profile to produce the retrieved column.The averaging kernel is proportional to this measurement sensitivity profile, and provides the relation between the retrieved quantities and the true tracer profile. The kernel therefore provides important information needed for a quantitative analysis of the satellite data (Eskes and Boersma, 2003 and references therein). The averaging kernel concept is by now well established in remote sensing. Applications are for instance the retrieval of profiles of atmospheric quantities like temperature and tracers like ozone from satellite measurements. Retrieval groups are increasingly including the kernel information in the profile data products disseminated to users. The look-up tables for total column ozone averaging kernels, provided by the NDACC UV-vis working group, have been developed based on the approach described by Eskes and Boersma (2003), i.e. the averaging kernel of a layer i can be approximated by the ratio of the box airmass factor of this layer i and the total airmass factor calculated from an O3 profile climatology. The availability of averaging kernel information as part of the total column retrieval product is important for the interpretation of the observations, and for applications like chemical data assimilation and detailed satellite validation studies. However, vertical averaging kernels (when provided based on a climatology) are only approximations of the real 3D averaging kernel of a retrieval and cannot fully account for the representativeness of the data.

Operational space missions or space instruments impacted: 
  • Copernicus Sentinel 4/5
  • Meteosat Third Generation (MTG)
  • MetOp
  • Polar orbiters
  • Geostationary satellites
  • UV/VIS nadir
Validation aspects addressed: 
  • Geophysical product (Level 2 product)
  • Time series and trends
  • Spectroscopy
Gap status after GAIA-CLIM: 
  • GAIA-CLIM has partly closed this gap

An in-depth uncertainty analysis has been undertaken under GAIA-CLIM but closure requires its verification and implementation.

Part II Benefits to resolution and risks to non-resolution

Identified benefitUser category/Application area benefittedProbability of benefit being realisedImpacts
If the source of the differences between fit uncertainty and expected uncertainty is better understood, this would lead to an improvement in the fit quality
  • Operational services and service development (meteorological services, environmental services, Copernicus services C3S & CAMS, operational data assimilation development, etc.)
  • Climate research (research groups working on development, validation and improvement of ECV Climate Data Records)
  • High
  • Medium
Improvement in overall data quality & more realistic uncertainty partitioning between the components
Standardisation of AMFs will improve the overall uncertainty in the measured total O3 columns retrieved from zenith sky UV-visible measurements
  • Operational services and service development (meteorological services, environmental services, Copernicus services C3S & CAMS, operational data assimilation development, etc.)
  • Climate research (research groups working on development, validation and improvement of ECV Climate Data Records)
  • High
  • Medium
Will improve the overall accuracy of the measured total ozone column retrieved from zenith sky UV-visible measurements.
Improving the climatological databases of a priori ozone profiles will improve the accuracy of the RT model calculations of the respective AMFs
  • Climate research (research groups working on development, validation and improvement of ECV Climate Data Records)
  • High
  • Medium
Will improve the overall accuracy of the measured total ozone column retrieved from zenith sky UV-visible measurements.
Including 3D averaging kernels for zenith-sky UV-visible ozone measurements
  • Operational services and service development (meteorological services, environmental services, Copernicus services C3S & CAMS, operational data assimilation development, etc.)
  • Climate research (research groups working on development, validation and improvement of ECV Climate Data Records)
  • High
  • Medium
Improvement in the agreement between the different data sets (different sites as well as satellite/ground-based).
Better agreement between observations at the edge of the polar vortex where the spatial and temporal gradients of the ozone field can be very large.
Identified riskUser category/Application area at riskProbability of risk being realisedImpacts
If a distinct difference remains between realistic uncertainty estimates and the uncertainty calculated by the fitting routines, this will lead to undue confidence in reported data values.
  • Operational services and service development (meteorological services, environmental services, Copernicus services C3S & CAMS, operational data assimilation development, etc.)
  • Climate research (research groups working on development, validation and improvement of ECV Climate Data Records)
  • High
  • Medium
Higher and poorly quantified uncertainty in data products (such as ozone) measured with the DOAS technique leading to reduced utility in applications.
AMFs used by different groups are not standardized.
  • Climate research (research groups working on development, validation and improvement of ECV Climate Data Records)
  • High
  • Medium
Ozone measurements provided by different groups are not homogenized and will likely show some unknown bias from site to site or group to group.
Including 3D averaging kernels for zenith-sky UV-visible ozone measurements
  • Operational services and service development (meteorological services, environmental services, Copernicus services C3S & CAMS, operational data assimilation development, etc.)
  • Climate research (research groups working on development, validation and improvement of ECV Climate Data Records)
  • High
  • Medium
Improvement in the agreement between the different data sets (different sites as well as satellite/ground-based) & better agreement between observations
at the edge of the polar vortex where the spatial and temporal gradients of the ozone field can be very large

Part III Gap remedies

Gap remedies: 

Remedy 1: Improve our understanding of the discrepancy between the calculated fitting uncertainty and the more realistically estimated total random error.

Primary gap remedy type: 
Research
Secondary gap remedy type: 
Technical
Proposed remedy description: 

The proposed action is to improve our understanding of the discrepancy between the calculated fitting uncertainty and the more realistically estimated total random error. This needs to be done, firstly, by evaluating all literature studies and other documentation available on this topic and, secondly, by using the results from the MAX-DOAS intercomparison campaign at Cabauw, the Netherlands, in September 2016, to provide more state-of-the-art data for further investigation specifically tailored to this issue. As part of GAIA-CLIM, we have developed a traceability chain for total column ozone measured by DOAS instruments and as part of this study we investigated, as a case study for two NDACC stations, the individual elements and their respective uncertainties leading up towards the DOAS fitting procedure and the uncertainties calculated within the fitting procedure. This is providing the first step for a quantitative investigation into the observed discrepancies which needs to be further extended e.g. with sensitivity studies of the uncertainties of the single components as well as an investigation of the potential of cancelling out of individual uncertainty components. The existing GAIA-CLIM work needs to be extended to be applicable across the full range of MAX-DOAS instrumentation in usage globally.

Relevance: 

This remedy is specific for measurements using UV-visible spectroscopic measurement techniques and it will address the existing gap by providing a better understanding on what causes the discrepancy between the calculated fitting uncertainty and the more realistically estimated total random uncertainty.

Measurable outcome of success: 

The success will be measured by how much we can improve our understanding of the difference between the individual uncertainty estimates versus the uncertainty provided by the data analysis fitting routines.

Expected viability for the outcome of success: 
  • Medium
Scale of work: 
  • Single institution
  • Consortium
Time bound to remedy: 
  • Less than 3 years
Indicative cost estimate (investment): 
  • Low cost (< 1 million)
Indicative cost estimate (exploitation): 
  • No
Potential actors: 
  • EU H2020 funding
  • Copernicus funding
  • National funding agencies
  • ESA, EUMETSAT or other space agency
  • Academia, individual research institutes

Remedy 2: Improvements to climatological databases of a priori ozone profiles for use in retrievals

Primary gap remedy type: 
Research
Secondary gap remedy type: 
Research
Proposed remedy description: 

Improve climatological databases of a priori ozone profiles, with particular emphasis on tropospheric ozone are required to inform improved retrievals. It is necessary to test the quality/suitability of the databases of ozone profiles through a comparison with ozonesonde profiles at a selection of stations. Preferably this is to be done at the actual measurement site or station where also the UV-visible measurements are made. The vertically high resolved ozonesonde profiles can then be used to validate in particular the tropospheric part of the climatological ozone database. This would then specifically validate and improve the input parameters for the AMF calculation for that specific station. For NDACC stations, for example, which have both measurement techniques on site, this is a very feasible approach. Additionally, ozone profiles measured as part of ozonesonde networks, such as SHADOZ, provide this kind of validation for the currently used climatological database in a more global sense.

Relevance: 

Improving the climatological databases of a priori ozone profiles will improve the accuracy of the a priori data used within the respective RT model to calculate the AMFs and hence to improve the overall accuracy of the measured total ozone column retrieved from zenith sky UV-visible measurements.

Measurable outcome of success: 

If we can show that the updated and improved ozone database, when used as a priori for the ozone AMF calculations, leads to a smaller uncertainty in the calculation of ozone AMFs then we know that we have succeeded.

Expected viability for the outcome of success: 
  • Medium
Scale of work: 
  • Single institution
  • Consortium
Time bound to remedy: 
  • Less than 3 years
Indicative cost estimate (investment): 
  • Low cost (< 1 million)
Potential actors: 
  • EU H2020 funding
  • Copernicus funding
  • National funding agencies
  • ESA, EUMETSAT or other space agency
  • Academia, individual research institutes

Remedy 3: Standardize AMF calculation methods and databases of a-priori information used in AMF calculations to improve the accuracy of the measured total column ozone

Primary gap remedy type: 
Research
Proposed remedy description: 

Differences between AMFs can cause discernible discrepancies between the ozone data sets. For example, some NDACC UV-visible groups use their own individual DOAS settings and ozone AMFs calculated with different RTMs and sets of ozone, pressure and temperature profiles as input data, and with or without latitudinal and seasonal variations. The objective of the recommendations formulated by the NDACC UV-visible WG previously was thus to reduce these discrepancies through the use of standardized DOAS settings and ozone AMF look-up tables that account for the latitudinal and seasonal dependencies of the ozone vertical profile (see Hendrick et al., 2011).

The next step is to review, update and expand these existing tables further by initiating a targeted effort which also incorporates all relevant findings previously attained within projects such as NORS as well as investigations undertaken within GAIA-CLIM. Projects such as FRM4DOAS which are using centralised processing for the ozone data analysis also promote the use of more standardized AMF calculations and databases. With all this in mind, setting up a project to review and investigate the best routines and input variables for the AMF calculations, and to then recalculate and update the NDACC AMF LUTs to be used to homogenise the ozone total column data measured at different locations would be an efficient way forward.

Relevance: 

Standardized AMFs will improve the overall accuracy of the measured total ozone column retrieved from zenith sky UV-visible measurements.

Measurable outcome of success: 

Determine the difference between standardized AMFs and individually calculated ones and, in turn, the difference in the calculated vertical ozone columns. If the standardized AMF lead to smaller uncertainties in the total column ozone datasets we know that the remedy was successful.

Expected viability for the outcome of success: 
  • Medium
Scale of work: 
  • Single institution
  • Consortium
Time bound to remedy: 
  • Less than 3 years
Indicative cost estimate (investment): 
  • Low cost (< 1 million)
Potential actors: 
  • EU H2020 funding
  • Copernicus funding
  • National funding agencies
  • ESA, EUMETSAT or other space agency
  • Academia, individual research institutes

Remedy 4: Evaluation of 3D averaging kernels for zenith-sky UV-visible ozone measurements

Primary gap remedy type: 
Research
Secondary gap remedy type: 
Technical
Proposed remedy description: 

An evaluation of 3D averaging kernels for zenith-sky UV-visible twilight measurements based on AMF look-up tables is needed and a comparison with averaging kernels derived using a direct coupling of the retrieval with the output of a chemistry-transport model, in which the a priori profile used in the AMF calculation is replaced by a more realistic model-derived time and space dependent profile. To tackle this issue further, one or two specific retrieval algorithms coupled with chemistry-transport model output need to be selected to run an in-depth comparison with the averaging kernels retrieved based on the AMF LUTs. An important focus is that the averaging kernel calculated based on the AMF LUTs are representative enough to provide the information expected to add additional value to the actual measurements.

Relevance: 

Many research groups are not setup to run their retrieval code coupled with a chemistry-transport model and so it is essential to have a less computationally demanding approach which can then be used much more widely. Hence it is vital to understand how the uncertainties increase using the method based on the look-up tables and how representative the vertical averaging kernel climatology is of real measurement conditions.

Measurable outcome of success: 

Including 3D averaging kernels for zenith-sky UV-visible ozone measurements in satellite and model validation studies should improve the agreement between the different data sets, especially for UV-visible stations located in winter/spring at the edge of the polar vortex where the spatial and temporal gradients of the ozone field can be very large.

Expected viability for the outcome of success: 
  • Medium
Scale of work: 
  • Single institution
  • Consortium
Time bound to remedy: 
  • Less than 3 years
Indicative cost estimate (investment): 
  • Low cost (< 1 million)
Indicative cost estimate (exploitation): 
  • No
Potential actors: 
  • EU H2020 funding
  • Copernicus funding
  • National funding agencies
  • ESA, EUMETSAT or other space agency
  • Academia, individual research institutes
References: 
  • Eskes, H. J., and Boersma, K. F.: Averaging kernels for DOAS total-column satellite retrievals, Atmos. Chem. Phys., 3, 1285–1291, 2003.
  • Hendrick, F., Pommereau, J.-P., Goutail, F., Evans, R.D., Ionov, D., Pazmino, A., Kyrö, E., Held, G., Eriksen, P., Dorokhov, V., Gil, M., and Van Roozendael, M.: NDACC/SAOZ UV-visible total ozone measurements: improved retrieval and comparison with correlative ground-based and satellite Observations, Atmos. Chem. Phys., 11, 5975–5995, doi:10.5194/acp-11-5975-2011, 2011.
  • Roscoe, H. K., Van Roozendael, M., Fayt, C., du Piesanie, A., Abuhassan, N., Adams, C., Akrami, M., Cede, A., Chong, J., Clémer, K., Friess, U., Gil Ojeda, M., Goutail, F., Graves, R., Griesfeller, A., Grossmann, K., Hemerijckx, G., Hendrick, F., Herman, J., Hermans, C., Irie, H., Johnston, P. V., Kanaya, Y., Kreher, K., Leigh, R., Merlaud, A., Mount, G. H., Navarro, M., Oetjen, H., Pazmino, A., Perez-Camacho, M., Peters, E., Pinardi, G., Puentedura, O., Richter, A., Schönhardt, A., Shaiganfar, R., Spinei, E., Strong, K., Takashima, H., Vlemmix, T., Vrekoussis, M., Wagner, T., Wittrock, F., Yela, M., Yilmaz, S., Boersma, F., Hains, J., Kroon, M., Piters, A., and Kim, Y. J.: Intercomparison of slant column measurements of NO2 and O4 by MAX-DOAS and zenith-sky UV and visible spectrometers, Atmos. Meas. Tech., 3, 1629–1646, doi:10.5194/amt-3-1629-2010, 2010.
  • Vandaele, A. C., Fayt, C., Hendrick, F., Hermans, C., Humbled, F., Van Roozendael, M., Gil, M., Navarro, M., Puentedura, O., Yela, M., Braathen, G., Stebel, K., Tørnkvist, K., Johnston, P., Kreher, K., Goutail, F., Mieville, A., Pommereau, J.-P., Khaikine, S., Richter, A., Oetjen, H., Wittrock, F., Bugarski, S., Frieß, U., Pfeilsticker, K., Sinreich, R., Wagner, T., Corlett, G., and Leigh, R., An intercomparison campaign of ground-based UV-visible measurements of NO2, BrO, and OClO slant columns: Methods of analysis and results for NO2, J. Geophys. Res., 110, D08305, doi:10.1029/2004JD005423, 2005.
  • Van Roozendael, M., Peters, P., Roscoe, H. K., De Backer, H., Jones, A. E., Bartlett, L., Vaughan, G., Goutail, F., Pommereau, J.-P., Kyrö, E., Wahlstrom, C., Braathen, G., and Simon, P. C., Validation of ground-based visible measurements of total ozone by comparison with Dobson and Brewer spectrophotometers, J. Atmos. Chem., 29, 55–83, 1998.