Day 4

Detailed paper information

Back to list

Paper title The Earth Console platform as an infrastructure solution for interoperability within IDEAS-QA4EO Earth Observation data validation activities
  1. Erminia De Grandis SERCO c/o ESA-ESRIN Speaker
  2. Fabrizio Niro SERCO c/o ESA-ESRIN
  3. Gabriele Brizzi SERCO
  4. Constant MAZERAN Solvo
  5. Massimo Cardaci Serco Italia S.p.A., Frascati, Italy
  6. Giancarlo Rivolta Progressive Systems
  7. Giovanni Sabatino Progressive Systems
Form of presentation Poster
  • Open Earth Forum
    • C5.03 Open Source, data science and toolboxes in EO: Current status & evolution
Abstract text The Instrument Data Evaluation and Analysis Service for Quality Assurance for Earth Observation (IDEAS-QA4EO) provides an operational solution to monitor the quality of Earth Observation (EO) instrument data from a wide range of ESA satellite missions currently in operations. Within the IDEAS-QA4EO service activities, it has emerged the need to promote better interoperability among the different domains and ease the access and exploitation of EO data, notably for Cal/Val activities.
To this end, a demonstrator pilot started in November 2020 with the main objective of implementing a new working environment where to effectively access the data archive, develop new algorithms, and integrate them into a performing processing environment, also with the possibility to upload ancillary and fiducial reference data and share the code and the results in a collaborative environment.

The Earth Console platform, operated by Progressive Systems, is a scalable cloud-based platform encompassing a set of services to support and optimize the use and analysis of EO data. The Earth Console services are available via the ESA Network of Resources (NoR) and interface the CREODIAS platform containing most of the Copernicus Sentinel satellite data and services, as well as Envisat, Landsat, and other EO data. During the user and system requirements analysis for the pilot project, the Earth Console platform has proved to be a very promising infrastructure solution, and the subsequent development and data analysis activities performed on this environment, focused on ad-hoc Cal/Val use cases, have shown interesting results.
This paper presents the main functionalities and data exploitations possibilities of the implemented solution, by illustrating some sample use cases and demonstrating the advantage of such platform for data validation purposes.
In detail, a statistical analysis of Sentinel-2 Bottom-Of-Atmosphere (BOA) reflectances over a subset of globally spread and spatially homogeneous land sites was performed to investigate the spatial-temporal consistency of these operational products and detect any potential land-cover dependent biases. Furthermore, a validation procedure of S2 BOA products has been implemented: the approach, already used in the Atmospheric Correction Intercomparison Exercise (ACIX), consists in building a synthetic surface reflectance dataset around the AERONET ground-based stations; this dataset is computed by correcting satellite Top-Of-Atmosphere (TOA) reflectances using the AERONET atmospheric state variables and an accurate Radiative Transfer Model (RTM).
As part of Sentinel-3/OLCI validation activities, an assessment of the Bright Pixel Correction algorithm has been performed: OLCI Level 1 products have been extracted over specific coastal areas and processed with the BPC processor to produce marine reflectance. The related turbidity maps were then compared with those obtained from operational Level-2 products. Within the same activity, a validation procedure of marine reflectances has been analyzed and its implementation has already started: given a list of in situ radiometric data, the matchups with Sentinel-3/OLCI data are identified and the related L1 products processed with the BPC algorithm, then the obtained marine reflectances are validated with the in-situ measurements.
A similar approach has been followed for the Sentinel-5p products validation activity: the objective is to implement a procedure to validate the operational products with ground truth datasets. To this end, a subset of in-situ measurements (e.g. AERONET, BAQUNIN) have been selected and the matchups with Sentinel-5p identified. Then, the aerosol and trace gases TROPOMI products have been validated against in-situ data extracted over a temporal window centered at Sentinel-5p overpass time.

The use of the Earth Console platform for these exercises allowed accessing the full S2, S3 and S5p archive together with in situ measurements uploaded to the platform for the purpose. In addition, the Jupyter Notebooks developed within these activities have been made available in a public knowledge library with the main purpose to build a collaborative environment for sharing code and results among different users, enriching the collections of available software, tools and ready-to-use notebooks, promoting algorithm development and fostering interoperability among QA4EO service domains.