|Paper title||LOOSE: Combining loosely coupled components into coherent architecture|
|Form of presentation||Poster|
The continuously increasing amount of long-term and of historic data in EO facilities in the form of online datasets and archives makes it necessary to address technologies for the longterm management of these data sets, including their consolidation, preservation, and continuation across multiple missions. The management of long EO data time series of continuing or historic missions, with more than 20 years of data available already today, requires technical solutions and technologies which differ considerably from the ones exploited by existing systems.
The ESA project LOOSE (Technologies for the Management of LOng EO Data Time Series) enables investigating, testing and implementing new technologies to support long time series processing.
For specific tasks (such as ingestion, discovery, access, processing, analysis of EO data) a multitude of completely different mature open source components is usually available. LOOSE aims at combining functionally similar solutions from different heritages into one comprehensive framework. LOOSE even supports parallelism in a way that multiple solutions for the identical task are available and the application developer is invited to chose between these different components during implementation (e. g. "GeoServer" versus "EOXServer").
In addition, LOOSE partners extended well-known existing components with new capabilities (=interfaces) to support efficient ingestion, discovery, exploitation optimized access, processing and optimized analysis of EO data timeseries. For example, GeoServer was extended with the capability to handle STAC metadata.
Overall outcome of the project is a "blueprint architecture concept" which focuses on the interfaces between components and takes innovative concepts such as Bulk data retrieval from dedicated archives, OGC's Data Analysis Processing API and Data Cubes offering Discrete Global Grid Systems into consideration (see enclosed viewgraph).
LOOSE partners are DLR (Oberpfaffenhofen), EOX (Vienna), Terrasigna (Bucharest) and Mundialis (Bonn).
The LOOSE system architecture is inspired by the EO Exploitation Platform Common Architecture (EOEPCA) and focuses on the technological evolution of selected services that enable the end-to-end workflow from retrieving long-term archived EO products to the extraction of high-level information based on processed value-added datasets. The architecture and interoperability is evaluated within LOOSE by using different implementations of these services (e.g. EOxServer and GeoServer) and deploying the whole system on two different infrastructures (DLR/LRZ and Mundi/OTC). The complete LOOSE infrastructure is built on Kubernetes and is therefore well transferrable between different cloud providers.
One of the major goals of the system design (see enclosed figure) is to define services (indicated in blue) as functional components with their internal (purple) and external interfaces (yellow).
The validity of the LOOSE blueprint architecture is demonstrated in three different real-world application pilots.
These applications are covering totally different thematic areas:
- Agricultural monitoring (based on Sentinel-1 and -2 data),
- monitoring urbanization globally (also based on Sentinel-1 and -2) and
- supporting fishery in the Black Sea (multi sensor approach, including in situ-data).
The agriculture use case applies Sentinel-1 and -2 time series in combination with land parcel information to monitor agricultural practices can be monitored and verified, e.g. the presence/absence of mowing in grassland, or the occurrence of ploughing during a specific seasonal time window or the rapid growth of vegetative cover during certain time period. In the context of the European Common Agricultural Policy (CAP), subsidy claims from the farmers require an in-depth check for their eligibility.
This use case specifically requires:
- Handling of and operations on very large vector datasets (filtering, buffering, grouping, merging);
- SAR and optical time series profile extraction through aggregation at land parcel level;
- Implementation of specific eligibility checks according to national CAP and LPIS requirements.
DLR’s World Settlement Footprint (WSF) suite determines built-up areas on the basis of Earth Observation data derived from Sentinel-1 GRD strip map datasets, Sentinel-2 and Landsat 4/5/7/8. Determination of the WSF is performed by evaluating high resolution backscatter ratios between different channels. Timeseries analysis of obtained results is necessary to smooth the computed index values with respect to time and to yield reliable values. This enables to detect the urban growth wordwide.
This pilot application aims at user-driven
- production of the required backscatter indices (for selected periods) performed via eWPS ("black box processing") and
- analysis via Data Analysis and Processing API (DAPA).
LOOSE Marine Pilot will process EO data as well as in-situ and numerical models outputs to accurately identify Potential Fishing Zones (PFZ) around Romanian and Bulgarian coastal areas to support efficient fishery.
In the LOOSE blueprint architecture user-driven "black box" processing is evaluated against user-defined "white box" analyses with respect to useability and performance. "Black box" processing refers to applying a pre-defined retrieval algorithm (processor) on the EO raw data where the user only has limited possibilities to influence the processing settings (such as only selecting the processing time period). "Black box" processing is investigated by using the eWPS, which is provided by the partner Terrasigna. In contrast, white box" processing provides the users the ability to provide an algorithm-graph to the LOOSE system by using the openEO / Actinia / GRASS GIS interface provided by LOOSE partner mundialis.
The LOOSE blueprint architecture concept provides all relevant functionalities (ingestion, discovery, processing, analysis) and can therefore be considered as blueprint for Kubernetes-based operational processing systems.