Day 4

Detailed paper information

Back to list

Paper title Data Management, Operating-on-Demand and Beyond - The Alpine Environmental Data Analysis Centre (AlpEnDAC)
Authors
  1. Johannes Munke Leibniz Supercomputing Centre (LRZ) Speaker
  2. Patrick Hannawald German Aerospace Center (DLR)
  3. Martin Schumann Leibniz Supercomputing Centre (LRZ)
  4. Inga Beck Environmental Research Station Schneefernerhaus (UFS)
  5. Alexander Götz Leibniz Supercomputing Centre (LRZ)
  6. Oleg Goussev
  7. Stephan Hachinger Leibniz Supercomputing Centre (LRZ) Speaker
  8. Jana Handschuh University of Augsburg
  9. Helmut Heller Leibniz Supercomputing Centre (LRZ)
  10. Roland Mair bifa Umweltinstitut GmbH
  11. Till Rehm Environmental Research Station Schneefernerhaus (UFS)
  12. Bianca Wittmann bifa Umweltinstitut GmbH
  13. Sabine Wüst German Aerospace Center (DLR)
  14. Michael Bittner German Aerospace Center (DLR), German Remote Sensing Data Center (DFD)
Form of presentation Poster
Topics
  • Open Earth Forum
    • C5.03 Open Source, data science and toolboxes in EO: Current status & evolution
Abstract text The AlpEnDAC (Alpine Environmental Data Analysis Center – www.alpendac.eu) is a platform with the aim of bringing together scientific data measured on high-altitude research stations from the alpine region and beyond. It provides research data management as well as on-demand analysis and simulation services via a modern web-based user interface. Thus, it supports the research activities of the VAO community (Virtual Alpine Observatory, including the major European alpine research stations – www.vao.bayern.de).

Our contribution gives an overview of our (meta-)data-management, ingest and retrieval capabilities for a Research Data Management (RDM) following FAIR principles (findable, accessible, interoperable, reusable). Furthermore, we give a technical glimpse on AlpEnDAC’s capabilities regarding “one-click” simulations and the integration of satellite data to allow for a side-by-side analysis with in-situ measurements.

We then focus on AlpEnDAC’s on-demand services, which are a principal result of the 2019-2022 development cycle (AlpEnDAC-II). We have implemented Computing-on-Demand (CoD, simulations on a click) and Operating-on-Demand (OoD, remote instrument control, based on measurement events when needed), with more “Service on Demand” (e.g. notifications on measurement events) applications to follow. Data from measurements (or also simulations) are normally ingested via a representational state transfer application programming interface (REST API) into the AlpEnDAC system. This interface is complemented by an asynchronous data-ingest layer, based on a message queue (Apache Kafka) and a series of specialized workers to process the data. For OoD, the data processing path is augmented with an interface to request observations of a FAIM (Fast Airglow IMager) camera, and with an automatic scheduler to optimally execute them. The schedule and the data retrieved according to it remain associated within the AlpEnDAC system, allowing for a complete understanding of the measurement process also in retrospect. All on-demand services are made configurable, as much as possible, via the AlpEnDAC web portal. With these developments, we aim to enable scientists – also the ones with a less computer-centric scope of work – to leverage NRT data collection and processing, as it is already an everyday tool e.g. in the Internet-of-Things sector and in commercial applications.

The AlpEnDAC platform has been using infrastructure of the German Aerospace Center (DLR) and the Leibniz Supercomputing Centre (LRZ), major players in Europe’s data and computing centre landscape. AlpEnDAC-II is funded by the Bavarian State Ministry of the Environment and Consumer Protection.