Day 4

Detailed paper information

Back to list

Paper title Towards operational annual forest mapping at continental scale using raw Sentinel-2 time series
  1. Martin Ickerott GAF AG Speaker
  2. Sybrand Jacobus Muller GAF AG
  3. André Stumpf GAF AG
  4. Christian Smit GAF AG
  5. Anna Grabatin-Homolka GAF AG
  6. Gernot Ramminger GAF AG
Form of presentation Poster
  • C1. AI and Data Analytics
    • C1.04 AI4EO applications for Land and Water
Abstract text The analysis of Sentinel-2 time series has already proven invaluable for mapping and monitoring the land cover of Europe [1, 2] and has great potential to contribute to monitoring forests in the tropics [e.g. 3, 4]. The implementation of an operational processing system for Sentinel-2 based forest monitoring is subject to several challenges including the need for an accurate analytical framework that is both robust against phenological shifts and cloud cover and scalable in terms of computation and I/O enabling continental wide mapping within an adequate time frame.

The usage of deep learning methods for operational EO applications is becoming more and more popular in recent years. This comprises, for example, the extraction of building footprints with semantic segmentation on VHR images [5], delineation of agricultural field boundaries [6] or land cover mapping with convolutional neural networks in the time domain [2].

While sequential deep learning models such as Recurrent Neural Networks (RNN) are in principle very well suited for the analysis of satellite image time series of arbitrary and varying length, they tend to under- or overfit the training data, which often degrades their performance for real world applications. Despite modifications to RNNs (e.g. long short-term memory – LSTM, Gate recurrent Units – GRU) designed to address such issues, the usage of RNNs for Sentinel-2 time series classification and land cover mapping on the continental or global scale are yet to be operationalized.

Inspired by recent advances in the design of RNNs for the analysis of satellite time series [7] our study explores how multi-layer RNN architectures can be used to classify raw Sentinel-2 time series at high accuracies, while taking certain measures to keep it computationally efficient and suitable for large-scale operational use. We identify three main contributors to overall processing time: loading of images, pre-processing steps (e.g. temporal resampling, which is a commonly applied to satellite image time series for land cover classification) and the actual inference of the land cover class. It is worth noting that – when compared to the pixel-wise inference of time series on a continental scale (i.e. billions of pixels) – model training and hyperparameter optimization is not necessarily a computational bottleneck because we consider rather lightweight RNN architectures.

In our study, we completely skip the pre-processing of the images by making predictions directly on raw Sentinel-2 Level-2A time series. Inference times of RNNs correlate with the length of the time series (i.e. number of satellite images), so considering less satellite images contributes to both decreased inference and download times. We therefore employ scene-filtering methods that automatically select suitable images at the level of sub-units (~20 km) of S-2 granules. The scene filtering method employed strikes a balance between the desire to achieve good coverage for each sub-unit with a suitable number of less clouded images and the need to keep the overall number of Sentinel-2 scenes at a reasonable level (with implications on download and inference time).

The above-mentioned techniques constitute a lightweight processing chain with drastically reduced I/O (when compared to methods where all or most of the available images are loaded from S3 storage) and computation (when compared to approaches where pre-processing steps are employed). We demonstrate that thematic accuracies achieved are comparable to methods that are much greedier in terms of number of images being used and pre-processing steps being applied. The processing chain used in the CLC+ Backbone project to derive a land cover map over Europe with 11 land cover classes [2] serves as a reference (the CLC+ classification processing chain includes loading of all Sentinel-2 bands up to a cloud cover of 80% and a temporal resampling as a pre-processing step before the prediction of the map).

We demonstrate the above method using reference samples largely based off the LUCAS 2018 survey, extended by additional samples acquired during the CLC+ Backbone project. The classes considered for this study are: coniferous trees, deciduous trees, and the background class (i.e. no trees).

[2] Probeck, M., Ruiz, I., Ramminger, G., Fourie, C., Maier, P., Ickerott, M., ... & Dufourmont, H. (2021). CLC+ Backbone: Set the Scene in Copernicus for the Coming Decade. In 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS (pp. 2076-2079). IEEE.
[3] Nazarova, T., Martin, P., & Giuliani, G. (2020). Monitoring vegetation change in the presence of high cloud cover with Sentinel-2 in a lowland tropical forest region in Brazil. Remote Sensing, 12(11), 1829.
[4] Chen, N., Tsendbazar, N. E., Hamunyela, E., Verbesselt, J., & Herold, M. (2021). Sub-annual tropical forest disturbance monitoring using harmonized Landsat and Sentinel-2 data. International Journal of Applied Earth Observation and Geoinformation, 102, 102386.
[5] Sirko, W., Kashubin, S., Ritter, M., Annkah, A., Bouchareb, Y. S. E., Dauphin, Y., ... & Quinn, J. (2021). Continental-Scale Building Detection from High Resolution Satellite Imagery. arXiv preprint arXiv:2107.12283.
[7] Turkoglu, M. O., D'Aronco, S., Wegner, J., & Schindler, K. (2021). Gating Revisited: Deep Multi-layer RNNs That Can Be Trained. IEEE Transactions on Pattern Analysis and Machine Intelligence.