Day 4

Detailed paper information

Back to list

Paper title Explaining Deep Learning Models for Earth Surface Forecasting
  1. Miguel-Ángel Fernández-Torres Image Processing Laboratory, University of Valencia, Valencia, Spain Speaker
  2. Michele Ronco Image Processing Laboratory, Universitat de València, València, Spain
  3. Vitus Benson Biogeochemical Integration, Max-Planck-Institute for Biogeochemistry, Jena, Germany
  4. Christian Requena Biogeochemical Integration, Max-Planck-Institute for Biogeochemistry, Jena, Germany;
  5. Miguel Mahecha Universität Leipzig
  6. Gustau Camps-Valls University of Valencia
Form of presentation Poster
  • C1. AI and Data Analytics
    • C1.07 ML4Earth: Machine Learning for Earth Sciences
Abstract text Climate change amplifies extreme weather events. Frequency is increasing and intensifying, and the impact location is becoming more and more uncertain. Anticipation is key, and for this accurate forecasting models are urgently needed. Many downstream applications can benefit from them; from vegetation and forest management and assessment to crop yield prediction and biodiversity monitoring. Recently, Earth surface forecasting was formulated as a video prediction task for which deep learning models show excellent performance [Requena-Mesa, 2021]. Here the goal is to forecast Earth surface reflectance with a given time horizon. Predicting surface reflectance helps in detecting and anticipating anomalies and extremes. The approaches include not only the past reflectances but also ingest topography and weather variables at coarser (mesoscale) resolutions.

We are here interested in understanding rather than fitting forecasting models, and thus analyzing standard DL architectures with eXplainable AI models (XAI) [Tuia, 2021; Camps-Valls, 2021]. Our purpose is twofold: 1) to evaluate and improve the performance of existing approaches, analyzing both correct and wrong predicted samples, and 2) to explain and illustrate the output of these models in a more intelligible way for climate and Earth science researchers. In particular, we will study standard pre-trained video prediction models in EarthNet 2021 (e.g. Channel-U-Net, Autoregressive Conditional -Arcon-) [Requena-Mesa, 2021] with integrated gradients, which have already been applied to drought detection [Fernandez-Torres, 2021], or Shapley values [Castro, 2009], among other techniques. This will allow us to derive spatially explicit and temporally resolved maps of salient regions impacting the prediction at Sentinel-2 spatial resolution, as well as a ranked order of input channels and weather variables.

Evaluating and visualizing the saliency maps is an elusive, subjective task though. Besides model visualization, we will study the impacts on vegetation by looking at vegetation indices, which describe the ecosystem state and evolution. We will evaluate both the standard Normalized Difference Vegetation Index (NDVI) time series and the kernel NDVI (kNDVI), which highly correlates with vegetation photosynthetic activity, and consistently improves accuracy in monitoring key parameters, such as leaf area index, gross primary productivity, and sun-induced chlorophyll fluorescence [Camps-Valls, 2021b]. The XAI methods could serve to explain a large portion of the detected impacts in NDVI, and also to provide improved sharper maps and correlations with the kNDVI index, thus suggesting this is a more realistic parameter to monitor changes, impacts and anomalies in vegetation functioning.


[Camps-Valls, 2021] Gustau Camps-Valls, Devis Tuia, Xiao Xiang Zhu, Markus Reichstein (Editors). Deep learning for the Earth Sciences: A comprehensive approach to remote sensing, climate science and geosciences, Wiley & Sons 2021

[Camps-Valls, 2021b] Camps-Valls, Gustau and Campos-Taberner, Manuel and Moreno-Martínez, Álvaro and Walther, Sophia and Duveiller, Gregory and Cescatti, Alessandro and Mahecha, Miguel D. and Muñoz-Marí, Jordi and García-Haro, Francisco Javier and Guanter, Luis and Jung, Martin and Gamon, John A. and Reichstein, Markus and Running, Steven W. A unified vegetation index for quantifying the terrestrial biosphere. Science Advances. American Association for the Advancement of Science (AAAS), Pubs. 7 (9) 2021

[Castro, 2009] Castro, J., Gómez, D., & Tejada, J. (2009). Polynomial calculation of the Shapley value based on sampling. Computers & Operations Research, 36(5), 1726-1730.

[Fernandez-Torres, 2021] Miguel-Ángel Fernández-Torres and J. Emmanuel Johnson and María Piles and Gustau Camps-Valls. Spatio-Temporal Gaussianization Flows for Extreme Event Detection. EGU General Assembly, Geophysical Research Abstracts, Online, 19-30 April 2021 Vol. 23 2021

[Requena-Mesa, 2021] Requena-Mesa, C., Benson, V., Reichstein, M., Runge, J., & Denzler, J. (2021). EarthNet2021: A large-scale dataset and challenge for Earth surface forecasting as a guided video prediction task. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1132-1142).

[Tuia, 2021] Tuia, D. and Roscher, R. and Wegner, J.D. and Jacobs, N. and Zhu, X.X. and Camps-Valls, G. Towards a Collective Agenda on AI for Earth Science Data Analysis, IEEE Geoscience and Remote Sensing Magazine 2021