Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset

Despite the increasingly successful application of neural networks to many problems in the geosciences, their complex and nonlinear structure makes the interpretation of their predictions difficult, which limits model trust and does not allow scientists to gain physical insights about the problem at hand. Many different methods have been introduced in the emerging field of eXplainable Artificial Intelligence (XAI), which aim at attributing the network s prediction to specific features in the input domain. XAI methods are usually assessed by using benchmark datasets (like MNIST or ImageNet for image classification). However, an objective, theoretically derived ground truth for the attribution is lacking for most of these datasets, making the assessment of XAI in many cases subjective. Also, benchmark datasets specifically designed for problems in geosciences are rare. Here, we provide a framework, based on the use of additively separable functions, to generate attribution benchmark datasets for regression problems for which the ground truth of the attribution is known a priori. We generate a large benchmark dataset and train a fully connected network to learn the underlying function that was used for simulation. We then compare estimated heatmaps from different XAI methods to the ground truth in order to identify examples where specific XAI methods perform well or poorly. We believe that attribution benchmarks as the ones introduced herein are of great importance for further application of neural networks in the geosciences, and for more objective assessment and accurate implementation of XAI methods, which will increase model trust and assist in discovering new science.

💡 Research Summary

This paper addresses the challenge of interpreting neural network predictions in geoscience applications, where the complex and nonlinear structure of these networks often makes their outputs difficult to understand. This limitation hinders model trust and prevents scientists from gaining physical insights into the problems they are studying. To tackle this issue, the authors explore methods within the emerging field of eXplainable Artificial Intelligence (XAI), which aims to attribute a network’s predictions back to specific features in its input domain.

The paper highlights that while benchmark datasets like MNIST and ImageNet are commonly used for assessing XAI methods, they often lack an objective, theoretically derived ground truth for attribution. Furthermore, there is a scarcity of benchmark datasets specifically designed for geoscience problems. To address these gaps, the authors propose a framework based on additively separable functions to generate benchmark datasets for regression problems where the true attribution is known beforehand.

The researchers create a large synthetic benchmark dataset and train a fully connected neural network to learn the underlying function used in simulations. They then compare heatmaps generated by different XAI methods against this ground truth, identifying scenarios where certain XAI approaches perform well or poorly. This approach allows for an objective assessment of various attribution methods and provides insights into their strengths and weaknesses.

The authors argue that such benchmark datasets are crucial for advancing the application of neural networks in geoscience, enabling more accurate and reliable implementation of XAI methods. By improving model trust and facilitating new scientific discoveries, this work contributes to making neural network predictions more interpretable and valuable in the field of geoscience.