GIAT: A Geologically-Informed Attention Transformer for Lithology Identification

Accurate lithology identification from well logs is crucial for subsurface resource evaluation. Although Transformer-based models excel at sequence modeling, their “black-box” nature and lack of geological guidance limit their performance and trustworthiness. To overcome these limitations, this letter proposes the Geologically-Informed Attention Transformer (GIAT), a novel framework that deeply fuses data-driven geological priors with the Transformer’s attention mechanism. The core of GIAT is a new attention-biasing mechanism. We repurpose Category-Wise Sequence Correlation (CSC) filters to generate a geologically-informed relational matrix, which is injected into the self-attention calculation to explicitly guide the model toward geologically coherent patterns. On two challenging datasets, GIAT achieves state-of-the-art performance with an accuracy of up to 95.4%, significantly outperforming existing models. More importantly, GIAT demonstrates exceptional interpretation faithfulness under input perturbations and generates geologically coherent predictions. Our work presents a new paradigm for building more accurate, reliable, and interpretable deep learning models for geoscience applications.

💡 Research Summary

The paper introduces the Geologically‑Informed Attention Transformer (GIAT), a novel deep‑learning framework that tightly integrates geological prior knowledge with the self‑attention mechanism of Transformers for lithology identification from well‑log data. Traditional Transformer‑based models achieve high predictive performance on sequential data but suffer from black‑box behavior and a lack of domain constraints, which limits their trustworthiness in subsurface decision‑making. Conversely, geological prior methods such as Category‑Wise Sequence Correlation (CSC) filters provide interpretable templates but are usually applied only as preprocessing steps, failing to influence the learning dynamics of deep networks. GIAT bridges this gap by converting CSC‑derived geological templates into an explicit attention‑bias matrix that is injected directly into the attention score computation.

The methodology consists of three modules: (1) CSC filter learning, where class‑specific filters are trained on labeled well‑log sequences to capture characteristic patterns for each lithology; (2) Geological attention fusion, which convolves the input logs with the learned CSC filters, constructs a geological feature vector for each position, computes a cosine‑similarity matrix S, and transforms S into a bias matrix M; (3) Geologically‑informed self‑attention, where the standard attention score QKᵀ is augmented with M before the softmax, yielding Attention(Q,K,V)=softmax((QKᵀ+M)/√d)·V. This bias steers the model to attend preferentially to positions that are geologically similar, providing a strong inductive bias that regularizes learning and improves interpretability. The network is trained end‑to‑end with a cross‑entropy loss using Adam (lr = 1e‑4) and early stopping.

Experiments were conducted on two challenging datasets: the public Kansas cross‑well dataset and a private, heterogeneous Daqing Oilfield dataset. A strict cross‑well validation scheme left one well completely blind for testing. GIAT was benchmarked against a range of strong baselines, including BiLSTM, ResGAT, vanilla Transformer, ReFormer, and DRSN‑GAF. Evaluation metrics comprised accuracy, precision, recall, Cohen’s κ, and, to assess interpretability faithfulness under input perturbations, Pearson Correlation Coefficient (PCC) and Structural Similarity Index (SSIM) of attention maps after adding bounded Gaussian noise.

Results show that GIAT achieves 94.7 % accuracy (κ = 0.92) on Kansas and 95.4 % accuracy (κ = 0.94) on Daqing, surpassing the previous best DRSN‑GAF by 3.9 % and 6.5 % respectively. Under perturbation, GIAT’s attention maps retain high fidelity (PCC = 0.85, SSIM = 0.82 on Kansas; PCC = 0.88, SSIM = 0.85 on Daqing), improving over ReFormer by up to 31 % in PCC and 18 % in SSIM. Qualitative visualizations demonstrate that GIAT produces smooth, geologically coherent lithology predictions, whereas competing models generate fragmented or discontinuous outputs, especially when noise is introduced.

Ablation studies confirm the critical role of the bias matrix M: removing it drops accuracy by roughly 10–14 % and reduces PCC by over 30 %, indicating that the geological bias is essential for both performance and stable interpretation.

The authors discuss that GIAT’s explicit incorporation of domain knowledge into the attention mechanism offers a new paradigm for geoscience deep learning, combining the expressive power of Transformers with the reliability of expert‑derived constraints. Limitations include reliance on labeled data for CSC filter training and the current focus on single‑modal well‑log inputs. Future work will explore automated prior extraction, multi‑scale geological information, and extension to multimodal seismic‑well data fusion.

In summary, GIAT sets a new state‑of‑the‑art for lithology identification, delivering superior accuracy and robust, geologically faithful explanations, and opens pathways for broader application of knowledge‑guided attention models in Earth‑science problems.