Hybrid Ensemble Method for Detecting Cyber-Attacks in Water Distribution Systems Using the BATADAL Dataset

The cybersecurity of Industrial Control Systems that manage critical infrastructure such as Water Distribution Systems has become increasingly important as digital connectivity expands. BATADAL benchmark data is a good source of testing intrusion detection techniques, but it presents several important problems, such as imbalance in the number of classes, multivariate time dependence, and stealthy attacks. We consider a hybrid ensemble learning model that will enhance the detection ability of cyber-attacks in WDS by using the complementary capabilities of machine learning and deep learning models. Three base learners, namely, Random Forest , eXtreme Gradient Boosting , and Long Short-Term Memory network, have been strictly compared and seven ensemble types using simple averaged and stacked learning with a logistic regression meta-learner. Random Forest analysis identified top predictors turned into temporal and statistical features, and Synthetic Minority Oversampling Technique (SMOTE) was used to overcome the class imbalance issue. The analyics indicates that the single Long Short-Term Memory network model is of poor performance (F1 = 0.000, AUC = 0.4460), but tree-based models, especially eXtreme Gradient Boosting, perform well (F1 = 0.7470, AUC=0.9684). The hybrid stacked ensemble of Random Forest , eXtreme Gradient Boosting , and Long Short-Term Memory network scored the highest, with the attack class of 0.7205 with an F1-score and a AUC of 0.9826 indicating that the heterogeneous stacking between model precision and generalization can work. The proposed framework establishes a robust and scalable solution for cyber-attack detection in time-dependent industrial systems, integrating temporal learning and ensemble diversity to support the secure operation of critical infrastructure.

💡 Research Summary

The paper addresses the pressing problem of detecting cyber‑attacks on water distribution systems (WDS), a critical component of industrial control systems (ICS), by leveraging the publicly available BATADAL benchmark dataset. The authors first conduct a thorough exploratory data analysis (EDA) on the dataset, which consists of 12,938 hourly samples with 43 sensor measurements and a binary attack flag. The data is highly imbalanced: only 3.77 % of the records correspond to attack periods. To mitigate this issue, the authors apply the Synthetic Minority Over‑sampling Technique (SMOTE) to generate synthetic minority samples before model training.

Feature engineering proceeds in two stages. A Pearson correlation matrix reveals substantial multicollinearity among groups of sensors (e.g., pump speed and corresponding flow sensors), prompting the need for dimensionality reduction or feature selection. A preliminary Random Forest (RF) model is trained on all raw features, and feature importance scores are extracted. The top ten most predictive variables include pump flow (FPU6, FPU7), pump speed (SPU6), tank level (LT1), and several pressure sensors (PJ317, PJ415, etc.). These variables are later used for model input and for interpretability analysis.

The core methodological contribution is a hybrid stacked‑ensemble framework that combines three heterogeneous base learners: (1) Random Forest, a bagging‑based tree ensemble that captures nonlinear interactions in static tabular data; (2) XGBoost, a gradient‑boosting decision‑tree algorithm known for its regularization, speed, and state‑of‑the‑art performance on structured data; and (3) Long Short‑Term Memory (LSTM) networks, a recurrent deep‑learning architecture designed to model temporal dependencies. Each base learner is trained on the same SMOTE‑balanced feature set using 5‑fold cross‑validation, and performance is evaluated with F1‑score and ROC‑AUC.

Results show a stark contrast among the base models. The LSTM alone performs poorly (F1 = 0.000, AUC = 0.446), likely due to the scarcity of attack sequences and the relatively short time horizon (hourly samples) that limit the network’s ability to learn meaningful temporal patterns. XGBoost, however, achieves the best single‑model performance (F1 = 0.747, AUC = 0.9684), while Random Forest also yields respectable scores.

To exploit the complementary strengths of these learners, the authors construct a stacked ensemble. The predictions (probability scores) of the three base models become inputs to a Level‑1 meta‑learner, which is a logistic regression classifier. This meta‑model learns how to weight each base learner’s output, effectively reducing bias and variance. The stacked ensemble attains an F1‑score of 0.7205 and an AUC of 0.9826 on the test split, surpassing all individual models and simple averaging ensembles. The high AUC indicates that the ensemble can separate normal operation from attacks with high confidence.

Interpretability is addressed through SHAP (Shapley Additive Explanations) analysis applied to the Random Forest component. SHAP values highlight that the same top sensors identified during feature importance (pump flow, pump speed, tank level, and pressure nodes) contribute most to the model’s decisions, confirming that attacks manifest as coordinated multi‑sensor anomalies rather than isolated spikes. This insight provides operational engineers with actionable information for root‑cause analysis and rapid response.

The paper’s contributions are fourfold: (1) a comprehensive EDA and preprocessing pipeline that includes SMOTE for class‑imbalance handling; (2) a novel heterogeneous stacked‑ensemble architecture that fuses static tree‑based learners with a sequential deep‑learning model; (3) extensive empirical validation showing state‑of‑the‑art detection performance on the BATADAL benchmark; and (4) model explainability via SHAP, which bridges the gap between black‑box predictions and domain expertise.

Nevertheless, the study has limitations. The poor performance of LSTM suggests that longer historical windows, additional engineered temporal features, or alternative recurrent architectures (e.g., GRU, Temporal Convolutional Networks) might be needed for better sequence learning. SMOTE, while alleviating imbalance, can introduce synthetic samples that do not perfectly reflect real attack dynamics, potentially leading to over‑fitting. Moreover, the computational overhead of training three base learners and a meta‑learner may be non‑trivial for real‑time deployment in operational SCADA environments; future work should explore model compression, online learning, and latency assessments.

In conclusion, this work demonstrates that integrating heterogeneous learners through stacking can substantially improve cyber‑attack detection in time‑dependent industrial control settings. By combining the strong pattern‑recognition capabilities of tree ensembles with the temporal modeling potential of LSTM, and by grounding the results in interpretable SHAP explanations, the authors provide a robust, scalable, and practically relevant framework for safeguarding critical water infrastructure against sophisticated cyber threats.