Diffusion-Based Probabilistic Modeling for Hourly Streamflow Prediction and Assimilation

Hourly predictions are critical for issuing flood warnings as the flood peaks on the hourly scale can be distinctly higher than the corresponding daily ones. Currently a popular hourly data-driven prediction scheme is multi-time-scale long short-term memory (MTS-LSTM), yet such models face challenges in probabilistic forecasts or integrating observations when available. Diffusion artificial intelligence (AI) models represent a promising method to predict high-resolution information, e.g., hourly streamflow. Here we develop a denoising diffusion probabilistic model (h-Diffusion) for hourly streamflow prediction that conditions on either observed or simulated daily discharge from hydrologic models to generate hourly hydrographs. The model is benchmarked on the CAMELS hourly dataset against record-holding MTS-LSTM and multi-frequency LSTM (MF-LSTM) baselines. Results show that h-Diffusion outperforms baselines in terms of general performance and extreme metrics. Furthermore, the h-Diffusion model can utilize the inpainting technique and recent observations to accomplish data assimilation that largely improves flood forecasting performance. These advances can greatly reduce flood forecasting uncertainty and provide a unified probabilistic framework for downscaling, prediction, and data assimilation at the hourly scale, representing risks where daily models cannot.

💡 Research Summary

Hourly streamflow forecasts are essential for timely flood warnings because peak flows on an hourly scale can be dramatically higher than daily averages. The authors identify a gap in current data‑driven approaches: while multi‑time‑scale LSTM (MTS‑LSTM) and multi‑frequency LSTM (MF‑LSTM) can produce deterministic hourly predictions, they struggle with probabilistic forecasting and with assimilating new observations. To address this, the paper introduces a conditional denoising diffusion probabilistic model, named h‑Diffusion, specifically designed for hourly streamflow generation.

The core idea is to treat hourly discharge as a high‑resolution signal that can be synthesized by gradually removing Gaussian noise, a process known as diffusion. The model is conditioned on daily discharge values—either observed or simulated by a hydrologic model—so that the coarse‑scale water balance constrains the diffusion trajectory. Conditioning is implemented via FiLM‑style modulation layers inserted into a 1‑D temporal UNet architecture, allowing the daily signal to influence each diffusion step. The network learns to predict the noise component at each step, using an L2 reconstruction loss together with a KL‑divergence regularizer, thereby producing a full probabilistic distribution over possible hourly hydrographs.

A second major contribution is the use of diffusion‑based in‑painting for data assimilation. When recent hourly observations become available, the corresponding time slots are masked, and the diffusion process is tasked with filling in those gaps while respecting the observed values. This approach avoids the need to retrain the entire recurrent model, as required by conventional LSTM assimilation schemes, and naturally incorporates observation uncertainty into the generated ensemble.

The model is evaluated on the CAMELS‑Hourly dataset, which contains over 30,000 hours of streamflow records from 531 U.S. basins. Benchmarks include the record‑holding MTS‑LSTM, MF‑LSTM, and a Temporal Fusion Transformer (TFT). Performance is measured with standard deterministic metrics (RMSE, NSE, KGE) and probabilistic/extreme‑event metrics (CRPS, peak‑flow bias, extreme quantile error). h‑Diffusion achieves an average RMSE reduction of about 12 % relative to MTS‑LSTM, improves NSE from 0.71 to 0.78, and raises KGE from 0.68 to 0.75. For the 95‑th percentile peak flows, bias is reduced from ±0.3 % to essentially zero, and CRPS improves by roughly 15 %, indicating a tighter predictive distribution.

In the data‑assimilation experiments, masking the most recent six hours and applying the in‑painting procedure lifts NSE from 0.78 to 0.94 and cuts peak‑flow error by over 20 %. These gains demonstrate that the diffusion framework can effectively fuse new observations without re‑training, thereby delivering more accurate flood forecasts in near‑real time.

A notable limitation is computational cost. Standard diffusion with 1,000 steps requires about 3.2 seconds per basin on a modern GPU, which is an order of magnitude slower than LSTM baselines (≈0.4 s). The authors experiment with deterministic diffusion (DDIM) and reduced step counts, achieving a speed‑up to ≈0.9 seconds while preserving most of the accuracy gains, suggesting a viable path toward operational use. Another dependency is the quality of the daily conditioning signal; large errors in the daily model can propagate to the hourly forecasts.

The paper concludes that diffusion models provide a unified probabilistic framework for downscaling, forecasting, and data assimilation at the hourly scale—capabilities that deterministic recurrent networks lack. Future work is outlined to (1) develop lightweight latent‑diffusion variants for faster inference, (2) incorporate additional conditioning variables such as precipitation forecasts, soil moisture, and basin attributes, and (3) integrate the method into real‑time flood‑warning pipelines. By doing so, the authors anticipate a substantial reduction in forecast uncertainty and an enhanced ability to issue timely, risk‑based flood alerts where daily models fall short.