ERF: Energy Research and Forecasting Model

High performance computing (HPC) architectures have undergone rapid development in recent years. As a result, established software suites face an ever increasing challenge to remain performant on and portable across modern systems. Many of the widely adopted atmospheric modeling codes cannot fully (or in some cases, at all) leverage the acceleration provided by General-Purpose Graphics Processing Units (GPGPUs), leaving users of those codes constrained to increasingly limited HPC resources. Energy Research and Forecasting (ERF) is a regional atmospheric modeling code that leverages the latest HPC architectures, whether composed of only Central Processing Units (CPUs) or incorporating GPUs. ERF contains many of the standard discretizations and basic features needed to model general atmospheric dynamics as well as flows relevant to renewable energy. The modular design of ERF provides a flexible platform for exploring different physics parameterizations and numerical strategies. ERF is built on a state-of-the-art, well-supported, software framework (AMReX) that provides a performance portable interface and ensures ERF’s long-term sustainability on next generation computing systems. This paper details the numerical methodology of ERF and presents results for a series of verification and validation cases.

💡 Research Summary

The paper introduces the Energy Research and Forecasting (ERF) model, a new regional atmospheric modeling code designed to fully leverage modern high-performance computing (HPC) architectures, including General-Purpose Graphics Processing Units (GPGPUs). It addresses a critical gap in the community, as widely-used codes like the Weather Research and Forecasting (WRF) model cannot fully utilize GPU acceleration, limiting their performance on increasingly GPU-centric supercomputers.

ERF is built from the ground up as an open-source C++ code on the AMReX software framework, which ensures performance portability across diverse architectures (CPUs, and NVIDIA/AMD/Intel GPUs). This foundational choice guarantees long-term sustainability on next-generation HPC systems. The model supports two core dynamical modes: a fully compressible equation set and an anelastic approximation, providing flexibility for simulating flows across different scales, from mesoscale to turbulence-resolving large-eddy simulations (LES). It employs terrain-following coordinates, allows for grid stretching, and incorporates Adaptive Mesh Refinement (AMR) to efficiently resolve complex topography and localized phenomena.

The numerical methodology uses a finite-volume approach on an Arakawa C-grid in space and a low-storage third-order Runge-Kutta scheme in time for the compressible equations. ERF integrates several subgrid-scale turbulence models (e.g., Smagorinsky, Deardorff TKE for LES; MYNN for PBL parameterizations) and cloud microphysics schemes (e.g., Kessler, Morrison). Crucially, all these components have been implemented to run efficiently on GPUs. Furthermore, ERF includes a Lagrangian particle-tracking capability, enabling simulations of aerosol transport, atmospheric chemistry, or wind-borne debris.

The paper presents a series of verification and validation cases, including idealized boundary layers, terrain-induced flows, and a 3D squall line simulation, demonstrating ERF’s accuracy and capabilities. Performance benchmarks highlight a significant speedup, with GPU-enabled ERF running approximately five times faster than a CPU-only WRF simulation for the squall line case. This underscores the practical benefit of its GPU-aware design.

In summary, ERF represents a modern, performant, and portable atmospheric modeling platform. Its modular design facilitates exploration of different physics parameterizations and numerical strategies. By effectively harnessing GPU acceleration and advanced meshing techniques, ERF aims to enable higher-resolution, larger-domain, and more ensemble-based simulations, thereby advancing research in numerical weather prediction, renewable energy resource assessment, and fundamental atmospheric dynamics.