Efficient Policy Adaptation for Voltage Control Under Unknown Topology Changes

Reinforcement learning (RL) has shown great potential for designing voltage control policies, but their performance often degrades under changing system conditions such as topology reconfigurations and load variations. We introduce a topology-aware online policy optimization framework that leverages data-driven estimation of voltage-reactive power sensitivities to achieve efficient policy adaptation. Exploiting the sparsity of topology-switching events, where only a few lines change at a time, our method efficiently detects topology changes and identifies the affected lines and parameters, enabling fast and accurate sensitivity updates without recomputing the full sensitivity matrix. The estimated sensitivity is subsequently used for online policy optimization of a pre-trained neural-network-based RL controller. Simulations on both the IEEE 13-bus and SCE 56-bus systems demonstrate over 90 percent line identification accuracy, using only 15 data points. The proposed method also significantly improves voltage regulation performance compared with non-adaptive policies and adaptive policies that rely on regression-based online optimization methods for sensitivity estimation.

💡 Research Summary

The paper addresses a critical gap in modern distribution‑grid voltage control: the inability of reinforcement‑learning (RL) policies to adapt when the network topology changes. While RL has shown promise for deriving decentralized voltage‑control actions directly from data, most existing works train a policy offline on a fixed network model and then deploy it unchanged. In practice, distribution systems frequently undergo reconfigurations—switch operations for fault isolation, maintenance, or load balancing—that alter the electrical topology and consequently the voltage‑reactive‑power sensitivity matrix. The authors propose a comprehensive online framework that detects topology changes, updates the sensitivity matrix efficiently, and adapts a pre‑trained neural‑network policy in real time.

System Modeling and Problem Formulation
The authors model a radial distribution network as a tree graph and use the linearized DistFlow (LinDistFlow) equations to express voltages as a function of real and reactive power injections. The key quantity for data‑driven control is the sensitivity matrix (X_P), which maps changes in controllable reactive power to voltage deviations. The control objective is to minimize a cumulative cost comprising voltage deviation and reactive‑power usage, while the policy (\pi(v,\theta)) is a monotone neural network whose parameters (\theta) can be updated online.

Topology‑Change Detection
Leveraging the sparsity of switching events (only a few lines change at a time) and the radial structure, the authors derive a simple detection rule. They define the predicted voltage change using the pre‑event sensitivity matrix and compare it with the measured change, obtaining an error vector (e_t). A non‑zero norm of (e_t) signals that either a load variation or a topology change occurred. By examining the error at the next time step, they distinguish the two: if the error persists, a topology change is inferred; otherwise, it is attributed to load variation. This criterion is mathematically proven under the LinDistFlow model and validated on the full nonlinear DistFlow equations.

Line Identification and Partial Sensitivity Update
Once a topology change is detected, the algorithm isolates the affected lines by exploiting the fact that only the rows/columns associated with those lines in the path‑incidence matrices need to be revised. Instead of recomputing the entire (X_P) (which would be (O(N^2))), the method updates only the sub‑matrix linked to the changed lines, reducing computational effort to (O(kN)) where (k) is the number of switched lines (typically very small). The updated sensitivity matrix reflects the new topology and line parameters without a full system identification.

Online Policy Optimization
The refreshed (X_P) is fed into an online policy‑gradient routine. Using the current sensitivity, the algorithm computes the instantaneous cost gradient with respect to the policy parameters and performs a stochastic gradient descent step. Because the sensitivity matrix is now accurate for the current topology, the policy quickly adapts, maintaining voltage regulation performance despite the underlying network change. The approach works with a decentralized policy that only requires local voltage measurements, preserving scalability.

Experimental Validation
Two test systems are used: the IEEE 13‑bus feeder and a realistic 56‑bus Southern California Edison (SCE) network. In the 13‑bus case, only 15 voltage‑control data points are needed to achieve >90 % line‑identification accuracy. In the 56‑bus scenario, the adaptive method reduces the cumulative voltage‑regulation cost by roughly 25 % compared with a static RL policy and cuts the sensitivity‑matrix estimation error by 75 % relative to conventional regression‑based online updates. The authors also compare against non‑adaptive RL and regression‑only adaptation, demonstrating superior speed and accuracy.

Contributions and Impact

1. An efficient, sparsity‑aware topology‑change detection and line‑identification algorithm tailored for radial distribution grids.
2. A partial‑update scheme for the voltage‑reactive‑power sensitivity matrix that avoids full recomputation.
3. Integration of the updated sensitivity into an online policy‑gradient framework, enabling real‑time adaptation of a pre‑trained neural‑network controller.
4. Extensive simulation results confirming the method’s effectiveness on both benchmark and realistic feeders.

Future Directions
The authors suggest extending the approach to unbalanced three‑phase systems, handling limited observability (e.g., using low‑cost voltage sensors instead of full PMU coverage), and providing formal convergence guarantees for the online policy updates. Real‑world field trials are also proposed to validate robustness under measurement noise and communication delays.

Overall, the paper delivers a practical, data‑driven solution that bridges the gap between RL‑based voltage control and the dynamic, reconfigurable nature of modern distribution networks, offering a pathway toward more resilient and autonomous smart‑grid operation.