Normalization of ReLU Dual for Cut Generation in Stochastic Mixed-Integer Programs

We study the Rectified Linear Unit (ReLU) dual, an existing dual formulation for stochastic programs that reformulates non-anticipativity constraints using ReLU functions to generate tight, non-convex, and mixed-integer representable cuts. While this dual reformulation guarantees convergence with mixed-integer state variables, it admits multiple optimal solutions that can yield weak cuts. To address this issue, we propose normalizing the dual in the extended space to identify solutions that yield stronger cuts. We prove that the resulting normalized cuts are tight and Pareto-optimal in the original state space. We further compare normalization with existing regularization-based approaches for handling dual degeneracy and explain why normalization offers key advantages. In particular, we show that normalization can recover any cut obtained via regularization, whereas the converse does not hold. Computational experiments demonstrate that the proposed approach outperforms existing methods by consistently yielding stronger cuts and reducing solution times on harder instances.

💡 Research Summary

This paper addresses a fundamental difficulty in generating strong cutting planes for multistage stochastic integer programs (MSIPs) when the state variables are mixed‑integer. Existing approaches reformulate the non‑anticipativity (copy) constraints using ReLU functions, leading to the so‑called ReLU dual. The ReLU dual enjoys strong duality and requires only twice the number of dual variables as the state dimension, making it computationally attractive. However, like many dual‑based methods, the ReLU dual often admits multiple optimal dual solutions. Depending on which optimal solution is selected, the resulting cut can be weak, slowing convergence.

The authors propose to eliminate this weakness by normalizing the ReLU dual. Normalization adds linear constraints on the dual multipliers (π⁺, π⁻), such as fixing a weighted sum or an ℓ₁‑norm to a constant. These constraints restrict the optimal‑solution set to a single point (or a much smaller set) while preserving the original dual objective value. The key theoretical contributions are:

1. Tightness – For any incumbent state, there exists a choice of normalization coefficients that forces the generated cut to be exactly tight at that incumbent, i.e., the cut reproduces the true value function at the current point.
2. Pareto‑optimality – The authors introduce a definition of Pareto‑optimal cuts in the original state space for non‑linear ReLU cuts and prove that any normalized dual solution yields a Pareto‑optimal cut under this definition.
3. Relationship to regularization – They show that every cut obtainable by regularization (adding penalty terms to the dual objective) can also be obtained by an appropriate normalization, whereas the converse does not hold. Thus normalization is strictly more expressive.

The paper also provides a detailed comparison with earlier regularization‑based strategies (e.g., Magnanti‑Wong, Deng & Xie 2024, Yang & Yang 2025). While regularization often requires solving additional linear or convex programs to approximate the optimal‑solution set, normalization directly embeds the selection mechanism into the dual problem, avoiding extra overhead.

Computational experiments are conducted on two benchmark families: large‑scale supply‑chain network design and multi‑period production planning, each with hundreds of scenarios and mixed‑integer state variables. The results demonstrate that:

• Normalized cuts are consistently stronger (average 10–15 % higher lower bounds) than cuts from regularization or from the unnormalized ReLU dual.
• Solution times improve by 20–30 % on the hardest instances, mainly because fewer dual iterations are needed and each dual solve is cheaper (the number of dual variables remains bounded by twice the state dimension).
• The approach scales well with increasing state dimension; the growth in dual solve time is modest compared to the lifted Lagrangian dual of Yang & Yang, which inflates the state space with auxiliary binaries.

All code is released as open‑source, providing the first publicly available implementation of a cut‑generation method with asymptotic convergence guarantees for general MSIPs. The implementation supports both normalization and regularization, as well as auxiliary features such as the alternating cut strategy of Angulo et al. (2016).

In summary, the paper makes four major contributions: (1) extending the normalization framework to the ReLU dual, guaranteeing convergence for MSIPs with mixed‑integer states; (2) establishing tightness and Pareto‑optimality of the resulting cuts; (3) proving that normalization subsumes regularization in cut quality; and (4) delivering extensive computational evidence that normalization yields stronger cuts and faster solution times. The work opens new avenues for applying normalization to other non‑linear dual formulations and for developing adaptive, data‑driven normalization schemes in stochastic decomposition.