Soft-Radial Projection for Constrained End-to-End Learning

Integrating hard constraints into deep learning is essential for safety-critical systems. Yet existing constructive layers that project predictions onto constraint boundaries face a fundamental bottleneck: gradient saturation. By collapsing exterior points onto lower-dimensional surfaces, standard orthogonal projections induce rank-deficient Jacobians, which nullify gradients orthogonal to active constraints and hinder optimization. We introduce Soft-Radial Projection, a differentiable reparameterization layer that circumvents this issue through a radial mapping from Euclidean space into the interior of the feasible set. This construction guarantees strict feasibility while preserving a full-rank Jacobian almost everywhere, thereby preventing the optimization stalls typical of boundary-based methods. We theoretically prove that the architecture retains the universal approximation property and empirically show improved convergence behavior and solution quality over state-of-the-art optimization- and projection-based baselines.

💡 Research Summary

The paper tackles a fundamental obstacle in integrating hard constraints into deep learning models, especially for safety‑critical applications. Conventional constructive layers enforce feasibility by orthogonal projection of the network’s raw output onto the constraint set C. While this guarantees that the final decision lies in C, it collapses the entire exterior space onto the lower‑dimensional boundary ∂C. Consequently, the Jacobian of the projection becomes rank‑deficient in directions orthogonal to the active constraints, leading to “gradient saturation”: gradients vanish for infeasible points and optimization stalls near the boundary.

To overcome this, the authors propose Soft‑Radial Projection (SRP), a differentiable re‑parameterization layer that maps any vector u∈ℝⁿ into the interior Int(C) via a radial homeomorphism. The construction proceeds in three steps:

1. Radial (hard) projection q(u). Choose a fixed interior anchor u₀∈Int(C). For any u, draw the ray from u₀ through u and find the farthest point on that ray still inside C. Formally, α*(u)=sup{α∈