Cascading Robustness Verification: Toward Efficient Model-Agnostic Certification

Certifying neural network robustness against adversarial examples is challenging, as formal guarantees often require solving non-convex problems. Hence, incomplete verifiers are widely used because they scale efficiently and substantially reduce the cost of robustness verification compared to complete methods. However, relying on a single verifier can underestimate robustness because of loose approximations or misalignment with training methods. In this work, we propose Cascading Robustness Verification (CRV), which goes beyond an engineering improvement by exposing fundamental limitations of existing robustness metric and introducing a framework that enhances both reliability and efficiency. CRV is a model-agnostic verifier, meaning that its robustness guarantees are independent of the model’s training process. The key insight behind the CRV framework is that, when using multiple verification methods, an input is certifiably robust if at least one method certifies it as robust. Rather than relying solely on a single verifier with a fixed constraint set, CRV progressively applies multiple verifiers to balance the tightness of the bound and computational cost. Starting with the least expensive method, CRV halts as soon as an input is certified as robust; otherwise, it proceeds to more expensive methods. For computationally expensive methods, we introduce a Stepwise Relaxation Algorithm (SR) that incrementally adds constraints and checks for certification at each step, thereby avoiding unnecessary computation. Our theoretical analysis demonstrates that CRV achieves equal or higher verified accuracy compared to powerful but computationally expensive incomplete verifiers in the cascade, while significantly reducing verification overhead. Empirical results confirm that CRV certifies at least as many inputs as benchmark approaches, while improving runtime efficiency by up to ~90%.

💡 Research Summary

The paper tackles the longstanding challenge of providing formal robustness guarantees for neural networks against adversarial perturbations, a problem that is NP‑hard due to non‑convexity and high dimensionality. Existing approaches fall into two categories: complete verifiers (e.g., SMT, MILP) that are exact but computationally prohibitive, and incomplete verifiers (e.g., linear programming (LP) and semidefinite programming (SDP) relaxations) that scale well but suffer from loose bounds. Relying on a single incomplete verifier leads to two critical issues: (1) false‑negative certifications caused by overly coarse relaxations, which underestimate the true robust set, and (2) misalignment between the robustness training method and the verification method, which can further degrade certified accuracy.

To address these problems, the authors propose Cascading Robustness Verification (CRV), a model‑agnostic framework that sequentially applies multiple verification methods. The cascade starts with the cheapest, least tight verifier (typically an LP‑based method). Inputs that are certified at this stage are accepted immediately, eliminating any further computation for them. For the remaining inputs, the framework proceeds to more expensive but tighter verifiers (e.g., SDP‑based). Crucially, for each expensive verifier the authors introduce a Stepwise Relaxation (SR) algorithm: verification begins with a coarse set of constraints and iteratively adds tighter constraints only when the current relaxation fails to certify robustness. This prevents unnecessary solving of full‑scale SDP problems for inputs that can already be certified with a lighter relaxation. An accelerated variant, Fast Stepwise Relaxation (FSR), further prunes constraint sets by discarding those with negligible impact on bound quality, thereby achieving additional speed‑ups.

Theoretical analysis shows that CRV never reduces verified accuracy relative to the strongest verifier in the cascade. Because the certification condition is “an input is robust if any verifier in the cascade certifies it,” the cascade’s certified set is the union of the individual verifier sets, guaranteeing at least the same robust accuracy as the most precise verifier while typically requiring far less computation. The authors also prove that SR preserves this guarantee by ensuring that each added constraint only tightens the bound without discarding any previously certified inputs.

Empirical evaluation spans several benchmark datasets (MNIST, CIFAR‑10, SVHN) and network architectures (fully‑connected, convolutional, ResNet‑18). In a representative MNIST experiment with an ℓ∞ budget of 0.3, a standalone LP verifier achieves 22 % verified accuracy in 156 minutes, whereas CRV (LP followed by SDP with SR) reaches 36 % verified accuracy in 72 minutes—a 54 % reduction in runtime and a 14 % absolute gain in certified robustness. Notably, the SDP verifier alone often times out on a subset of inputs, causing unverified cases; CRV mitigates this by handling many inputs already at the LP stage. On deeper models, CRV‑FSR attains up to 90 % runtime reduction compared to a pure SDP approach, with less than 1 % loss in verified accuracy.

In summary, CRV offers a practical, training‑agnostic solution that combines the speed of coarse relaxations with the precision of tight relaxations, delivering equal or higher certified robustness at substantially lower computational cost. The paper suggests future directions such as learning optimal verifier ordering via meta‑learning, extending the cascade to other norm balls (ℓ₂, ℓ₁), and leveraging hardware accelerators for real‑time certification in safety‑critical deployments.