Scalable Neural Network Verification with Branch-and-bound Inferred Cutting Planes

Recently, cutting-plane methods such as GCP-CROWN have been explored to enhance neural network verifiers and made significant advances. However, GCP-CROWN currently relies on generic cutting planes (cuts) generated from external mixed integer programming (MIP) solvers. Due to the poor scalability of MIP solvers, large neural networks cannot benefit from these cutting planes. In this paper, we exploit the structure of the neural network verification problem to generate efficient and scalable cutting planes specific for this problem setting. We propose a novel approach, Branch-and-bound Inferred Cuts with COnstraint Strengthening (BICCOS), which leverages the logical relationships of neurons within verified subproblems in the branch-and-bound search tree, and we introduce cuts that preclude these relationships in other subproblems. We develop a mechanism that assigns influence scores to neurons in each path to allow the strengthening of these cuts. Furthermore, we design a multi-tree search technique to identify more cuts, effectively narrowing the search space and accelerating the BaB algorithm. Our results demonstrate that BICCOS can generate hundreds of useful cuts during the branch-and-bound process and consistently increase the number of verifiable instances compared to other state-of-the-art neural network verifiers on a wide range of benchmarks, including large networks that previous cutting plane methods could not scale to. BICCOS is part of the $α,β$-CROWN verifier, the VNN-COMP 2024 winner. The code is available at http://github.com/Lemutisme/BICCOS .

💡 Research Summary

The paper addresses a critical bottleneck in neural network verification: the scalability of cutting‑plane methods. Recent work such as GCP‑CROWN has shown that adding linear constraints (cuts) to the verification formulation can dramatically tighten bounds and prune the branch‑and‑bound (BaB) search tree. However, GCP‑CROWN relies on generic cuts generated by an external mixed‑integer programming (MIP) solver. As network size grows, the MIP problem becomes intractable, and the solver fails to produce useful cuts within reasonable time, limiting the approach to relatively small models.

To overcome this limitation, the authors propose a novel methodology called Branch‑and‑bound Inferred Cuts with COnstraint Strengthening (BICCOS). The key insight is that every subproblem that is proved UNSAT (or verified) during BaB carries valuable logical information about the ReLU activation states that led to the proof. If a subproblem is verified because neurons in sets (Z^{+}) (forced to the active regime) and (Z^{-}) (forced to the inactive regime) have been split, then the combination “all neurons in (Z^{+}) are active and all neurons in (Z^{-}) are inactive” cannot occur in any feasible solution. This observation yields a globally valid linear inequality:

\