Implicit Decision Diagrams

Implicit Decision Diagrams Isaac Rudic h 1 , 2 [0000 − 0002 − 3106 − 1020] and Louis-Martin Rousseau 1 [0000 − 0001 − 6949 − 6014] 1 P olytechnique Mon tréal, Quebec, Canada 2 Carnegie Mellon Univ ersit y , Pittsburgh P A, USA Abstract. Decision Diagrams (DDs) ha ve emerged as a pow erful tool for discrete optimization, with rapidly gro wing adoption. DDs are directed acyclic lay ered graphs; restricted DDs are a generalized greedy heuristic for finding feasible solutions, and relaxed DDs compute combinatorial relaxed bounds. There is substantial theory that lev erages DD-based b ounding, yet the complexit y of constructing the DDs themselves has receiv ed little attention. Standard restricted DD construction requires O ( w log w ) per lay er; standard relaxed DD construction requires O ( w 2 ) , where w is the width of the DD. Increasing w improv es b ound quality at the cost of more time and memory . W e in troduce implicit De cision Diagr ams , storing arcs implicitly rather than explicitly , and reducing per-lay er complexit y to O ( w ) for restricted and relaxed DDs. W e prov e this is optimal: any framework treating state- up date and merge operations as black b o xes cannot do b etter. Optimal complexity shifts the c hallenge from algorithmic ov erhead to lo w-level engineering. W e show how implicit DDs can drive a MIP solver, and release ImplicitDDs.jl , an op en-source Julia solv er exploiting the implemen tation refinemen ts our theory enables. Exp erimen ts demon- strate the solv er outp erforms Gurobi on Subset Sum. Keyw ords: Decision Diagrams · Com binatorial Optimization · Mixed- In teger Programming. 1 In tro duction The remark able success of Mixed-In teger Programming (MIP) solv ers stems not just from theoretical and algorithmic adv ances, but from highly optimized im- plemen tations [18]. Solvers like Gurobi [15], SCIP [1], and HiGHS [16] succ eed in part through lo w-lev el optimization, transforming theory in to practice. While MIP solvers ha v e b enefited from decades of theoretical and engineering refinemen t, Decision Diagrams (DDs) [6, 5] ha v e not. DDs represen t discrete optimization problems as acyclic directed la y ered graphs, where each ro ot-to- terminal path enco des a potential solution. S ince 2016 [4], DD-based techniques ha v e seen rapid adoption, with a 2022 surv ey reviewing o v er 100 pap ers [7]. DD-based solvers include ddo [14, 12, 9, 13], CODD [20], Peel-and-Bound [22, 23], DIDP [19, 3], and HADDOCK [10, 11], each comp etitiv e on sp ecific problem 2 I. Rudich and L.M. Rousseau classes. In general, DDs are often effective for problems that can b e form ulated more compactly as Dynamic Programs than as MIPs [8]. Despite growing adoption, DD-based solv ers remain in their infancy com- pared to MIP solv ers. There is substan tial theory that lev erages DD-based b ounding techniques, yet the complexity of constructing the underlying DDs themselv es has received little atten tion. Addressing this matters: optimal al- gorithmic complexity shifts the challenge from algorithmic design to lo w-lev el engineering. Restricted DDs pro vide primal b ounds by encoding feasible solu- tions; standard construction requires O ( w log w ) per la y er (where w is the width of the DD). Relaxed DDs provide dual b ounds by ov er-appro ximating the feasi- ble region; standard construction requires O ( w 2 ) per la y er. This dependency on w is critical, b ecause in both cases b ound qualit y impro v es with w . This pap er mak es fiv e contributions to the theory of DDs. (1) W e dev elop a framew ork for analyzing DD construction complexity that separates the inherent costs from problem-sp ecific costs. Inherent costs are the costs incurred b y DD construction, independent of the problem b eing solved. (2) W e design implicit De cision Diagr ams , where arcs are stored implicitly , reducing p er-la y er com- plexit y from O ( w log w ) to O ( w ) for restricted DDs and from O ( w 2 ) to O ( w ) for relaxed DDs. (3) W e prov e this construction is optimal: any general framework treating state-up date and merge as black b o xes cannot b eat the O ( w ) factor. (4) F or integer programming (IP), we show that problem-sp ecific costs are subsumed b y inheren t costs. (5) W e sho w ho w implicit DDs can driv e a MIP solv er. W e also contribute to DD practice. Having addressed the problem of optimal DD construction, w e turn to lo w-lev el engineering. W e release ImplicitDDs.jl , a highly optimized op en-source MIP solv er in Julia for problems with b ounded v ariables. The solver accepts mo dels via JuMP , Julia’s standard optimization framew ork. W e v alidate our theoretical claims exp erimen tally , and demonstrate the solv er outp erforms Gurobi on Subset Sum. The pap er pro ceeds as follows. Section 2 and 3 provide general framing. Section 4 analyzes standard DDs. Section 5 presents implicit DDs. Section 6 addresses MIP . Section 7 pro vides experiments, and Section 8 concludes. 2 In teger Programming Notation Our framework is general-purp ose; it can b e used for Constraint Programming as w ell as IP . W e use IP as our running example b ecause its problem-sp ecific costs are subsumed by inheren t costs, providing a clean baseline where only inherent costs matter. Section 6 extends the approac h to MIP . W e assume all IPs hav e ob jectiv es normalized for minimization, and con- strain ts normalized for ≤ , matching our solv er and framework. Let n b e the n um b er of in teger v ariables and m b e the n um ber of constraints. Let A ∈ R m × n b e the constraint co efficien t matrix with en tries a ij . Let c ∈ R n b e the ob jective co efficien ts, and b ∈ R m b e the right-hand side (RHS) v alues. Let lb , ub ∈ Z n b e the low er and upp er b ounds on the v ariables, resp ectiv ely . Our approach requires Implicit Decision Diagrams 3 all v ariables to hav e finite bounds. Thus: min c ⊤ x s.t. Ax ≤ b lb ≤ x ≤ ub x ∈ Z n Let P I P b e an IP with n v ariables { x 1 , . . . , x n } ; S ol ( P I P ) the set of feasible solutions; x ∗ an optimal solution with v alue z ∗ ( P I P ) ; and d ( x i ) the domain of x i . Let K = max i ∈{ 1 ,...,n } | d ( x i ) | b e the size of the largest domain. 2.1 Example W e use the follo wing example P ex throughout: min − 3 x 1 − 2 x 2 − 2 x 3 − x 4 s.t. x 1 + x 4 ≤ 1 x 2 + x 3 ≤ 1 x i ∈ { 0 , 1 } , i = 1 , . . . , 4 2.2 Monotonic Constraint Residuals Our IP-sp ecific approach uses c onstr aint r esiduals that are normalized to b e monotonic, inspired b y b ound strengthening in MIP presolv e [2]. Giv en a con- strain t i ∈ { 1 , . . . , m } , let the monotonic c onstr aint r esidual r i b e the remaining slac k in i when all unfixed v ariables minimize their con tribution to the left hand side (LHS) of i . F or k fixed v ariables { x 1 , . . . , x k } , the residual for i is: r i = b i − k X j =1 a ij x j − n X j = k +1 a ij · ( lb j if a ij > 0 ub j if a ij < 0 (1) The initial r esiduals ( k = 0 ) are: r (0) i = b i − n X j =1 a ij · ( lb j if a ij > 0 ub j if a ij < 0 (2) Assigning x k = v updates residuals recursively: r ( k ) i = r ( k − 1) i + a ik · ( lb k − v if a ik > 0 ub k − v if a ik < 0 (3) Residuals can only decrease (or sta y constan t) as v ariables are fixed. A neg- ativ e residual ( r i < 0 ) indicates constraint i is violated. Henceforth, we refer to monotonic constrain t residuals simply as residuals. The initial residuals for P ex are r (0) = [1 , 1] , since all v ariables contribute 0 at their minima. Given the partial solution x 1 = 1 , the residuals b ecome r = [0 , 1] . 4 I. Rudich and L.M. Rousseau 3 Complexit y Analysis Strategy Our general framework for constructing DDs includes problem-sp ecific functions. T o separate inheren t costs from problem-specific costs, we treat these functions as black boxes. Ho w ev er, there is a minimum cost to using these functions that is inherent to DD construction. W e explicitly identify those as inherent costs. F or example, Upda teSt a te m ust read and write a state, so calling it incurs an inheren t cost prop ortional to the state size. 4 Standard Decision Diagrams This section reviews [4]; the complexit y analysis is original. 4.1 Exact Decision Diagrams A DD for a com binatorial optimization problem P is a directed lay ered acyclic graph with ro ot r and terminal t . An exact DD represents P if every path from r to t encodes a feasible solution to P , and ev ery feasible solution to P is encoded as a path from r to t . A DD is exact for P iff the enco ded solutions equal S ol ( P ) . W e now construct an exact DD for P ex using an approach similar to a decision tree. Initialize ro ot r on lay er 0 with initial residuals r = [1 , 1] and ob jective 0 . F or eac h lay er i = 0 , . . . , n − 1 : (1) for each node in the la y er, create a c hild node for each v alue in d ( x i +1 ) , (2) up date ob jective v alues and residuals, (3) merge no des with identical residuals, k eeping only the minimal ob jectiv e v alue, and (4) remov e infeasible no des (negative residuals). Merging preserves optimality b ecause identical residuals represen t equiv alent constraint states. Finally , feasible no des on lay er n merge into terminal t (state is no w irrelev an t). The shortest r - t path is optimal (Figure 1). This is exp onen tial, and th us generally in tractable. r = [1 , 1] z = 0 [1 , 1] z = 0 [0 , 1] z = − 3 [1 , 1] z = 0 [1 , 0] z = − 2 [0 , 1] z = − 3 [0 , 0] z = − 5 [1 , 1] z = 0 [1 , 0] z = − 2 [1 , − 1] z = − 4 [0 , 1] z = − 3 [0 , 0] z = − 5 [0 , − 1] z = − 7 [ − 1 , 1] z = − 4 [ − 1 , 0] z = − 6 t z ∗ = − 5 root x 1 x 2 x 3 x 4 x i = 1 x i = 0 Fig. 1: Exact decision diagram for P ex . X indicates a constrain t violation. Implicit Decision Diagrams 5 4.2 Restricted Decision Diagrams A r estricte d DD M − for P is a generalized greedy heuristic for finding feasible solutions. Unlik e exact DDs, restricted DDs enco de only a subset of feasible so- lutions [4]. Let S ol ( M ) b e the solutions encoded b y M . F or a restricted DD: S ol ( M − ) ⊆ S ol ( P ) . The optimal path through M − (minim um-cost for mini- mization) is a feasible solution, pro viding an upp er bound for P . Restricted DD construction mirrors exact construction, except each la y er is limited to width w b y heuristically removing no des. Let ℓ ( u ) b e the lay er of no de u , d ( u ) b e the domain of the next v ariable x ℓ ( u )+1 , s ( u ) b e the state v ector of u (whic h includes the ob jectiv e v alue at u , f ( u ) , as well as other problem-specific information), and | s ( u ) | b e the length of s ( u ) . Algorithm 1 formalizes standard restricted DD construction, and Figure 2 sho ws a restricted DD for P ex . r = [1 , 1] z = 0 [1 , 1] z = 0 [0 , 1] z = − 3 [1 , 1] z = 0 [1 , 0] z = − 2 [0 , 1] z = − 3 [0 , 0] z = − 5 [1 , 0] z = − 2 [1 , − 1] z = − 4 [0 , 1] z = − 3 [0 , 0] z = − 5 [0 , − 1] z = − 7 [ − 1 , 1] z = − 4 t z = − 5 [ − 1 , 0] z = − 6 root x 1 x 2 x 3 x 4 x i = 1 x i = 0 Fig. 2: Restricted DD for P ex , w = 3 . X marks infeasible or trimmed no des. Generalized Complexit y Analysis for Restricted DDs W e analyze Al- gorithm 1, which p erforms fiv e op erations p er la y er: (1) child generation, (2) merging, (3) feasibilit y pruning, (4) width limiting, and (5) other. Recall that K is the size of the largest domain of any v ariable, and | s | is the length of the state vector b eing stored at each no de. The highligh ted term is the ineren t cost. Child generation: Creates up to K · w new no des by calling Upda teSt a te for each no de-label pair, requiring O ( | s | ) + C updt p er call, where C updt is the problem-sp ecific cost of Upd a teSt a te . P er la y er: O ( K · w · | s | ) + O ( K · w ) · C updt Merging without hashing: Merges equiv alen t nodes by calling Mergeable on pairs, requiring O ( | s | ) + C mrg p er comparison, where C mrg is the problem- sp ecific cost of Mer geable . Finding all equiv alen t pairs is done b y sorting and requires O ( K · w · log ( K · w )) node comparisons. Per la y er: O ( K · w · | s | · log( K · w )) + O ( K · w · log ( K · w )) · C mrg 6 I. Rudich and L.M. Rousseau Algorithm 1: Standard Restricted Decision Diagrams [4] 1 Input: The root no de r for a giv en P . Problem-specific functions: Upd a teSt a te and Mergeable . 2 Initialize Q ← { r } , Q next ← ∅ 3 while Q  = ∅ do 4 Q next ← ∅ // 1. Child generation 5 foreac h no de u ∈ Q do 6 foreac h lab el l ∈ d ( u ) do 7 Create a new no de v : Q next ← Q next ∪ { v } 8 A dd arc a uv , lab eled l , from u to v ; s ( v ) = Upda teSt a te ( s ( u ) , x ℓ ( u )+1 , l ) 9 end 10 end // 2. Merging 11 (Optional) F or eac h pair of no des where Mer geable ( s ( u ) , s ( v ) ) = true, k eep only the no de with minim um f ( · ) // 3. Feasibility pruning 12 foreac h no de u ∈ Q next do 13 if IsFeasible ( s ( u ) ) = false then 14 Q next ← Q next \{ u } 15 end 16 end // 4. Width limiting 17 if | Q next | > w then 18 Sort u ∈ Q next b y increasing ob jective v alue ( f ( u ) ) 19 T rim Q next to length w b y removing the last elements 20 end // 5. Additional operations 21 (Optional) Apply additional op erations (e.g., rough b ounding [13]) 22 Q ← Q next 23 end 24 return M − Merging with hashing: F or man y problems, nodes can be group ed by hashing instead of sorting, requiring O ( | s | ) op erations p er no de [19]. Matching hashes define equiv alence. This eliminates the problem-sp ecific cost of calling Mer geable . P er la y er: O ( K · w · | s | ) F easibility pruning: Calls IsFeasible on each no de, requiring O ( | s | )+ C isf p er node, where C isf is the problem-sp ecific cost of IsFeasible . P er la y er: O ( K · w · | s | ) + O ( K · w ) · C isf Width limiting: Sorts no des b y ob jective v alue and trims to w . P er l a yer: O ( K · w · log ( K · w )) Implicit Decision Diagrams 7 A dditional op erations: Optional op erations with cost C othr (excluded). The inheren t framew ork cost is the component without an y problem-sp ecific costs ( C updt , C mrg , C isf , C othr ). F or n lay ers: Merging without hashing: O  n · K · w · | s | · log( K · w )  + O ( n · K · w ) ·  C updt + C isf + O  log( K · w )  · C mrg  Merging with hashing (or without merging): O ( n · K · w · | s | ) + O ( n · K · w · log( K · w )) + O ( n · K · w ) · ( C updt + C isf ) O  n · K · w · max  | s | , log ( K · w )   + O ( n · K · w ) · ( C updt + C isf ) The goal is to maximize w while maintaining tractability . When log ( K · w ) > | s | , b oth hashing and no merging ac hiev e O ( n · K · w · log( K · w )) . The complexit y is dominated by width limiting; scaling to larger w requires a different approach. Application to Integer Programs F or P I P , s ( u ) = r ( u ) , where r ( u ) is the residual v ector at u , so | s | = m . Upda teSt a te computes residuals using Equa- tion 3, O ( m ) . IsFeasible c hec ks if all residuals are non-negativ e O ( m ) . Resid- uals can use hash-based merging (no problem-sp ecific cost). The total problem- sp ecific costs are O ( m ) . When log( K · w ) > m , problem-specific costs are domi- nated b y inheren t costs; otherwise they are subsumed. 4.3 Relaxed Decision Diagrams A r elaxe d DD M + pro vides a low er (relaxed) b ound by enco ding all feasible solu- tions plus infeasible ones: S ol ( P ) ⊆ S ol ( M + ) [4]. T wo standard approac hes are top-down mer ging and c onstruction by sep ar ation , both in O ( w 2 ) time. Section 5.2 uses separation, so w e presen t it here; top-do wn is in App endix A.1. Construction b y separation starts with a w =1 DD and splits nodes. Three new op erations: MergeSt a tes ( s ( u ) , s ( v )) , SelectNodeToSplit ( L ) , and P ar- titionAr cs ( u ) . F or arc a , let p ( a ) and l ( a ) b e its parent and lab el. Algorithm 2 formalizes this; Figure 3 demonstrates it for P ex . Complexit y Analysis Algorithm 2 has tw o phases. Phase 1: Initial w =1 DD: F or each lay er, creates single no de v with O ( K ) in-arcs (one p er paren t label) and computes merged state. Per la y er: O ( K · | s | ) + O ( K ) · ( C updt + C mrg ) Phase 2: F easibility Pruning and No de Splitting: Arc feasibilit y pruning chec ks up to K · w arcs, calling Upd a teSt a te and IsFeasible for eac h  O ( K · w · | s | ) + O ( K · w ) · ( C updt + C isf )  . 8 I. Rudich and L.M. Rousseau r = [1 , 1] z = 0 [1 , 1] z = 0 [0 , 1] z = − 3 [1 , 1] z = − 2 [0 , 1] z = − 3 [0 , 0] z = − 5 [1 , 1] z = − 4 [0 , 0] z = − 5 [0 , 0] z = − 5 infeasible t z = − 5 infeasible root x 1 x 2 x 3 x 4 x i = 1 x i = 0 Fig. 3: Relaxed DD for P ex , w = 3 . T erminal z is a low er bound. No de splitting performs up to w − 1 splits  O ( w )  . Eac h split: (1) Calls SelectNodeToSplit ( C selspl ). (2) Calls P ar titionAr cs ( C part ) to partition in-arcs in to sets I u and I u ′ ,  O ( | I u | + | I u ′ | ) + C part  . In the w orst case the node with the most in-arcs is selected, and one arc is separated. So the first split pro cesses O ( K · w ) arcs, the next O ( K · w ) − 1 = O ( K · w ) arcs, and the last O ( K · w ) − ( w − 2) = O ( K · w ) arcs. Thus, in the worst case O ( | I u | + | I u ′ | ) = O ( K · w ) . (3) Redirects in-arcs: Arcs in I u ′ redirected from u to u ′ (subsumed b y step (2)). (4) Copies out-arcs: No de u has up to K out-arcs  O ( K )  . (5) Recomputes states: Computes merged states from the partitioned in-arc sets. The worst case from step (2) (splitting one arc p er iteration), applies here. Thus this step requires O ( K · w · | s | ) + O ( K · w ) · ( C updt + C mrg ) . The ov erall phase 2 p er la y er cost: O ( K · w 2 · | s | ) + O ( w ) · ( C selspl + C part ) + O ( K · w ) · C isf + O ( K · w 2 ) · ( C updt + C mrg ) Phase 2 dominates phase 1. The o v erall complexit y for n la y ers: n ·  O ( K · w 2 · | s | ) + O ( w ) · ( C selspl + C part )+ O ( K · w ) · C isf + O ( K · w 2 ) · ( C updt + C mrg )  The O ( w 2 ) factor arises from performing O ( w ) splits, with eac h split recom- puting a merged state from O ( K · w ) in-arcs. This quadratic growth is evident in practice. Application to In teger Programs F or P I P , | s | = m . C updt = O ( m ) (com- puting residuals via Equation 3); C mrg = O ( m ) (elemen t-wise maxim um of residuals); C isf = O ( m ) (c hec king residuals); C selspl = O ( w ) amortized (sorted Implicit Decision Diagrams 9 no des: O ( w log w ) sort, O ( w ) inserts); C part = O ( K · w ) (arc partitioning). So the o v erall complexit y b ecomes: n ·  O ( K · w 2 · m ) + O ( K · w 2 ) + O ( w log w ) + O ( K · w · m ) + O ( K · w 2 · m )  All problem-sp ecific terms are subsumed b y inheren t costs. Algorithm 2: Relaxed DD Construction by Separation [4] 1 Input: Ro ot no de r for a given P . Problem-specific functions: Upda teSt a te , Mer geSt a tes , IsFeasible , SelectNodeToSplit , and P ar titionArcs . // Phase 1: Construct initial w = 1 relaxed DD 2 Initialize L 0 ← { r } 3 for i = 1 to n do 4 Let u be the single node in L i − 1 5 Create single node v in la y er L i 6 A dd arcs from u to v lab eled l for each l ∈ d ( u ) 7 Let l 0 b e the first label in d ( u ) 8 Initialize s ( v ) ← Upd a teSt a te ( s ( u ) , x i , l 0 ) 9 foreac h lab el l ∈ d ( u ) \{ l 0 } do 10 s ( v ) ← Mer geSt a tes ( s ( v ) , Upd a teSt a te ( s ( u ) , x i , l )) 11 end 12 end // Phase 2: Refine via node splitting 13 for i = 1 to n do // Feasibility pruning 14 foreac h ar c a into L i do 15 s a ← Upda teSt a te ( s ( p ( a )) , x i , l ( a )) 16 if not IsFeasible ( s a ) then remo v e a ; 17 end // Node splitting 18 while | L i | < w do 19 u ← SelectNodeToSplit ( L i ) (break lo o p if no b eneficial split) 20 ( I u , I u ′ ) ← P ar titionAr cs ( u ) (partition in-arcs into tw o sets) 21 Create new node u ′ : L i ← L i ∪ { u ′ } 22 Redirect arcs in I u ′ from u to u ′ 23 Cop y all out-arcs of u to u ′ 24 foreac h ( β , I ) ∈ { ( u ′ , I u ′ ) , ( u, I u ) } do 25 Let a 0 b e the first arc in I 26 s ( β ) ← Upd a teSt a te ( s ( p ( a 0 )) , x i , l ( a 0 )) 27 foreac h r emaining ar c a ∈ I do 28 s ( β ) ← Mer geSt a tes ( s ( β ) , Upda teSt a te ( s ( p ( a )) , x i , l ( a ))) 29 end 30 end 31 end 32 end 33 return M + 10 I. Rudich and L.M. Rousseau 5 Implicit Decision Diagrams W e present alternative frameworks for constructing restricted and relaxed DDs, impro ving on the inherent costs from Sections 4.2 and 4.3. Prior w ork on a DD pro cessing algorithm called Peel and Bound [21], introduced implicit DDs that store arc lab els implicitly; w e take this idea to its conclusion by storing the arcs themselv es implicitly . This achiev es substan tially wider DDs than prior w ork. This section parallels Section 4: w e describe and formalize each algorithm, then analyze its complexit y . Since implicit DDs are the pap er’s main contribu- tion, the exp osition is more thorough, and it includes optimalit y proofs. 5.1 Implicit Restricted Decision Diagrams Algorithm 1 generates O ( K · w ) children b efore width limiting, explicitly storing no des that ma y b e discarded. W e av oid creating such no des by discriminating based on an ob jective threshold τ . Histogram Construction Algorithm 3 constructs a histogram o v er arc ob jec- tiv e v alues; it is used for both restricted and relaxed implicit DDs. Let f ( u, l ) b e the ob jective v alue of the arc from u with label l , ( u, l ) . The algorithm finds ( τ min , τ max ) , the min and max arc v alues, partitions [ τ min , τ max ] into O ( w ) bins, and assigns each arc to a bin. The cumulativ e histogram C [ j, l ] gives the num ber of arcs with lab el l in bins ≤ j . Algorithm 3 runs in O ( K · w ) time. T wo-Zone Selectiv e Child Generation Using Algorithm 3, we select thresh- old τ as the largest bin where P l C [ τ , l ] ≤ w . This approximates the w -th small- est arc ob jectiv e v alue in O ( K · w ) time, av oiding O ( K · w · log( K · w )) sorting. Algorithm 4 formalizes a two-zone approac h, pro cessing b oth zones in a single pass: (1) the b aseline zone includes all arcs in bins ≤ τ , con taining at most w arcs; (2) the extr as zone uses a budget to greedily select arcs from bin τ + 1 . This eliminates merging and width limiting from Algorithm 1. When the num ber of feasible arcs exceeds w , this ensures | Q next | = w ; otherwise all arcs are kept. The only tradeoff vs. standard width-limiting is that arcs from bin τ + 1 are selected arbitrarily . Figure 4 demonstrates the pro cedure for P ex . Generalized Complexit y Analysis for Implicit Restricted DDs W e ex- tract the inheren t costs to compare with Algorithm 1. The algorithm p erforms three operations per lay e r: (1) his togram construction and threshold selection, (2) selectiv e c hild generation, and (3) feasibilit y pruning. Histogram construction and threshold selection: Algorithm 3 runs in O ( K · w ) time. In Algorithm 4, τ is found by scanning the w cumulativ e totals  O ( w )  . P er la y er: O ( K · w ) Implicit Decision Diagrams 11 Algorithm 3: Histogram Construction for Arc Ob jectiv e V alues 1 Input: Paren t lay er Q , width limit w , domain size K 2 Output: Bin boundaries ( τ min , τ max ) , cumulativ e histogram C // Pass 1: Compute objective bounds 3 ( τ min , τ max ) ← (min , max) of f ( u, l ) ov er all com binations of u ∈ Q , l ∈ d ( u ) // Pass 2: Build histogram 4 H ← w × K matrix of zeros 5 foreac h no de u ∈ Q do 6 foreac h lab el l ∈ d ( u ) do 7 j ← min  w, 1 + ⌊ f ( u,l ) − τ min τ max − τ min · w ⌋  (with 0 0 := 0 ) 8 H [ j, l ] ← H [ j, l ] + 1 9 end 10 end 11 foreac h lab el l ∈ { 1 , . . . , K } do 12 C [0 , l ] ← 0 13 C [1 , l ] ← H [1 , l ] 14 for j = 2 to w do 15 C [ j, l ] ← C [ j − 1 , l ] + H [ j, l ] 16 end 17 end 18 return ( τ min , τ max , C ) Selectiv e child generation: Ev aluates all O ( K · w ) p oten tial arcs in a single pass, creating at most w children. F or eac h selected arc, Upd a teSt a te creates c hild v  O ( | s | ) + C updt  . P er la y er: O ( K · w + w · | s | ) + O ( w ) · C updt F easibility pruning: Chec ks all O ( w ) child no des for feasibilit y . F or eac h c hild, IsFeasible c hec ks the state  O ( | s | ) + C isf  . P er la y er: O ( w · | s | ) + O ( w ) · C isf A dditional op erations: Optional op erations with cost C othr (excluded). The inherent framew ork cost is the comp onen t that excludes problem-sp ecific costs ( C updt , C isf , C othr ). Comparing implicit DDs (Algorithm 4) to standard DDs (Algorithm 1) for n la y ers: Standard: O  n · K · w · max  | s | , log ( K · w )   + O ( n · K · w ) · ( C updt + C isf ) Implicit: O  n · w · ( K + | s | )  + O ( n · w ) · ( C updt + C isf ) F actoring out O ( n · w ) and subtracting ( C updt + C isf ) : O  K · max  | s | , log ( K · w )   + O ( K ) · ( C updt + C isf ) ≫ O  K + | s |  12 I. Rudich and L.M. Rousseau Algorithm 4: Implicit Restricted Decision Diagrams 1 Input: The root no de r for a giv en P , width limit w , domain size K . Problem-sp ecific functions: Upda teSt a te and IsFeasible . 2 Initialize Q ← { r } , Q next ← ∅ 3 while Q  = ∅ do 4 Q next ← ∅ // 1. Histogram construction and threshold selection 5 ( τ min , τ max , C ) ← Algorithm 3 ( Q, w , K ) 6 τ ← max { j : P l C [ j, l ] ≤ w } // 2. Two-zone selective child generation 7 budget ← w − P l C [ τ , l ] 8 foreac h no de u ∈ Q do 9 foreac h lab el l ∈ d ( u ) do 10 j ← min  w, 1 + ⌊ f ( u,l ) − τ min τ max − τ min · w ⌋  11 if j ≤ τ or ( j = τ + 1 and budget > 0) then 12 Create no de v : Q next ← Q next ∪ { v } 13 s ( v ) ← Upda teSt a te ( s ( u ) , x ℓ ( u )+1 , l ) 14 if j = τ + 1 then budget ← budget − 1 ; 15 end 16 end 17 end // 3. Feasibility pruning 18 foreac h no de v ∈ Q next do 19 if IsFeasible ( s ( v ) ) = false then 20 Q next ← Q next \{ v } 21 end 22 end 23 (Optional) Apply additional op erations (e.g., rough b ounding [13]) 24 Q ← Q next 25 end 26 return M − When log ( K · w ) > | s | , standard incurs O ( K · log( K · w )) from width limiting, while implicit achiev es O ( K + | s | ) , eliminating the log factor. When | s | ≥ log ( K · w ) , the comparison b ecomes O ( K · | s | ) v ersus O ( K + | s | ) , reducing | s | from a m ultiplicativ e factor to an additive factor. Algorithm 4 scales linearly in w ; Algorithm 1 scales as O ( w · log ( w )) . Application to In teger Programs F or P I P , state and op erations matc h Section 4.2, except Upd a teSt a te is called ≤ w times p er la y er, reducing reads from O ( K · w · m ) to O ( w · m ) . As | s | = m , the complexity is: O  n · w · ( K + m )  Implicit Decision Diagrams 13 r  1 1  x 1 − 3 − 2 − 1 0 − 3  0 1  0  1 1  x 2 − 5 − 3 . 3 − 1 . 7 0 − 5  0 0  − 3  0 1  − 2  1 0  0 τ x 3 − 7 − 5 . 3 − 3 . 7 − 2 − 7  0 − 1  − 5  0 0  − 4  1 − 1  − 3 − 2 τ x 4 − 6 − 5 . 7 − 5 . 3 − 5 − 6  − 1 0  − 5  0 0  kept rejected infeasible τ x i = 1 x i = 0 Fig. 4: Implicit restricted DD for P ex with w = 3 . Arcs b elo w threshold τ are k ept (dots); excess arcs are rejected (squares). The gray ticks on the num b er line indicate histogram bin edges. On x 2 , one of four candidates is rejected. On x 3 , one node is rejected arbitrarily from the bin ab o v e τ , and t w o of the k e pt no des are infeasible (X). Though visually leftmost (lo w est ob jective), the rejected node is rightmost in memory order (highest parent index), so the budget expires b efore it is pro cessed. Only one feasible node remains for x 4 . 5.2 Implicit Relaxed Decision Diagrams Relaxed DD construction by separation (Section 4.3) has O ( K · w 2 · | s | ) inherent cost p er lay er b ecause intermediate states are generated explicitly . W e present an alternativ e framew ork with O ( K · w · | s | ) inherent cost p er lay er by using interval- b ase d ar c r epr esentation and thr eshold-b ase d no de splitting . W e first explain the algorithm in detail, then formalize it in Algorithm 6. In terv al-Based Arc Storage Standard construction stores arcs explicitly; we store them as in terv als. W e construct our initial diagram with up to K nodes instead of 1 , suc h that all in-arcs to a no de hav e the same lab el, and the no des are sorted by their asso ciated label. First, feasible domain b ounds: w e define D [ p ] = [ lb p , ub p ] as the constraint-feasible lab el range for children of p , where lb p is the smallest feasible lab el, and ub p is the largest. W e define a subroutine ComputeDomainInter v al to compute these via problem-sp ecific op erations 14 I. Rudich and L.M. Rousseau ( Upd a teSt a te , IsFeasible ). Let idx ( u ) be the index of no de u in its lay er, and l ( u ) b e the label of all the arcs that p oin t to u . Second, in v erted in terv als: each c hild no de with lab el l stores [ ψ ( l ) , ϕ ( l )] , the in terv al of paren t indices where l is feasible. Let the inverte d interval for l b e: ψ ( l ) = min n idx ( p ) : p ∈ L i − 1 , l ∈ D [ p ] o ϕ ( l ) = max n idx ( p ) : p ∈ L i − 1 , l ∈ D [ p ] o This in terv al may include infeasible parents (a v alid relaxation), but no infor- mation is lost; individual arcs can b e verified via IsAr cFeasible , defined via Upd a teSt a te and IsFeasible . In the final DD, each no de u with lab el l stores in-arcs as ψ ( l ) , ϕ ( l ) and out-arcs as lb u , ub u , a v oiding the O ( w ) in-arc p oin ters and O ( K ) out-arc p oin ters used b y standard DDs. Theorem 1 (IP Exactness). F or IPs, the interval r epr esentation is exact: idx ( p ) ∈ [ ψ ( l ) , ϕ ( l )] AND l ∈ D [ p ] ⇐ ⇒ ar c ( p, l ) is fe asible The pro of appears after Lemma 1. Width- K Construction W e require w ≥ K since w e start at width- K . W e iterativ ely construct the width- K DD starting from the ro ot no de r . Given parent la y er L i − 1 with ≤ K no des, lay er L i is constructed as follows: (1) Compute feasible domains: F or each paren t p ∈ L i − 1 , compute D [ p ] = [ lb p , ub p ] via ComputeDomainInter v al . (2) Inv ert in terv als: F or eac h lab el l ∈ d ( x i ) , compute [ ψ ( l ) , ϕ ( l )] . (3) Create child no des: F or eac h lab el l ∈ d ( x i ) where ψ ( l ) and ϕ ( l ) are defined, create a child node with lab el l and inv erted in terv al [ ψ ( l ) , ϕ ( l )] . (4) Up date no de states: F or each no de, compute its state via ComputeSt a te , defined via Upda teSt a te , IsFeasible , and MergeSt a tes . Algorithm 5 formalizes this pro cedure, and Figure 5 (left) sho ws it for P ex . IsAr cFeasible : General and IP Given parent p and lab el l , determine if arc ( p, l ) is feasible. Equiv alent to IsFeasible ( Upd a teSt a te ( s ( p ) , x i , l )) . The complexit y is b ounded b y O ( | s | ) for state access, plus C updt and C isf . F or IPs, once the in terv als are computed, IsArcFeasible is O (1) by Theorem 1. ComputeDomainInter v al : General and IP Giv en parent p on L i − 1 , compute the interv al D [ p ] = [ lb p , ub p ] of feasible v alues for x i from p . This finds: lb p = min { l : IsArcFeasible ( p, l ) } and ub p = max { l : IsArcFeasible ( p, l ) } In general, this requires iterating o v er the domain  O ( K )  , calling IsAr cFea- sible for each v alue. Thus, the complexit y is b ounded b y O ( K · | s | ) + O ( K ) · ( C updt + C isf ) . F or IPs, each constrain t derives a b ound on x i from its residual; taking the tigh test requires O ( | s | ) = O ( m ) . Implicit Decision Diagrams 15 Width- K DD ro ot x 1 x 2 x 3 x 4 r : 1 1 [ ψ , ϕ ] = [1 , 1] 1 1 [1 , 1] 0 1 [1 , 1] 1 1 [1 , 2] 1 0 [1 , 2] 1 1 [1 , 2] 1 0 [1 , 2] t [1 , 2] l = 0 l = 1 Threshold Selection x 2 − 5 − 3 . 3 − 1 . 7 0 2 − 5 2 − 3 1 − 2 1 0 j low = j high x 3 − 5 − 4 − 3 − 2 1 3 − 5 1 − 3 2 − 2 j low j high infeasible: 2 3 x 4 infeasible: 2 3 k l = 0 k l = 1 k = parent idx Width- ω DD ro ot x 1 x 2 x 3 x 4 r : 1 1 [ ψ , ϕ ] = [1 , 1] z = 0 1 1 [1 , 1] 0 0 1 [1 , 1] − 3 1 1 [1 , 2] − 3 1 0 [1 , 1] − 2 0 0 [2 , 2] − 5 1 1 [1 , 2] − 3 0 0 [3 , 3] − 5 1 0 [1 , 1] − 5 t [1 , 3] − 5 Fig. 5: Implicit relaxed DD construction for P ex with w = 3 . The width- K DD (left) is constructed first. Then, for each lay er, we select thresholds (middle) and generate width- w children (righ t). Left: Width- K DD; eac h node shows state and inv erted interv al. Middle: Arcs placed on a num ber line by ob jectiv e v alue; thresholds j low and j hig h determine which arcs are individualized versus merged. F or x 2 , there are four arcs at z ∈ {− 5 , − 3 , − 2 , 0 } with threshold j low = j hig h = − 3 . 3 : the z = − 5 arc will b e individualized, and the remaining arcs will b e merged by lab el. Right: Width- w DD; each no de sho ws state, in verted interv al, and ob jectiv e v alue. F or x 2 , the l = 0 no de with interv al [1 , 2] is the merged no de (the in terv al indicates it came from paren ts 1 and 2). Lemma 1. F or IPs, D [ p ] = { l ∈ Z : IsAr cFeasible ( p, l ) } . Pr o of. Linear constrain ts define a con v ex region, so if lb p and ub p are feasible (non-negativ e residuals), then ev ery in teger l ∈ [ lb p , ub p ] is feasible. ⊓ ⊔ Pr o of (The or em 1). By Lemma 1, l ∈ D [ p ] determines if arc ( p, l ) exists; idx ( p ) ∈ [ ψ ( l ) , ϕ ( l )] determines which no de it connects to. Since interv als are non- o v erlapping, exactly one no de with lab el l con tains eac h parent, so the chec ks together iden tify the unique arc. ⊓ ⊔ ComputeSt a te : General and IP Given inv erted in terv al [ ψ ( l ) , ϕ ( l )] , label l , and parent la y er L i − 1 , compute s ( q ) by merging all feasible parent states. F or eac h p with idx ( p ) ∈ [ ψ ( l ) , ϕ ( l )] , compute Upda teSt a te ( s ( p ) , x i , l ) , c hec k IsFeasible , and if feasible, merge via MergeSt a tes . Let | I | denote the n um b er of parents in the in terv al. The complexity is O ( | I | · | s | ) + O ( | I | ) · ( C updt + C isf + C mrg ) . F or IPs, IsArcFeasible is O (1) b y Theorem 1, so we chec k feasibility 16 I. Rudich and L.M. Rousseau Algorithm 5: Initial Width- K Relaxed DD Construction 1 Input: The root no de r for a giv en P with initial state s ( r ) . 2 L 0 ← { r } 3 M + ← { L 0 } 4 for i = 1 to n do 5 foreac h p ar ent no de p ∈ L i − 1 do 6 D [ p ] = [ lb p , ub p ] ← ComputeDomainInter v al ( p ) 7 end 8 L i ← ∅ 9 foreac h lab el l ∈ d ( x i ) do 10 ψ ( l ) ← min { idx ( p ) : l ∈ D [ p ] } , ϕ ( l ) ← max { id x ( p ) : l ∈ D [ p ] } 11 Create no de q with label l and in v erted interv al [ ψ ( l ) , ϕ ( l )] : L i ← L i ∪ { q } 12 end 13 foreac h no de q ∈ L i with lab el l and inverte d interval [ ψ ( l ) , ϕ ( l )] do 14 s ( q ) ← ComputeSt a te ( L i − 1 , [ ψ ( l ) , ϕ ( l )] , l ) 15 end 16 M + ← M + ∪ { L i } 17 end 18 return M + first, calling Upda teSt a te and MergeSt a tes only for feasible arcs. IP states are residuals up dated via Equation 3, and merged via elemen t-wise maxim um. Efficien t No de Splitting The initial width- K DD has one no de p er domain v alue, eac h storing in v erted interv al [ ψ ( l ) , ϕ ( l )] spanning up to K paren ts. After splitting to width w via threshold-based splitting (b elow), paren t lay ers ma y con tain up to w no des, so each child’s inv erted interv al may span up to w parents. Ho w ev er, our pro cedure ensures that when a no de from the K -width lay er splits, the resulting no des partition the parent la y er into non-ov erlapping in terv als . These interv als span at most w paren ts in total. Thus, ComputeSt a te requires O ( K · w ) parent accesses p er lay er, av oiding the O ( w 2 ) of standard relaxed DDs. Threshold Selection Giv en a threshold τ , each arc with f ( p, l ) < τ b ecomes a no de, while each contiguous sequence of arcs with f ( p, l ) ≥ τ (consecutive paren ts, same label) shares a single no de. Using Algorithm 3, we bin arcs into O ( w ) bins, then compute b ounds on the la y er size for eac h candidate threshold to select a τ yielding ≤ w no des. F or bin b oundary j , consider one of the K no des from Algorithm 5. Let n low b e the num ber of in-arcs to this node in bins < j (arcs that are alwa ys individualized), and n hig h the num ber in bins ≥ j (arcs that may b e group ed). If n hig h = 0 , the no de splits into exactly n low no des (all arcs individualized); if n low = 0 , it do esn’t split. When b oth n low > 0 and n hig h > 0 , the no de coun t is b ounded by [ n low + 1 , n low + min( n low + 1 , n hig h )] , derived as follows. Implicit Decision Diagrams 17 The low er b ound assumes abov e-threshold arcs form one node; the upp er b ound treats individualized arcs as separators creating at most n low + 1 contiguous runs of ab o v e-threshold arcs. Summing o v er all K no des yields global la y er size b ounds for threshold j . Computing these b ounds is O ( K ) for each of the O ( w ) candidate thresholds, totaling O ( K · w ) op erations. W e select j low as the highest bin with upp er bound ≤ w , and j hig h as the highest bin with lo w er b ound ≤ w . Arcs with j < j low are individualized; contiguous runs of arcs with j ≥ j low share a single no de. Arcs with j low ≤ j < j hig h are individualized un til none remain or | L | = w . Figure 5 (middle) demonstrates this for P ex . La y er Construction The tw o passes describ ed ab o v e are formalized in Algo- rithm 6; Figure 5 (right) shows the result for P ex . Our implementation combines them in to a single pass using a budget to track remaining capacity . F or eac h no de, states are computed using ComputeSt a te , then domain in terv als are computed using ComputeDomainInter v al . The in v erted in terv als [ ψ ( l ) , ϕ ( l )] are recomputed from these domain interv als for use in the next lay er. Since eac h of the O ( w ) no des merges O ( K ) parents on a v erage, the cost of calling ComputeSt a te w times is O ( K · w · | s | ) + O ( K · w ) · ( C updt + C isf + C mrg ) . Theorem 2 (Tightness). The c omplexity is tight: Θ ( K · w · | s | ) when Upd a t- eSt a te and Mer geSt a tes ar e black-b ox op er ations over | s | -c omp onent states. Pr o of. When Upd a teSt a te is not a homomorphism with resp ect to Mer geS- t a tes , parent states cannot b e merged b efore up dating. The algorithm must in v ok e Upda teSt a te on each of the O ( K · w ) arcs. Each inv o cation reads Ω ( | s | ) comp onen ts since the output may depend on any input. F or example, Upd a teSt a te ( s, v ) = s · v with Mer geSt a tes ( s 1 , s 2 ) = max( s 1 , s 2 ) yields dif- feren t results dep ending on the order they are applied when v < 0 . ⊓ ⊔ Generalized Complexity Analysis for Implicit Relaxed DDs W e analyze Algorithm 6. The cost of Algorithm 5 is subsumed by Algorithm 6, as Phase 1 is subsumed b y Phase 2 in Algorithm 2. Algorithm 6 p erforms three operations p er la y er: (1) threshold selection, (2) threshold-based splitting, and (3) other. Threshold selection: Algorithm 3 runs in O ( K · w ) time. Computing b ounds l [ j ] and u [ j ] for each bin b oundary requires O ( K · w ) op erations. Finding j low and j hig h via linear scan is O ( w ) . Per la y er: O ( K · w ) La y er construction via threshold-based splitting: No de splitting re- quires tw o passes ov er O ( K · w ) arcs with O (1) work p er arc, creating at most w no des. By Theorem 2, calling ComputeSt a te w times costs O ( K · w · | s | ) + O ( K · w ) · ( C updt + C isf + C mrg ) . Calling ComputeDomainInter v al w times costs O ( K · w · | s | ) + O ( K · w ) · ( C updt + C isf ) . Inv erting domain in terv als to compute [ ψ ( l ) , ϕ ( l )] for eac h label is O ( K · w ) . P er la y er: O ( K · w · | s | ) + O ( K · w ) · ( C updt + C isf + C mrg ) 18 I. Rudich and L.M. Rousseau A dditional op erations: Optional op erations with cost C othr (excluded). Algorithm 6: Implicit Relaxed Decision Diagram Refinement 1 M + ← width- K relaxed DD (Algorithm 5) 2 for i = 1 to n do // Threshold selection 3 ( τ min , τ max , C ) ← Algorithm 3 ( L i − 1 , w, K ) 4 foreac h bin j , no de q ∈ L i do 5 n low ← C [ j − 1 , l ( q )] ; n high ← C [ w , l ( q )] − n low 6 end 7 l [ j ] ← P q ( n low + min(1 , n high )) ; u [ j ] ← P q ( n low + min( n low + 1 , n high )) 8 j low ← max { j : u [ j ] ≤ w } ; j high ← max { j : l [ j ] ≤ w } // Pass 1: individualize safe arcs 9 L i ← ∅ 10 foreac h lab el l ∈ d ( x i ) in sorte d or der do 11 ψ ( l ) ← min { idx ( q ) : l ∈ D [ q ] } ; ϕ ( l ) ← max { idx ( q ) : l ∈ D [ q ] } 12 start ← ∅ 13 foreac h no de p ∈ L i − 1 with idx ( p ) ∈ [ ψ ( l ) , ϕ ( l )] do 14 j ← bin of f ( p, l ) 15 if j < j low then 16 if start  = ∅ then // a node is defined by its parents and label 17 add no de ([ start , idx ( p ) − 1] , l ) ; start ← ∅ 18 end 19 add no de ( idx ( p ) , l ) 20 else if start = ∅ then start ← idx ( p ) ; 21 end 22 if start  = ∅ then add node ([ start , ϕ ( l )] , l ) ; 23 end // Pass 2: fill remaining width 24 foreac h no de q ∈ L i do 25 foreac h ar c ( p, l ( q )) with j low ≤ j < j high do 26 if | L i | ≥ w − 1 then stop ; 27 split q to individualize arc ( p, l ( q )) 28 end 29 end 30 foreac h no de q ∈ L i do 31 s ( q ) ← ComputeSt a te ( L i − 1 , arcs ( q ) , l ( q )) 32 D [ q ] = [ lb q , ub q ] ← ComputeDomainInter v al ( q ) 33 end 34 (Optional) Apply additional op erations (e.g., rough b ounding [13]) 35 M + ← M + ∪ { L i } 36 end 37 return M + Implicit Decision Diagrams 19 The inherent framework cost is the comp onen t that excludes problem-sp ecific costs. F or n lay ers: Separation: O  n · K · w 2 · | s |  + O ( n · w ) · ( C selspl + C part ) + O ( n · K · w ) · C isf + O ( n · K · w 2 ) · ( C updt + C mrg ) T op-down: O  n · K · w · ( K · w + | s | )  + O ( n · K · w ) · ( C updt + C isf + C mrg + C selmr g ) Implicit: O  n · K · w · | s |  + O ( n · K · w ) · ( C updt + C isf + C mrg ) Implicit relaxed DDs (Algorithm 6) outperform relaxed DDs by separation (Algorithm 2). Remo ving common factors and simplifying: O  w  + C selspl + C part + O ( w ) · ( C updt + C mrg ) ≫ O  1  Similarly , Algorithm 6 outp erforms top-down relaxed DDs (App endix A.1, Algorithm 7). Remo ving common factors and simplifying: O  K · w  + C selmr g ≫ O  1  The key improv emen t is the elimination of the w 2 factor. Separation incurs O ( w 2 ) pointer updates during no de splitting; top-down incurs O ( K 2 · w 2 ) arc redirections during merging. Implicit relaxed DDs av oid these quadratic costs b y represen ting arcs as in terv als rather than explicit p oin ters. Corollary 1 (Optimality). Under the assumptions of The or em 2, the inher ent c ost of Algorithm 6 is asymptotic al ly optimal. Pr o of. The framew ork achiev es O ( K · w · | s | ) p er lay er. By Theorem 2, this is una v oidable when up date and merge are blac k-box operations. ⊓ ⊔ Theorem 3 (F easibilit y Lo w er Bound). O ( w ) · C isf fe asibility che cks p er layer ar e unavoidable when IsFeasible is a black-b ox pr e dic ate and the algorithm must distinguish fe asible fr om infe asible no des. Pr o of. Without inv oking IsFeasible , the algorithm cannot determine whether a no de’s state is feasible. A relaxed DD that cannot distinguish feasible from infeasible nodes can compute b ounds, but cannot identify feasible solutions or generate meaningful subproblems for branc h-and-bound. T o distinguish feasibil- it y for w output no des, at le ast w inv ocations are required. ⊓ ⊔ Corollary 2 (Problem-Sp ecific Gap). The gap b etwe en Algorithm 6 and a the or etic al ly p erfe ct algorithm is a factor O ( K ) IsFeasible c al ls p er layer. Pr o of. By Theorem 2, O ( K · w ) · ( C updt + C mrg ) is unav oidable. By Theorem 3, O ( w ) · C isf is unav oidable. The framework achiev es O ( K · w ) · ( C updt + C isf + C mrg ) . Th us, the gap is a factor O ( K ) . ⊓ ⊔ This gap reflects c hec king feasibilit y at arcs rather than no des; we hypothesize that the resulting improv emen t in b ound quality makes this tradeoff worth while. 20 I. Rudich and L.M. Rousseau Application to Integer Programs F or P I P , the state representation and op erations remain identical to Section 4.3. As before | s | = m , C updt = C isf = C mrg = O ( m ) , and all problem-sp ecific op erations are subsumed by inherent cost. The o v erall comp elxit y is: O  n · K · w · m  6 Mixed Integer Programming DDs can drive a MIP solver by pro viding com binatorial relaxations ov er integers while treating con tin uous v ariables separately . 6.1 MIP F orm ulation Consider a MIP with n I in teger v ariables and n C con tin uous v ariables. Partition the constraint matrix A = [ A I | A C ] and ob jective c = [ c I ; c C ] accordingly . Let lb C , ub C b e the contin uous v ariable b ounds. The initial residuals (Equation 2) no w include contin uous con tributions. Define the c ontinuous c ontribution δ C i as: δ C i = n C X j =1 a C ij · ( lb C j if a C ij > 0 ub C j if a C ij < 0 (4) Then r (0) i = b i − δ I i − δ C i , where δ I i is the in teger con tribution (Equation 2). 6.2 DD-Driv en MIP Solving Build the DD ov er integers only , propagating residuals via Equation 3. T erminal residuals enco de slac k after in tegers are fixed, but with δ C still subtracted. R estricte d DDs. Normally the b est solution that reaches the terminal la y er of a restricted DD is the b est solution found by that restricted DD. How ev er for MIP , the con tin uous v ariables hav e to be accoun ted for. Instead of merging the no des on the terminal la y er into one node, they ha v e to b e hadled separately . Eac h terminal no de represents a complete integer assignmen t with residual vector r . T o obtain the true RHS for the contin uous LP , add back δ C : min c ⊤ C x C s.t. A C x C ≤ r + δ C , lb C ≤ x C ≤ ub C (5) If the LP for a particular terminal no de is feasible, the integer assignment plus LP solution yields a feasible MIP solution; otherwise, the integer assignmen t is infeasible. A restricted DD can only b e marked exact if every terminal no de’s LP is c hec k ed. Implicit Decision Diagrams 21 R elaxe d DDs. Merge all terminal no des into one, combining residuals via element- wise maximum. Let r (1) , . . . , r ( k ) b e the residuals b efore merging, and let r mrg i = max j r ( j ) i . Then r mrg ≥ r ( j ) comp onen t-wise for all j . Solve the LP from Equa- tion (5) with r = r mrg to obtain z ∗ LP . Since larger RHS w eak ens the constrain ts, z ∗ LP ( r mrg ) ≤ z ∗ LP ( r ( j ) ) for all j . Thus the shortest ro ot-to-terminal path in the relaxed DD, plus z ∗ LP , is a v alid low er (relaxed) bound on the MIP . Conver genc e. Restricted and relaxed DDs yield upper and lo w er b ounds, resp ec- tiv ely . A no de is exact if all paths to it yield the same state; an exact cutset is a set of exact no des intersecting every ro ot-to-terminal path. DD-based branc h- and-b ound [6, 5] prunes no des whose low er b ound exceeds the b est upp er b ound, and branc hes on exact cutsets. F or MIP , b ounds are t w o-part (in teger contribu- tion plus LP con tribution), but con v ergence follo ws from the same argumen ts. 7 Exp erimen tal Results Our goal is to la y the groundw ork for comp etitiv e DD-based solvers by ensur- ing efficient DD construction. State-of-the-art solvers leverage coun tless tech- niques; we focus purely on what DDs achiev e alone. W e use naive DD-based branc h-and-b ound [6, 5] without initial presolv e; the only presolv e at eac h node is F easibility-Based Bound Tigh tening [2]. This minimal setup isolates DD con- struction cost, directly v alidating our theoretical analysis. T o minimize o v erhead, our implementation preallo cates and reuses DD memory . Exp erimen ts ran on AMD EPYC 9655 nodes (2.7 GHz) with up to 300 GB RAM on a single-thread. 7.1 Width Scaling T o v alidate linear scaling in w , we use random binary knapsac k instances with K = 2 and | s | = m = 1 , isolating the w dep endence and enabling scaling to w = 10 8 within memory limits. Instances are inten tionally hard ( n = 100 , capacit y at 50% of exp ected w eigh t); with a 10-min ute time limit, none are solv ed, maximizing the n um b er of full-width la yer constructions. Figure 6 sho ws p er-la y er construction time, av eraged across all lay ers and 10 instances, for widths from 10 3 to 10 8 . All subroutines scale linearly with w , confirming O ( w ) p er-la y er complexit y . Linear regression yields p er-la y er run- times of 21 . 7 ± 0 . 6 ms p er 10 6 w for restricted DDs ( R 2 = 0 . 988 ) and 34 . 7 ± 0 . 6 ms p er 10 6 w for relaxed DDs ( R 2 = 0 . 994 ). A t w = 10 8 , measured per-lay er times are 2.2 s (restricted) and 3.6 s (relaxed). Memory scales at 2 . 58 GB per 10 6 w ( R 2 > 0 . 999 ); optimalit y gap improv es with larger widths (App endix B). Memory , not time, was the bottleneck for scaling these experime n ts further. 7.2 Domain and State Scaling T o v alidate the K (domain size) and | s | (state size) comp onen ts, we use multi- constrain t generalized knapsack instances with w = 10 4 , n = 100 , and scaling factor f from 1 to 1000 ( K = 2 f , | s | = f ). 22 I. Rudich and L.M. Rousseau (a) Restricted DD subroutines (b) Relaxed DD subroutines Fig. 6: Per-la y er run time vs. width on hard binary knapsac k instances. Figure 7 sho ws p er-la y er construction time, av eraged across all lay ers and 10 instances p er configuration. F or restricted DDs, all subroutines scale linearly with f , confirming O ( K + | s | ) dependence; linear regression yields 3 . 87 ± 0 . 07 ms per unit f ( R 2 = 0 . 989 ). F or relaxed DDs, all subroutines except state com- putation scale linearly , fitting 0 . 31 ms p er unit f ; state computation is O ( K · | s | ) (Theorem 2), fitting 0 . 0025 ms p er f 2 (com bined R 2 = 0 . 986 ). State computa- tion dominates at large f , consistent with O ( K · w · | s | ) total complexity . Memory remains modest ( ∼ 1 GB at f = 1000 ) because allocation dep ends on w · | s | , not K , and our implementation double-buffers state information ( 2 · w · | s | storage rather than n · w · | s | ). Memory scales at 0 . 98 ± 0 . 02 MB p er unit f ( R 2 = 0 . 981 ). (a) Restricted DD subroutines (b) Relaxed DD subroutines Fig. 7: P er-la y er runtime vs. scaling factor f ( K = 2 f , | s | = f ) at fixed w = 10 4 . Implicit Decision Diagrams 23 7.3 Solv er Comparison W e no w compare our solv er against established MIP solv ers on Subset Sum, one of Karp’s original NP-Complete problems [17]. Each instance has n binary v ariables, m = 1 equalit y constraint with co efficien ts sampled uniformly from [1 , 2 b ] , and a hidden feasible solution guaranteeing satisfiability . ImplicitDDs uses w = 10 5 and all solv ers receiv e a 15-min ute time limit. W e test n ∈ { 20 , 25 , 30 , 35 , 40 } and b ∈ { 16 , 20 , 24 , 28 , 32 } bits p er co efficien t, with 40 random seeds p er configuration (1000 instances total). Each instance was giv en to ImplicitDDs (default), ImplicitDDs (reordered) with v ariables sorted by co efficien t magnitude, Gurobi, HiGHS, and SCIP . Figure 8 shows p erformance profiles. SCIP times out on 40% of instances; HiGHS solv es all but one. The other 3 solvers all finish within 1 minute. Gurobi had a geometric mean of 4.99s and a max of 55.2s. ImplicitDDs (default) had a geometric mean of 2.4s, a max of 8.4s, and a median 2.2x sp eedup o v er Gurobi. ImplicitDDs (reordered) had a geometric mean of 0.12s, a max of 1.6s, and a median 45.5x sp eedup o v er Gurobi. ImplicitDDs is a researc h prototype rather than a pro duction solver, and omits queue management heuristics that would b e necessary for general-purp ose b enc hmarking. Subset Sum is a suitable testb ed b ecause it yields hard problems with few v ariables. Queues remain small and construction sp eed is the dominant factor, making the b enc hmark a direct test of our core con tribution. Fig. 8: Performance profile on 1000 Subset Sum instances. 24 I. Rudich and L.M. Rousseau 8 Conclusion Standard Decision Diagrams hav e a fundamental b ottlenec k: O ( w log w ) for re- stricted DDs, O ( w 2 ) for relaxed. Implicit DDs eliminate those b ottlenec ks by reducing b oth to O ( w ) , the theoretical limit. On Subset Sum, our implementa- tion yields a 45.5x median sp eedup o v er Gurobi. Y et construction is merely one comp onen t of a DD-based solver. MIP technol- ogy has matured through decades of refinement in presolv e, branching, cutting planes, decomp osition, and LP solving; DD-based optimization is only b egin- ning this journey . A problem-sp ecific v ariable ordering contributed 20x here; the DD literature offers general alternatives we did not explore. Many MIP staples lac k DD-sp ecific counterparts but offer promising research directions. Among them: presolve, decomposition, and custom LP solvers. The gap b et w een a fast subroutine and a broadly comp etitiv e solv er is wide; this work b egins to close it. A cknow le dgments. This w ork was funded by the Canada Research Chair on Healthcare Analytics and Logistics (CR C-2021-00556). References 1. A c hterberg, T.: Scip: solving constraint integer programs. Mathematical Program- ming Computation 1 (1), 1–41 (Jul 2009) 2. A c hterberg, T., Bixb y , R.E., Gu, Z., Rothberg, E., W eninger, D.: Presolve reduc- tions in mixed in teger programming. INF ORMS J. on Computing 32 (2), 473–506 (Apr 2020) 3. Bec k, J.C., Kuroiwa, R., Lee, J.H.M., Stuck ey , P .J., Zhong, A.Z.: T ransition Dominance in Domain-Independ e nt Dynamic Programming. In: CP 2025. LIPIcs, v ol. 340, pp. 5:1–5:23. Sc hloss Dagstuhl (2025) 4. Bergman, D., Cire, A., v an Hoeve, W.J., Ho ok er, J.: Decision Diagrams for Opti- mization. Springer In ternational Publishing (01 2016) 5. Bergman, D., Cire, A., v an Ho ev e, W.J., Hooker, J.: Discrete optimization with decision diagrams. INF ORMS Journal on Computing 28 , 47–66 (02 2016) 6. Bergman, D., Cire, A.A., Sabharwal, A., Samulo witz, H., Saraswat, V., v an Ho ev e, W.J.: P arallel com binatorial optimization with decision diagrams. In: CP AIOR 2014. pp. 351–367 (2014) 7. Castro, M.P ., Cire, A.A., Bec k, J.C.: Decision diagrams for discrete optimization: A surv ey of recen t adv ances. INFORMS Journal on Computing 34 (4), 2271–2295 (2022) 8. Cire, A.A., v an Ho ev e, W.J.: Multiv alued decision diagrams for sequencing prob- lems. Op erations Research 61 (6), 1259, 1462 (2013) 9. Coppé, V., Gillard, X., Schaus, P .: Decision diagram-based branch-and-bound with cac hing for dominance and suboptimality detection. INF ORMS Journal on Com- puting (2024) 10. Gen tzel, R., Mic hel, L., Hoeve, W.J.v.: Haddock: A language and arc hitecture for decision diagram compilation. In: CP 2020. pp. 531–547. Springer (2020) 11. Gen tzel, R., Mic hel, L., v an Ho ev e, W.J.: Optimization bounds from decision dia- grams in haddo c k. In: CP AIOR 2023. pp. 150–166. Springer (2023) Implicit Decision Diagrams 25 12. Gillard, X.: Discrete optimization with decision diagrams: design of a generic solver, impro ved b ounding techniques, and discov ery of go od feasible solutions with large neigh b orhoo d searc h. Ph.D. thesis, UCL-Université Catholique de Louv ain (2022) 13. Gillard, X., Copp é, V., Schaus, P ., Cire, A.A.: Improving the filtering of branch- and-b ound mdd solver. In: CP AIOR 2021. Springer International Publishing (2021) 14. Gillard, X., Schaus, P ., Coppé, V.: Ddo, a generic and efficient framework for mdd- based optimization. In: IJCAI 2021. pp. 5243–5245 (2021) 15. Gurobi Optimization, LLC: Gurobi Optimizer Reference Man ual (2024) 16. Huangfu, Q., Hall, J.J.: P arallelizing the dual revised simplex metho d. Mathemat- ical Programming Computation 10 (1), 119–142 (2018) 17. Karp, R.M.: Reducibilit y among Combinatorial Problems, pp. 85–103. Springer US, Boston, MA (1972) 18. K o c h, T., Berthold, T., Pedersen, J., V anaret, C.: Progress in mathematical pro- gramming solvers from 2001 to 2020. EUR O Journal on Computational Optimiza- tion 10 , 100031 (2022) 19. Kuroiw a, R., Bec k, J.C.: Domain-indep enden t dynamic programming: Generic state space search for com binatorial optimization. In: ICAPS 2023. vol. 33, pp. 236–244 (2023) 20. Mic hel, L., v an Ho ev e, W.J.: CODD: A decision diagram-based solver for combi- natorial optimization. In: ECAI 2024. F ron tiers in Artificial Intelligence and Ap- plications, vol. 392, pp. 4240–4247. IOS Press (2024) 21. Rudic h, I.: Peel and Bound: Solving Discrete Optimization Problems with Decision Diagrams and Separation. Ph.D. thesis, Polytec hnique Montréal (2024) 22. Rudic h, I., Cappart, Q., Rousseau, L.M.: P eel-and-bound: Generating stronger relaxed b ounds with m ultiv alued decision diagrams. In: CP 2022 (2022) 23. Rudic h, I., Cappart, Q., Rousseau, L.M.: Improv ed p eel-and-bound: Metho ds for generating dual b ounds with multiv alued decision diagrams. Journal of Artificial In telligence Researc h 77 , 1489–1538 (2023) A Relaxed Decision Diagram by T op Do wn Merging W e analyze top-down merging and compare to separation (Section 4.3). A.1 T op-Down Construction via Merging This differs from restricted DD construction (Algorithm 1) only in the metho d of width control: no des are mer ge d rather than remov ed. One new op eration: SelectNodesToMer ge ( Q ) selects t w o nodes. Algorithm 7 formalizes this. Complexit y Analysis W e extract the inherent costs to compare with Algo- rithm 2. The algorithm performs three op erations p er lay er: (1) c hild generation, (2) feasibilit y pruning, and (3) width limiting via merging. Child generation: Identical to Algorithm 1. P er la y er: O ( K · w · | s | ) + O ( K · w ) · C updt F easibility pruning: Iden tical to Algorithm 1. P er la y er: O ( K · w · | s | ) + O ( K · w ) · C isf 26 I. Rudich and L.M. Rousseau Algorithm 7: T op-Down Relaxed Decision Diagrams [4] 1 Input: The root no de r for a giv en P . Problem-specific functions: Upd a teSt a te , IsFeasible , SelectNodesToMerge , and Mer geSt a tes . 2 Initialize Q ← { r } , Q next ← ∅ 3 while Q  = ∅ do 4 Q next ← ∅ // 1. Child generation 5 foreac h no de u ∈ Q do 6 foreac h lab el l ∈ d ( u ) do 7 Create a new no de v : Q next ← Q next ∪ { v } 8 A dd arc a uv , lab eled l , from u to v ; s ( v ) = Upda teSt a te ( s ( u ) , x ℓ ( u )+1 , l ) 9 end 10 end // 2. Feasibility pruning 11 foreac h no de u ∈ Q next do 12 if IsFeasible ( s ( u ) ) = false then 13 Q next ← Q next \{ u } 14 end 15 end // 3. Width limiting via merging 16 while | Q next | > w do 17 ( u, v ) ← SelectNodesToMer ge ( Q next ) 18 Create a new no de β : Q next ← Q next \{ u, v } ∪ { β } 19 Redirect in-arcs of u and v to β 20 s ( β ) ← Mer geSt a tes ( u, v ) 21 end // 4. Additional operations 22 (Optional) Apply additional op erations (e.g., rough b ounding [13]) 23 Q ← Q next 24 end 25 return M + Width limiting via merging: Reduces | Q next | from up to K · w no des to w no des via repeated pairwise merging  O ( K · w ) merges  . Each merge: (1) Calls SelectNodesToMer ge ( C selmr g ). (2) Redirects in-arcs from no des u and v to merged no de β . In the worst case (growing c hain), successive merges handle 2, 3, ..., up to K · w − w + 1 arcs, O ( K · w ) on a v erage. (3) Calls Mer geSt a tes ,  O ( | s | ) + C mrg  . P er la y er: O ( K 2 · w 2 + K · w · | s | ) + O ( K · w ) · ( C selmr g + C mrg ) The O ( K 2 · w 2 ) factor arises from O ( K · w ) merges, each redirecting O ( K · w ) in-arcs on a v erage. F or n lay ers: O ( n · ( K 2 · w 2 + K · w · | s | )) + O ( n · K · w ) · ( C selmr g + C mrg + C updt + C isf ) Implicit Decision Diagrams 27 Application to In teger Programs F or P I P , | s | = m . C updt = O ( m ) (com- puting residuals via Equation 3); C mrg = O ( m ) (elemen t-wise maxim um of residuals); C isf = O ( m ) (c hec king residuals); C selmr g : selecting from K · w can- didates con tributes O ( K · w ) per lay er. So the o v erall complexit y becomes: O ( n · ( K 2 · w 2 + K · w · m )) + O ( n · K 2 · w 2 ) + O ( n · K · w · m ) All problem-sp ecific terms are subsumed b y inheren t costs. A.2 Comparison Comparing inheren t costs, top-do wn (left) vs. separation (righ t): O  n · ( K 2 · w 2 + K · w · | s | )  ? O  n · K · w 2 · | s |  O  ( n · K · w ) · ( K · w + | s | )  ? O  ( n · K · w ) · ( w · | s | )  O  K · w + | s |  ? O  w · | s |  T op-down has an extra K factor; separation has an extra | s | factor. T op-down is faster when K < | s | , and vice v ersa. Since DD solv ers fav or smaller K , we exp ect K < | s | for comp etitiv e instances, sligh tly fa v oring top-do wn. B A dditional Width Scaling Figures Figure 9a shows preallo cated DD memory scaling linearly with w . Figure 9b sho ws optimalit y gap ev olution o v er time for v arious DD widths. (a) Memory vs. width (b) Optimality gap o v er time Fig. 9: Additional width scaling results on hard knapsac k instances.