Real-Time Predictive Control Strategy Optimization

1 Real-T ime Predicti ve Control Strate gy Optimization Samarth Gupta, Ravi Seshadri, Bilge Ataso y , A. Arun Prakash, Francisco Pereira, Gary T an, and Moshe Ben-Aki va Abstract —T raffic congestion has lead to an increasing emphasis on management measures f or a more efficient utilization of existing infrastructure. In this context, this paper pr oposes a novel framework that integrates real-time optimization of control strategies (tolls, ramp metering rates, etc.) with guidance gen- eration using predicted network states for Dynamic T raffic As- signment systems. The efficacy of the framework is demonstrated through a fixed demand dynamic toll optimization problem which is f ormulated as a non-linear pr ogram to minimize pr edicted network trav el times. A scalable efficient genetic algorithm is applied to solve this problem that exploits parallel computing. Experiments using a closed-loop approach are conducted on a large scale r oad network in Singapore to in vestigate the performance of the proposed methodology . The results indicate significant improvements in network wide travel time of up to 9% with real-time computational performance. Index T erms —dynamic toll optimization, dynamic traffic as- signment (DT A), pr edictive contr ol optimization, large-scale net- work, real-time traffic management I . I N T RO D U C T I O N . U RB AN transportation networks are subject to large de- gree of v ariability due to the fluctuating supply and de- mand characteristics. These fluctuations result in the pervasiv e phenomena of recurrent and non-recurrent congestion, which is an escalating problem worldwide. The adverse impacts of the resulting congestion include high trav el delays, high trav el costs, and significant costs to the economy and en vi- ronment. Consequently , there has been an increased emphasis on dev eloping tools to mitigate congestion and efficiently utilize existing infrastructure. In this context, we propose an integrated framework —within a Dynamic T raffic Assignment (DT A) system— to optimize network control strategies in real- time considering network state predictions. Specifically , the generated control strategies are predictive (or proactiv e) as opposed to being just reactiv e. The framework also incorpo- rates the generation of consistent guidance —it ensures that the guidance disseminated considers the travelers response to it, thereby increasing the reliability of the provided information. Further , we demonstrate the effecti veness of the proposed framew ork through a real-world application to the predictiv e optimization of network tolls. The motiv ation for this study is fourfold. First, the need for decision support tools to facilitate a more efficient utilization S. Gupta, A. Prakash and M. Ben-Akiv a are with Massachusetts Institute of T echnology (MIT). { samarthg, arunprak , mba } @mit.edu R. Seshadri is with Singapore-MIT Alliance for Research and T echnology (SMAR T). ravi@smart.mit.edu B. Atasoy is with Delft Uni versity of T echnology (TU Delft). b .atasoy@tudelft.nl G. T an is with the National University of Singapore (NUS). gtan@comp.nus.edu.sg F . Pereira is with the T echnical University of Denmark. camara@dtu.dk of existing infrastructure. Second, most studies on optimal network control do not combine the optimization of network control strategies with the generation of guidance information. The third motiv ating factor is the complexity and scale of the problem. As the objecti ve function inv olves simulation, it tends to be non-linear and non-con vex making it challenging for a real-time application. Finally , the study is also moti vated by important applications in real-time traffic management and incident response systems. In view of the aforementioned motiv ations, the follow- ing objecti ves are identified: 1) T o develop an integrated framew ork within a real-time DT A system that determines optimal control strategies and consistent guidance information considering traf fic state predictions; 2) T o propose a real- time solution methodology to efficiently solve for the optimal strategies under the framework proposed in objectiv e 1; 3) T o ev aluate the proposed framework using a closed-loop approach (where the DT A system is interfaced with a traffic microsimulator that emulates the stochasticity in real world, thus providing a platform for realistic ev aluation) on a large real-world network with link tolls as control strategies. The salient contributions of this work are, first, the proposed simulation-optimization framework simultaneously optimizes network control strategies and computes consistent guidance information based on traf fic state predictions. Utilizing traf fic state predictions aids in accurately ev aluating the ef fect of control strate gies. Furthermore, the control strategy at any location is determined based on global traffic state predictions and not just local predictions, thereby explicitly considering the system-le vel effects. The consistency in guidance ensures that the information disseminated by the traffi c management center is reliable, an important issue that has been overlooked in the literature on control strategy optimization. The second contribution is that we apply a highly parallelizable genetic algorithm to solve for the optimal control strate gy (within the proposed framework) that maintains computational tractabil- ity to achiev e real-time performance on a large real-world network. Third, we ev aluate the proposed framework using a rigorous closed-loop approach that ensures that impacts of the control strategy are not overestimated. The experiments demonstrate the effecti veness of the proposed system which can yield trav el time improvements of up to 9%, and av erage computational times of less than 5 minutes. In addition, a sensitivity analysis is performed with respect to network demand le vels. I I . L I T E R A T U R E R E V I E W Although the frame work presented in this paper is appli- cable to other control strategies including ramp-metering, the 2 revie w here focuses on real-time congestion pricing in view of the application presented. The reader is referred to Chung and Recker (2011) for a revie w of existing toll facilities in the US [1] and to de Palma and Lindsey (2011) [2] for a discussion of congestion pricing technologies. There are two broad categories of tolling strategies: fixed pricing strategies and dynamic pricing strategies. In fixed pricing strategies, the tolls are predetermined; they can be a time-in variant or can vary in a predetermined manner during the day (time-of-day tolling). Further , in a fixed pricing strategy , tolls can also v ary based on location and vehicle type. In the dynamic pricing strate gies, the tolls are continually determined based on the current/future traffic conditions and are not predetermined. A dynamic tolling strategy can be either reacti ve or predicti ve. In a reacti ve tolling strategy , the tolls are determined based on the current traffic conditions. In contrast, in predictiv e tolling, the tolls are determined considering predicted traffic states. Y ang (2005) [3] and Tsekeris and V oss (2009) [4] should be referred for a revie w of work on static and fixed congestion pricing. Among the studies that determine time-dependent and fixed pricing, De Palma et al. (2005) was one of the earliest to study the effect of time-inv ariant vs. time-dependent pricing using a simulator [5]. Their experiments sho w that time- dependent tolls can generate twice the welfare gains compared to time-in variant tolls. Xu (2009) presented an optimization framew ork with the travel time objecti ve and solved the problem using the SPSA (Simultaneous Perturbation Stochas- tic Approximation) algorithm [6]. Chen et al. (2014) solve the similar problem with the travel time objectiv e [7]. The problem was solved by statistically modeling the objectiv e function (calculated from the output of DynusT) using Kriging. The same authors later extended the work to objectiv es of throughput and revenue [8]. The tolling scheme was based on the vehicle miles traveled. [9] has also studied distance based tolling along with elastic demand. The studies on the dynamic reactiv e pricing hav e predom- inantly been in the context of managed-lane operations. Y in and Lou (2009) propose two dynamic pricing approaches for managed toll lanes: a feedback-control approach and reactiv e self-learning approach [10]. The pricing decisions are based on real-time traffic conditions and the objective is to improve the free-flow travel service on the toll lanes while maximizing total throughput. Similar approaches —based on feedback control— hav e been used to optimize for various other objectives like speed, trav el time, delays, and revenues [11]–[13]. Morgul (2010) studied dynamic reactiv e pricing for different tolled links in a network by employing the traf fic simulation software Paramics and TransModeler [14]. The algorithm applied was from Zhang et al. (2008) [11]; it is a feedback controller based on speed measurements. It was sho wn that dynamic tolling results in lower queue lengths and higher speeds. Dong et al. (2011) studied the predicti ve tolling strat- egy , where the predicted traffic conditions pro vided by D YN ASMAR T -X were used to generate the tolls [15]. A feed- back control approach was adopted where the toll at a location is determined by adjusting the pre vious toll based on the deviation of predicted concentration on the corresponding link from the desired lev el. Hassan et al. (2013) [16] also studied predictiv e tolling in order to maximize rev enue. The toll is optimized based on a formulation where a Greenshields model is embedded to represent traffic dynamics and a binary logit model is incorporated for route choice. A linear approximation is used for the solution of the optimization model and the optimized toll is e valuated through a simulation-based DT A system (DIRECT) with prediction capabilities. They applied the tolling methodology on a synthetic corridor network with two gantries where the tolls need to be optimized. More recently , Hashemi and Abdelghany (2016) [17] provided a predictiv e control framew ork with an example of timing deci- sions on signalized intersections. They also used DIRECT for state estimation and prediction and for the optimization of the control they used genetic algorithm similar to our approach. They applied the methodology to the US-75 corridor in Dallas. In summary , a considerable number of studies hav e a reactiv e setting, i.e., they do not consider the effects in future time-periods while determining tolls in the current time- period. This myopic tolling policy can result in undesirable and fluctuating tolls and traffic conditions. Additionally , a common approach to determine the dynamic tolls is based on feedback control, where the tolls are adjusted based on either observed or predicted characteristics like speed or queues. Howe ver , as the characteristics of only the tolled links are used to determine the corresponding tolls, the system-level interactions are ignored and hence makes them inef ficient for large scale networks. Recent studies mov ed to predictiv e tolling as some major examples are cited above. Ho wever , few studies optimize for predictive dynamic pricing strategies at a network-lev el, most consider corridor type networks. Furthermore, consistency between the provided guidance and the resulting network conditions is not handled fully in most of the studies. Finally , the ev aluation of the optimized tolls is done through the same simulator that is used to optimize the tolls [18]. This may overestimate the network performance improv ements. This study addresses these gaps in real-time predictiv e control systems, more specifically tolling. I I I . I N T E G R A T E D F R A M E W O R K F O R R E A L T I M E C O N T R O L S T R A T E G Y O P T I M I Z A T I O N A N D G U I DA N C E G E N E R AT I O N This section briefly describes the proposed framework for the integrated optimization of control strategies and generation of consistent travel time guidance. For the ease of exposition, the framework is illustrated using DynaMIT2.0, a simulation based DT A system for traf fic state estimation and prediction dev eloped at the MIT Intelligent Systems Laboratory [19], [20]. Howe ver , it is noted that the frame work is generic and applies to any real-time DT A system. The DynaMIT2.0 system is first very briefly introduced followed by a discussion of the proposed frame work. DynaMIT2.0 is composed of two core modules, state es- timation and state prediction, and operates in a rolling hori- zon mode. During each ex ecution cycle, the state estimation module uses a combination of historical information and real- time data from various sources (surveillance sensors, traf fic information feeds, weather forecasts) to first calibrate the de- mand and supply parameters of the simulator so as to replicate 3 Fig. 1: Framework for Integrated Guidance Generation and Control Strategy Optimization prev ailing traf fic conditions as closely as possible. The updated parameters are then utilized to estimate the state of the entire network for the current time interv al. Based on this estimate of the current network state, the state prediction module predicts future traffic conditions for a prediction horizon and generates consistent guidance information (refer to [19] for more details on the DynaMIT) that is disseminated to the travelers. The integrated framework is summarized in Figure 1. During each execution cycle, following state estimation , the Pr ediction based Information Generation and Strate gy opti- mization process is in voked. W ithin this process, the opti- mization module generates a series of control strategies (for example network tolls, signal timings, etc.) for the prediction horizon period which are to be ev aluated on the basis of a specific objective. This can include the minimization of total system trav el time, maximization of consumer surplus, maximization of operator rev enues and so on. The e valuation of each control strategy in volves running the state prediction module iteratively to ensure that the predicted network state is consistent with the provided guidance. More specifically , the state prediction module (expanded in the right half of Figure 1) begins with the most recently disseminated guidance (for instance, the guidance may be in the form of network link tra vel times) as a trial solution. The coupled demand and supply simulators are then used to predict the network state based on the given control strategy and assumed guidance as inputs (note that the route choices of driv ers change in response to the control strategy and guidance). This yields predicted network travel times which are then combined with the original guidance (using the method of successiv e a verages or MSA) to obtain a revised trav el time guidance solution. This procedure is iteratively performed until con ver gence, i.e., the pro vided tra vel time guidance and predicted network travel times are within a pre- specified tolerance limit  P . Once con vergence is achie ved, the state prediction and guidance strategy are termed ’consistent’ and the corresponding network state is then used by the opti- mization module to ev aluate the objective function and search for the optimal control strate gy . F ollowing the completion of the optimization procedure, the Prediction based Infor - mation Generation and Strate gy optimization process returns an optimal control strate gy that is applied to the network and consistent trav el time guidance that is disseminated to trav elers. The proposed framework is demonstrated in the subsequent sections through an application to the dynamic toll optimiza- tion problem. I V . F O R M U L A T I O N O F D Y N A M I C T O L L O P T I M I Z AT I ON P RO B L E M Fig. 2: Illustration of the rolling horizon approach for toll optimization The transportation network of interest is represented as a directed graph G ( N , A ) where N represents the set of n network nodes and A represents the set of m directed links. Let e A ⊆ A represent a subset of network links that are tolled with e m = | e A | . Consider an arbitrary time interv al [ t 0 − ∆ , t 0 ] where ∆ is the size of the state estimation interv al (typically 5 minutes in real time DT A systems). Assume that the length of the current state prediction horizon is equal to H ∆ (each ∆ interval within the prediction horizon is termed a prediction interval) and extends from [ t 0 , t 0 + H ∆] . In addition, assume that the link tolls are set for intervals of size ∆ (this period is 4 referred to as the tolling interval) and that the tolling intervals are aligned with the state estimation/prediction intervals. Let τ h = ( τ h 1 , τ h 2 . . . τ h e m ) represent the vector of link tolls for the time period [ t 0 + ( h − 1)∆ , t 0 + h ∆] where h = 1 . . . H . The vector of tolls for the current prediction horizon is thus gi ven by τ = ( τ 1 , τ 2 , . . . τ H ) . In real world applications, giv en that the state estimation and solution of the optimization problem will require a finite computational time (assume that this is at most equal to the interval length ∆ ), it will not be possible to implement the optimal toll v ector for the first tolling interv al within the prediction horizon. Consequently , the size of the optimization horizon is assumed to be one tolling interval less than the size of the prediction horizon and the decision variables in our optimization problem are in fact τ 0 = ( τ 2 , . . . τ H ) . τ 1 is set to the optimal v alue for the same prediction interval from the previous execution cycle (denoted by λ ), so that τ = ( λ , τ 0 ) . This is illustrated in the example in Figure 2 for a case where H = 3 . In execution cycle 1 (denoted by C1), the decision vector consists of the toll values ( τ 2 C 1 , τ 3 C 1 ) for the prediction intervals P2 and P3. The toll vector τ 1 C 1 is set as the optimal value from the previous ex ecution cycle (denoted by λ 1 ). Subsequently , in the second execution cycle, the decision vector consists of the toll values ( τ 2 C 2 , τ 3 C 2 ) and λ 2 = τ 2 ∗ C 1 , where τ 2 ∗ C 1 is the optimal value of τ 2 C 1 from execution cycle 1. Furthermore, consider the collection of vehicles ν = 1 , . . . V on the network during the prediction horizon [ t 0 , t 0 + H ∆] . Let the tra vel time of vehicle ν be represented by tt ν and the predictiv e travel time guidance be denoted by tt g = ( tt g i ; ∀ i ∈ A ) , where tt g i represents a vector of the time dependent link trav el times (guidance) for link i . Note that the vehicle travel times tt = ( tt ν ; ν = 1 , . . . V ) are a result of the state prediction module of the DT A system and cannot be written as an explicit function of the tolls and predictiv e guidance. W e characterize the complex relationship through a function S ( . ) that represents the coupled demand and supply simulators as, S ( x p , γ p , tt g , τ ) = tt , (1) where x p , γ p represent the forecasted demand and supply parameters for the prediction horizon. Also note the iterativ e procedure described in Section III ensures consistency between tt g and tt . It is assumed that the total network demand is fixed (in- elastic) and the behavioral response of users to the tolls and predictiv e trav el time guidance is solely through route choice which is modeled within the demand simulator of DynaMIT2.0 using a path size logit model wherein the utility of a vehicle ν on path k is giv en by , ∪ ν k = β c e τ k + β t ¯ tt g k + l og ( P S k ) + C k +  ν k , (2) where e τ k is the toll on route k , ¯ tt g k is the travel time on route k as per the guidance information (which is the sum of travel times on component links), β c and β t represent the cost and trav el time coefficients respecti vely , P S k represents the path size variable for path k , C k represents a composite utility pertaining to additional variables including path length, number of left turns and number of signalized intersections,  ν k represents a random error term. Note that first, for vehicles that do not hav e access to the guidance information, historical trav el times are used and second, similar model structures are used for both the pre-trip and en-route choice models. The reader is referred to [19] for more details. It should be also be pointed out that since the optimization is performed within a rolling horizon framework and giv en that the tolls change e very fiv e minutes, it is likely that the toll values on which the driv er based his pre-trip (or en-route) route choice decision are significantly dif ferent from the tolls he pays in reality . T o mitigate the public opposition that may arise from this, we impose a limit on ho w much the tolls can vary across successiv e tolling interv als on a given gantry . Thus we ha ve, τ h − 1 − δ ≤ τ h ≤ τ h − 1 + δ , h = 2 , . . . H , (3) where δ = ( δ i ; ∀ i ∈ e A ) represents the vector of limits on the change in tolls across successiv e intervals. W ith this background, the dynamic toll optimization prob- lem in our context is formulated as a non-linear program in Equation 4. The objecti ve function considered here is the total trav el time of all vehicles on the network, but can be suitably modified to accommodate other objectives such as consumer surplus, operator re venues or social welfare depending on the context. The decision variables are the vector of toll values for the optimization horizon period. The constraints are the DT A system, upper and lo wer bounds on the toll values (denoted by vectors τ LB and τ U B ), and the constraints on changes in toll v alues across successiv e tolling interv als. DTOP : MIN τ 0 V X ν =1 tt ν ( τ 0 ) s.t. S ( x p , γ p , tt g , τ ) = tt , τ h − 1 − δ ≤ τ h ≤ τ h − 1 + δ , h = 2 , . . . H , τ LB ≤ τ h ≤ τ U B , h = 2 , . . . H. (4) In case of computational performance constraints, the di- mensionality of the DTOP problem abov e may be significantly reduced by assuming that the vector of tolls does not change across prediction intervals within the optimization horizon. In other words, we assume that ( τ 2 = τ 3 . . . = τ H = ¯ τ ) which reduces the number of decision variables from e m ( H − 1) to e m . In this case, the constraints defined by Equation 3 are replaced by , λ − δ ≤ ¯ τ ≤ λ + δ (5) V . S O L U T I O N A L G O R I T H M As noted earlier , since the objectiv e function of the dy- namic toll optimization problem in our context does not have a closed form and is the output of a complex simulator , ev olutionary algorithms and meta-heuristics are preferable to classical gradient based approaches. Hence, a real-coded 5 Genetic Algorithm (GA) [21] is applied to solve the DTOP problem formulated in Section IV. The algorithm starts by randomly generating a set of control strategies or indivi duals ( C S p i , i = 1 . . . N , p : par ent ) collec- tiv ely known as the initial population or parent population of size N . Each single control strategy C S p i comprises the vector of tolls τ 0 = ( τ 2 , . . . τ H ) for the current optimization horizon. These N different control strategies C S p i , i = 1 . . . N (or individuals) are ev aluated (i.e the objectiv e function value is computed) in parallel by running the state prediction module of DynaMIT2.0 ( P D i , i = 1 . . . N ) independently for each control strategy C S p i . The modularity of DynaMIT2.0 pro- vides the functionality to execute a single state estimation run followed by multiple parallel state predictions with different control strategies and makes real time optimization (within the budget of 5 minutes) computationally possible. Different control strategies (or individuals) in the parent population are assigned a rank R i , i = 1 . . . N based on their respective objectiv e function values O bj i , i = 1 . . . N . The objecti ve function value for each individual is calculated from the output of the state prediction module P D i . From this set of control strategies (individuals) in the parent population, a new set of control strategies (new individuals), collectiv ely called the child population of size N is generated using genetic operators, i.e., the SBX crossover and polynomial mutation. The newly generated N control strategies (or individuals) of the child population ( C S c i , i = 1 . . . N , c : child ) are ev aluated in parallel by running the state prediction module (on the same estimated state used to ev aluate parent population) to get their objectiv e function v alues. Strategies in the child population C S c i , i = 1 . . . N and parent population C S p i , i = 1 . . . N are merged together to form a combined set of strategies (mixed population) of size 2 N , C S p + c k ( k = 1 . . . 2 N ) and are ranked based on their objectiv e function values. From these 2 N different strategies (or individuals) C S p + c k , the best N strategies (or individuals) are selected based on their rank to form the parent population for the next iteration. The procedure of generating a ne w set of strategies (or child population) continues uptill the termination criteria are met. The termination criteria can be a predefined number of iterations/generations G max , a threshold for the impro vement in the objectiv e function value or a computational time budget T max . In order to facilitate r eal time performance, it is impera- tiv e to compute parallel tasks in a computationally efficient manner using a multi-core architecture. In the context of our framew ork, ev aluation of dif ferent control strategies C S i s in a particular iteration are independent of each other, therefore ev aluating them in parallel significantly reduces computational time and makes the approach scalable. Different parallel computing architectures and libraries hav e been proposed in the literature, but for our application we adopt a Master- Slav e architecture using the GNU 1 Parallel library [22]. The main benefit of using GNU Parallel is that processor lev el parallelism can be achiev ed. T o ev aluate each control strategy , a new process is launched on a dif ferent CPU. Managing and 1 GNU is a recursiv e acronym for G NU’ s N ot U nix. scheduling of different processes is taken care by the GNU library at the operating system level. In practice, it is observed that even with processor level parallelism the speed-ups are not linear ev en if no inter-process communication is present. Therefore, the frame work is designed so as to allow Batch- W ise ev aluation of different control strate gies. Specifically , during each iteration, all N different control strategies can be launched as dif ferent processes on different CPUs, or they can be launched in batches of size n, n < N . In this batch wise implementation, dif ferent batches can either be launched sequentially on a single cluster of CPUs or they can ev en be launched in parallel on multiple clusters of CPUs. This framework exploits both parallel as well as distributed computing simultaneously as shown in Figure 3. Fig. 3: Genetic Algorithm with parallel ev aluation of popula- tion using parallel & distributed computing techniques. V I . E X P E R I M E N T S This section discusses results from a set of experiments conducted to in vestigate the performance of the proposed strategy optimization approach using DynaMIT2.0 on the Singapore expressw ay netw ork. The numerical e xperiments are conducted using a closed-loop framework, interfacing DynaMIT2.0 and MITSIMLab (MITSIM), a microscopic sim- ulator [23]. MITSIM is run concurrently with DynaMIT and mimics the real network, providing sensor counts for the current interval to DynaMIT which in turn provides predictiv e guidance and tolls to MITSIM (see Figure 4). The ef fect of the 6 Algorithm 1 Control Strategy Optimization Algorithm 1) Store the output of the State Estimation module for interval [ t 0 − ∆ , t 0 ] 2) Replicate the cached estimated state to N different DynaMIT clones 3) Initialize N dif ferent control strategies to form initial population 4) Evaluate N dif ferent control strategies by running State Pr ediction module iterativ ely for interval [ t 0 , t 0 + H ∆] using N different DynaMIT clones in parallel 5) Assign rank to N individuals on the basis of their Objectiv e Function v alue. 1: for all g ← 2 to G max ∩ clock-time ≤ T max do a) Generate N new control strategies using tourna- ment selection with SBX crossover to form child population b) Mutate N newly generated control strategies using polynomial muatation c) Evaluate N child strategies by running State Pr ediction module for interval [ t 0 , t 0 + H ∆] using N different DynaMIT clones in parallel d) Merge the child population and parent popula- tion strategies to form mixed population of size 2 N e) Assign rank to 2 N indi viduals on the basis of their Objcti ve Function v alue f) Select the best N individuals (strategies) to form the parent population 2: end for 6) Select the best Control Strategy C S best from the final set of strategies Fig. 4: Closed-Loop Framework guidance and tolls can then be examined by extracting relev ant performance measures from MITSIM av oiding overestimation of the benefits. The experiments are conducted on the network of major arterials and expressways in Singapore (Figure 5) which consists of 948 nodes, 1150 links, 3891 segments, and 4123 origin-destination (OD) pairs, and 16 tolled links. The labels represent the links where there is a toll gantry . Fig. 5: Network of Expressways and Major Arterials in Sin- gapore The section is organized into fiv e parts. The first part dis- cusses the setup of the closed-loop framework and calibration, the second describes the experimental design and inputs. The third section analyzes the results in terms of travel time savings and the effect of network demand, fourth part discusses the optimal tolls through few gantries, and finally the fifth part discusses computational performance. A. Closed-Loop Calibration In order to set up the closed-loop environment, a two stage calibration procedure is adopted using the w-SPSA algorithm [24] (for other approaches see [25]). In the first stage, dynamic OD demand (for a period between 06:30 AM and 12:00 PM), driv er behavior and route choice parameters of MITSIM are calibrated by minimizing a two component objectiv e function. The first component is the sum of squared deviations between simulated counts and actual counts (on a set of 325 sensors for 5 minute time intervals a veraged across 30 weekdays in February and March 2015) obtained from the Singapore Land T ransport Authority (L T A). The second component is the difference between the parameter v alues and apriori estimates. The inputs for the calibration process is a set of a priori parameter v alues and a seed OD matrix obtained from a prior calibration procedure [24]. The normalized root mean square error in the sensor counts before and after the calibration process were 73% and 34% respectiv ely . In the second stage, the historical OD matrix, supply and route choice parameters of DynaMIT2.0 are calibrated against the outputs (sensor counts on 650 network segments) generated by MITSIM. The normalized root mean square error in the sensor counts before and after the calibration process were 56% and 19% respecti vely . Further, the RMSN in time- dependent link travel times after calibration was found to be 24%. The results of the second stage of the calibration are summarized in Figures 6a and 6b which show scatter plots of the simulated (DynaMIT) versus actual (MITSIM) sensor counts and link travel times respectiv ely . B. Experimental Setup The numerical experiments are conducted using a simula- tion period from 6:30 AM to 12:00 PM which includes the 7 (a) Simulated vs Actual Count (b) Simulated vs Actual Trav el T imes Fig. 6: Closed Loop Calibration morning peak in Singapore. The state estimation interval (and OD demand interval) is fiv e minutes ( ∆ = 300 seconds) and the prediction horizon is 15 minutes ( H = 3 ). The simulation period is composed of three parts: a W arm-up period from 6:30-7:30 AM where no tolls are imposed, a tolling period from 7:30 - 11:00 AM, and a post-tolling period from 11:00 AM to 12:00 PM where again no tolls are imposed. The impact of the predictiv e toll optimization is examined against two benchmarks using the closed-loop frame work described earlier . It is assumed that the base demand (MITSIM OD demand obtained from the closed-loop calibration) repre- sents the historical demand or an ”average” day . This demand is then perturbed to reflect day to day variability by sampling from a normal distribution with expected v alue as the base demand and a coefficient of variation of 0.2. The first benchmark is the no toll scenario where the closed-loop is simulated using the perturbed demand with zero tolls. The second scenario consists of static optimized tolls . In this scenario, we first compute the optimum static tolls which inv olves minimizing the total travel times for the entire simulation period (obtained from the state estimation) by implementing a single vector of tolls for the complete tolling period. The closed-loop is no w simulated using the perturbed demand with the static optimum tolls. Finally , in the third scenario the closed-loop is simulated using the perturbed demand and the predictive optimized tolls based on the pro- posed frame work in Section III. In all three scenarios, MITSIM receiv es predictiv e travel time guidance from DynaMIT2.0 and in turn provides sensor counts to DynaMIT2.0 e very estimation interval (or ex ecution cycle). Further , to in vestigate the effect of the overall demand lev el, all the three aforementioned scenarios are simulated for four different demand lev els: low (base demand reduced by 10%), base (closed-loop calibration as noted earlier) , high (base demand increased by 10%) and very high (base demand increased by 20%). Note that the demands referred to here are the actual MITSIM (real world) demands. For the scenarios with predictive optimization, the DynaMIT2.0 historical demand (obtained from the second stage in the closedloop calibration) remains unchanged for all demand scenarios. For the scenarios with the static optimum tolls, note that the regulator must perform the determination of the optimum tolls ’offline’ using an estimate of historical demand. Giv en that dif ferent levels of actual demand (unkno wn to the regulator) are tested, we assume that a single computation of the static optimum tolls is performed by considering a worst case scenario where the calibrated DynaMIT2.0 historical demand is increased by 20%. In addition to the comparison with predictive optimization, this allows us to also test the robustness of the static optimum tolls to both systematic and random variation in the actual OD demands (from historical estimates). The performance measures are: 1) av erage trav el times (across v ehicles) for each departure time interv al obtained from MITSIM, 2) computational time for each execution cycle of DynaMIT2.0. Note that for each scenario and demand lev el, the performance measures reported are a verages across 10 different runs to account for stochasticity in the ov erall system. A High Performance Computing Cluster (HPCC) with 120 CPUs and 256 GB of memory is used to run the experiments. For the parameters of GA, we use a population size of 60, probability of cross-ov er and mutation as 0.7 and 0.1 respec- tiv ely with a computation budget of 300 seconds. The number of iterations may vary from interval to interv al depending on the demand, i.e., peak or off-peak periods. C. Analysis of T ravel T ime In order to compute and compare average time-dependent trav el times across scenarios, for the entire population, all the driv ers departing in a given time interval (e.g., 07:00-07:05) are identified and their av erage trip travel time is calculated. This process is repeated for each consecutiv e 5 min interval in the entire simulation period, i.e., starting from 6:30-6:35, 6:35-6:40, ...., up to 12:25 -12:30. The results indicate that the use of predictive optimized tolls yields significant trav el time savings over both the no toll and static optimum scenarios. The percentage impro vement in trav el times of the predicti ve optimized toll scenarios over the two benchmark scenarios for the tolling period and peak period (for all demand lev els) is summarized in T able I. The av erage travel times (over the 8 tolling period) in the case of the predictiv e optimized tolls are lower than the static optimum and no toll cases by 9.12% and 6.74% in the base demand case. Interestingly , the static optimum is worse than the no toll case for the lo w , base and high demand scenarios (see also Figures 7 to 10). This indi- cates that the static optimum based on historical demands is not robust when the actual demands v ary significantly from the historical estimates. Note that the historical demand was scaled up by 20% when computing the static optimum and hence, in the very high demand case where the historical estimates are closest to the actual demands, the static outperforms the no toll scenario. The percentage decrease in tra vel time is 5.39% and 3.71% in the low demand case, 8.88% and 8.24% in the high demand case and 4.00% and 8.38% in the very high demand case. In addition, the percentage improvement for the peak period (between 8:00 am and 9:30 am) is 7.94% with respect to the no toll scenario and 8.36% with respect to the static optimum scenario for the base demand case. It should be noted that in event of non-recurrent scenarios (like a special ev ent or an incident) one would expect a significantly higher impact of the toll optimization and guidance provision. Furthermore, for all demand cases, a standard two sided t- test indicates that the mean travel time (for all departure time intervals within the peak period) of the predicti ve optimized tolling scenario has a statistically significant difference from that of the no toll/static optimum scenarios at a confidence lev el of α = 95 %. Figures 7 to 10 plot the mean travel times (shaded region represents the standard error in estimate of the mean) versus departure time interval for the three scenarios and each de- mand le vel. W ith regard to the ef fect of the overall demand lev el on the improvement in travel time savings with respect to the static/no-toll toll case, the results indicate the lowest improv ements (during the peak period) are attained when the congestion le vels are either very low or very high. This occurs because in the low demand scenario the relati vely uncongested state of the network reduces the impact of toll optimization. On the other hand, the sev erely congested network state in the very high demand scenario also reduces the possibility of alleviating congestion through the re-routing of vehicles leading once again to smaller benefits of the toll optimization. The probability density and cumulative density functions of vehicle trav el time are plotted in Figure 11 for the no toll and predictiv e optimized toll scenarios in the base and high demand cases. The plots highlight the reduction in frequency of trips with higher tra vel time due to the predictiv e optimization of tolls. D. Analysis of Optimized T olls Here we provide fe w examples in order to analyze the optimized tolls under predictiv e optimization with respect to static optimization. First, we gi ve an example of two gantries on links 45 and 83. W e present the optimized tolls under static and predictiv e strategies in Figure 12 and 13. The most preferred path for one of the ODs with a very high demand during the morning peak uses these gantries (first 83 and then 45). The predictive tolls T ABLE I: T ravel Time Improvement Demand Level % Trav el T ime improvement T otal Drivers Simulated T olling Period Peak Period No T oll Static No T oll Static Low 3.71 5.39 7.61 6.25 275,000 Base 6.74 9.12 8.36 7.94 300,000 High 8.24 8.88 9.65 10.74 325,000 V ery High 8.38 4.00 8.20 7.01 350,000 are optimized at higher values compared to the static case and this indicates that real-time predictiv e tolls are adjusted better with respect to demand. Fig. 12: Gantry on link 45 Fig. 13: Gantry on link 83 Fig. 14: Gantry on link 225 Fig. 15: Gantry on link 226 Second, gantries on links 225 and 226 are optimized at lower values during the peak compared to static strategy as shown in Figure 14 and 15. It is observed that these gantries are used towards destinations that hav e very low demand in the morning peak. Predictive toll optimization is able to lo wer the tolls during the peak in order to account for lower demand values tow ards better trav el times. 9 Fig. 7: Tra vel Time for Low Demand Fig. 8: Tra vel Time for Base Demand Fig. 9: Tra vel Time for High Demand Fig. 10: T ravel T ime for V ery High Demand 10 (a) Base Demand pdf (b) Base Demand cdf (c) High Demand pdf (d) High Demand cdf Fig. 11: Peak Period Tra vel T ime Distributions E. Computational P erformance The results also indicate that the proposed solution algo- rithm achieves real-time performance, i.e the a verage compu- tational time per execution cycle (across all demand le vels) is within the five minute time budget (less than a single state estimation interval) discussed in Section IV. The plot of average computational time versus time interval is shown in Figure 16. The tractable computational times are the result of three contributing factors. The first is the imposition of the constraint on the extent to which tolls on a given gantry can vary across successiv e tolling intervals which significantly reduces the search space for the GA. This ensures that a population size of 60 suffices to attain a significant reduction in trav el times within a low computational time budget. Secondly , the rolling horizon approach implies that the system is re-optimized every fiv e minutes and consequently a poor solution in one interval can be quickly rectified or improved in subsequent intervals. This along with the feedback from the real network to the DT A system (through the online calibration) makes the control strategy optimization frame work more robust. Finally and most importantly , the synchronous parallel ev aluation of strategies in each iteration of the optimization procedure allows for e val- uation of a sufficiently large number of candidate solutions. V I I . C O N C L U S I O N This paper proposes an integrated framework that com- bines the optimization of network control strategies with the generation of consistent guidance information for real-time DT A systems. The ef ficacy of the proposed frame work is demonstrated through a fixed demand dynamic toll optimiza- tion problem. Furthermore, a highly parallelizable genetic algorithm based solution approach is adopted. Numerical experiments conducted on a large scale real world network (expressways and major arterials in Singapore) indicate that use of the proposed framew ork can yield significant network- wide travel time savings of up to 8.36% and 7.94% over the no toll and static optimum scenarios respectiv ely . A sensitivity analysis of demand le vels further indicate that the highest improv ements are attained at moderate and high demand lev els. Finally , the proposed solution algorithm achieves real- time performance with a computational time of less than 5 minutes for each ex ecution cycle within the rolling horizon scheme. The proposed framework and solution approach hav e important applications for real-time traf fic management and advanced trav eler information systems. Some directions for future research include the application of the strategy optimization framew ork under non-recurrent scenarios including crisis-management [26], consideration of other objectiv es such as consumer surplus, operator rev enue and multiple objectiv es; incorporation of traf fic state prediction errors [27] and the modeling of elastic demand through trip 11 Fig. 16: Computational Performance cancellation and departure time shifts in response to tolls. The application to other network control strategies and examina- tion of the suitability of alternative solution algorithms also promise to be interesting areas for future research. A C K N O W L E D G M E N T W e would like to thank the Singapore Land T ransport Authority (L T A) ITS division, for providing the data for this project. This work was supported by the National Research Foundation of Singapore (SMAR T program). R E F E R E N C E S [1] C.-L. Chung and W . Recker , “State-of-the-art assessment of toll rates for high-occupancy and toll lanes, ” in Tr ansportation Resear ch Board 90th Annual Meeting , no. 11-1331, 2011. [2] A. de Palma and R. Lindsey , “T raffic congestion pricing methodologies and technologies, ” Tr ansportation Researc h P art C: Emerging T echnolo- gies , vol. 19, no. 6, pp. 1377–1399, 2011. [3] H. Y ang and H.-J. Huang, Mathematical and economic theory of road pricing , 2005. [4] T . Tsekeris and S. V oß, “Design and e valuation of road pricing: state-of-the-art and methodological advances, ” NETNOMICS: Economic Resear ch and Electronic Networking , vol. 10, no. 1, pp. 5–52, 2009. [5] A. De Palma, M. Kilani, and R. Lindsey , “Congestion pricing on a road network: a study using the dynamic equilibrium simulator metropolis, ” T ransportation Researc h P art A: P olicy and Practice , vol. 39, no. 7, pp. 588–611, 2005. [6] S. Xu, “Development and test of dynamic congestion pricing model, ” Master’ s thesis, Massachusetts Institute of T echnology , 2009. [7] X. M. Chen, L. Zhang, X. He, C. Xiong, and Z. Li, “Surrogate-based optimization of expensi ve-to-ev aluate objective for optimal highway toll charges in transportation network, ” Computer-Aided Civil and Infras- tructur e Engineering , vol. 29, no. 5, pp. 359–381, 2014. [8] X. M. Chen, C. Xiong, X. He, Z. Zhu, and L. Zhang, “Time-of-day vehicle mileage fees for congestion mitigation and rev enue generation: A simulation-based optimization method and its real-world application, ” T ransportation Research P art C: Emerging T echnologies , vol. 63, pp. 71–95, 2016. [9] V .-A. V u, H. Hashemi, R. Seshadri, S. Gupta, G. T an, A. A. Prakash, and M. Ben-Akiva, “Predictiv e distance-based toll optimization under elastic demand for real-time traffic management, ” T ransportation Resear ch Boar d 97th Annual Meeting , 2018. [10] Y . Y in and Y . Lou, “Dynamic tolling strategies for managed lanes, ” Journal of T ransportation Engineering , vol. 135, no. 2, pp. 45–52, 2009. [11] G. Zhang, Y . W ang, H. W ei, and P . Y i, “ A feedback-based dynamic tolling algorithm for high-occupancy toll lane operations, ” T ransporta- tion Researc h Recor d: Journal of the T ransportation Research Board , no. 2065, pp. 54–63, 2008. [12] Y . Lou, Y . Yin, and J. A. Lav al, “Optimal dynamic pricing strategies for high-occupancy/toll lanes, ” T ransportation Researc h P art C: Emerging T echnologies , vol. 19, no. 1, pp. 64–74, 2011. [13] K. Jang, K. Chung, and H. Y eo, “ A dynamic pricing strategy for high occupancy toll lanes, ” T ransportation Resear ch P art A: P olicy and Practice , vol. 67, pp. 69–80, 2014. [14] E. F . Morgul, “Simulation based evaluation of dynamic congestion pric- ing algorithms and strategies, ” Ph.D. dissertation, Rutgers Univ ersity- Graduate School-New Brunswick, 2010. [15] J. Dong, H. S. Mahmassani, S. Erdo ˘ gan, and C.-C. Lu, “State-dependent pricing for real-time freeway management: Anticipatory versus reactiv e strategies, ” T ransportation Resear ch P art C: Emerging T echnologies , vol. 19, no. 4, pp. 644–657, 2011. [16] A. Hassan, K. Abdelghany , and J. Semple, “Dynamic road pricing for rev enue maximization: Modeling framework and solution methodology , ” T ransportation Researc h Record , vol. 2345, pp. 100–108, 2013. [17] H. Hashemi and K. F . Abdelghany , “Real-time traffic network state estimation and prediction with decision support capabilities: Application to integrated corridor management, ” T ransportation Resear ch P art C , vol. 73, pp. 128–146, 2016. [18] S. Gupta, R. Seshadri, B. Atasoy , F . C. Pereira, S. W ang, V .-A. V u, G. T an, W . Dong, Y . Lu, C. Antoniou, and M. Ben-Akiva, “Real time optimization of network control strategies in dynamit2.0, ” 2016. [19] M. Ben-Akiv a, H. N. Koutsopoulos, C. Antoniou, and R. Balakrishna, “T raffic simulation with dynamit, ” in Fundamentals of T raf fic Simula- tion , J. Barcel ´ o, Ed. Ne w Y ork, NY , USA: Springer, 2010, pp. 363–398. [20] Y . Lu, R. Seshadri, F . Pereira, A. OSulliv an, C. Antoniou, and M. Ben- Akiv a, “Dynamit2.0: Architecture design and preliminary results on real-time data fusion for traf fic prediction and crisis management, ” in 2015 IEEE 18th International Confer ence on Intelligent T ransportation Systems , Sept 2015, pp. 2250–2255. [21] K. Deb, A. Pratap, S. Agarwal, and T . Meyari van, “ A fast and elitist multiobjectiv e genetic algorithm: Nsga-ii, ” Evolutionary Computation, IEEE Tr ansactions on , vol. 6, no. 2, pp. 182–197, Apr 2002. [22] O. T ange, “Gnu parallel - the command-line power tool, ” ;login: The USENIX Magazine , vol. 36, no. 1, pp. 42–47, Feb 2011. [Online]. A vailable: http://www .gnu.org/s/parallel [23] M. Ben-Akiv a, H. Koutsopoulos, T . T oledo, Q. Y ang, C. Choudhury , C. Antoniou, and R. Balakrishna, “Traf fic simulation with mitsimlab, ” in Fundamentals of T raf fic Simulation , ser . International Series in 12 Operations Research & Management Science. Springer New Y ork, 2010, vol. 145, pp. 233–268. [24] L. Lu, X. Y an, C. Antoniou, and M. Ben-Akiv a, “W -spsa: An enhanced spsa algorithm for the calibration of dynamic traffic assignment models, ” T ransport Researc h P art C , Accepted - Forthcoming. [25] B. Ciuffo and C. L. Azevedo, “ A sensitivity-analysis-based approach for the calibration of traffic simulation models, ” IEEE T ransactions on Intelligent T ransportation Systems , vol. 15, no. 3, pp. 1298–1309, 2014. [26] S. N. Hetu, S. Gupta, V .-A. V u, and G. T an, “ A simulation framework for crisis management: Design and use, ” Simulation Modelling Practice and Theory , vol. 85, pp. 15 – 32, 2018. [Online]. A vailable: http://www .sciencedirect.com/science/article/pii/S1569190X18300273 [27] F . C. Pereira, C. Antoniou, J. A. Fargas, and M. Ben-Akiv a, “ A metamodel for estimating error bounds in real-time traffic prediction systems, ” IEEE Tr ansactions on Intelligent T ransportation Systems , vol. 15, no. 3, pp. 1310–1322, 2014.