Network learning via multi-agent inverse transportation problems

1 Network learnin g via multi - agent inverse transpor tation problems Susan Jia Xu 1 , Mehdi Nourinejad 2 , Xuebo Lai 3 , Joseph Y. J. Chow 1* 1 Department of Civil & Urban Engineering, New York University, New York, NY, USA 2 Department of Civil & Mining Engineering, University of Toronto, Toronto, ON, Canada 3 Department of Computer Science, Courant Institute, New York University, New York, NY, USA * Corresponding author email: joseph.chow@nyu.edu Abstract Despite the ubiquity of tra nsportation data, methods to infer the stat e parameters of a network either ignor e sensitivit y of route decision s, require ro ute enumera tion for param eterizing descriptive models of route sel ection , or re quire complex bilevel models of route assignment behavior. These limitati ons prevent modelers from fully exploi t ing ubiquitous data in monitoring transportatio n networks. Inverse optim ization methods that capture network route choice behavior can address this gap, but they are designed to take observations of the sam e model to learn the parameters of that m odel, which is statistical ly inefficient (e.g. requires estimating population route and link flows) . N ew inverse optimization models and supporting algorithms are proposed to lea rn the parameters of hete rogeneous travelers’ rou te behavior to infer shared network state parameters (e.g. link capacity dual prices). The i nf erred values are consistent with observations of each agent’s optimization behavior . W e prove that the method can obtain unique dual prices for a network shared by these agents in polynomial time. Four experiments are conducted. The first one , conducted on a 4 - node network , verifies the methodology to obtain heterogeneous link cost parameters even when multinomial or mixed logit model s would not be meaningfully estim ated . The second is a parameter recovery test on the Nguyen - Dupuis network that shows that uni que latent link capacity dual prices can be inferred using the proposed method. The third test on the same network demonstrates how a monitoring system in an onl ine learning environment can be designed using this method. The last test d emonstrates this learning on real dat a obtained f rom a freeway network in Queens, New York, using only real -ti me Google Maps qu eries. Keywords inverse optimization, learning, network analysis, route choice, multi- agent system Accepted f or publication i n Transportat ion Science 2 1 Introduction Travel data has become increasingly abundant and ubiquitous in recent years due to ad vances in information and communications technologies (ICTs) and Big Data. The shift to data-driven methods in transportation is apparent . N otable efforts include the Mobile Millennium project to track traffic using GPS phone data ( Herrera et al., 2010), u se of transit s mart card da ta in Santiago to estimate travel dem and patterns (Munizaga and Palma, 2012), urban link performance inferen ce from New York City taxi data (Zhan et al., 2013), an d the Digital Matatu Project in Nairobi, Kenya, to characterize th eir f lexible bus sy stem with ce ll phone data ( William s et al., 2015) , among others . Such methods make use of observations from data to inf er or learn specific character istics of system components that are not directly observable. For example, the study from Zhan et al. (2013) infers link travel tim es using taxi GPS data. The m ethodology is ge nerally classified as an inverse model (see Tarantola, 2 005), which assumes th ere exists a m odel  that transforms a s et of parameters  to a set of outputs  as  =  (  ) . The inverse model deals with finding a set of parameter estimates   based on observed outputs  as   =   (  ) . In a network setting, the state of a network may be defined by several types of parameters that requir e estimation: link costs, link/path flo ws, origin -destination (OD ) flows, link capacities or congestion effects. Inference in a network context using inverse models has a long history and literature. The primary factor distinguishing inference methods for network parameters is whether, and how, the method handles sensitivity of route decisions by travelers or (in the case of non- trans portation networks) packets to the state of the netwo rk. A second consideration is that system parameters (e.g., link costs and capacity effects) need to be inferred as they depend on exter nal factors like weather, presence of inci dents, etc. , par ticularly in transpo rtation networks. The simplest inferenc e methods relate an obse rvable set of att ributes to the desired p arameters directly thro ugh network structu re (e.g. link flow is the sum of the proportions of all OD flows that use that link ) abse nt of any route decision sensitivity. One example is Van Zuylen and Willumsen (1980), who propose an OD estimation model from link traffic counts using entropy maximization to determine the proportion of OD f lows that are most likely to traverse an observed link. The variables are connected b y an incidence matrix that refle cts the network structure. Ins tead of using entropy maximization, Bell (1991) uses a constrained gener alized least squares approach with additional information from prior surveys/studies to regularize the inverse problem. Path flows are not explicit in such models. Inverse models that deal with route sensitivity do so by explicitly mapping observed variables like link counts to latent path flows, and then mapping those path flows to desired parameters like OD flows. Examples include Vardi (1996) and Tebaldi and West (1998). Vardi (1996) coins this topic as “network tomography”, and proposes methods to estima te parametric dist ributions of f low from observed link count data for a single known path or for a Markovian distribution of paths. Tebaldi and West (1998) use Bayesian inf erence to estimate the parametric d istributions considered by Vardi (1996). Since these methods require path enumeration, it may b e difficult to apply them to large networks. Recent efforts (Airoldi and Blocker , 2013; Hazelton , 2015) use mor e efficient route flow sampling strategies to make these infe rence models more scalable. These methods are tested on networks with up to 21 nod es. In the case of transportation networks, two qualities offer unique challenges and opportunities to network inference : (1) state chan ges can involve system parameters like l ink capacities due to weather or incidents; and (2) route decisions are fundamentally governed by behavioral mechanisms like route choice behavior. Changes that involve link capacities or other similar 3 system param eters may be much harder to anticipate using param etric models (e.g. n ormal distributions) for path assignment based only on prior data . T o compensate for this, transportation researchers have inserted behavio ral mechanisms (Cascetta et al., 1996; Vovsha and B ekhor, 1998; Srinivasan and Mahmassani, 2000; Dia, 2002; Frejinger and Bierlaire, 2007; Ben - Elia and Shiftan, 2010; Gao, 2012; Fosgerau et al., 2013) into the inverse models so that route sensit ivity can rely on the mechanism where data is sparse . Route enumeration remains an issue in some cases , even as researchers seek way s to address that with ch oice set select ion. F or example, Fosgerau et al. (2013), Baillong and Cominetti (2006), and Akamatsu (1996) make use of Markovian models to overcome route generation. A literatu re survey is avail able from Prato (2009). To overcome the route enumeration problem, more integration of behavior with network structure has been sought ( see W atling et al., 2015) . For example, when considering congestion or capacity effects of networks, the pre sence of a stochast ic user equilibrium behavior (Daganzo an d Sheffi, 1977) may be as sumed. In the case of OD estim ation, Yang et al. (1992) assume Wardrop’s user e quilibrium behavioral principle to assign observed link flows to route flows without having to enumerate paths . Other efforts include Ashok and Ben - Akiva (2002), who allow for p erturbations from the behavioral mechanism with stochastic assign ment. A new problem emerges, however, with the complexity of having a nonconvex bilevel optimization problem for the inverse model. To summarize, metho ds to infer the state param eters of a network either ignore sensitivity of route decisions, require route enumeration for parameterizing models of route selection, or requi re complex bilevel models of route assignment behavior. Some of these inference methods, particularly for demand patterns in a network, may not even be necessary in an age of data ubiquity. For instance, methods like Tang et al. (2015) and Alexander et al. (2015) use GPS or mobile phone data to reduce OD estimation to a basic sampling problem without having to infer them from other observed variables. Nev ertheless, there remains a challenge of being able to explain network state changes, as illustrated in Fig. 1. The classic Nguyen- Dupuis network (Nguyen and Dupuis, 1984) is used to show two states of latent link capacities i mposed on the network. In the figure, the change from one state on the left to the next on the right is due to a change in the link 7 capaci ty . Based on observation (and hypothetically from methods in Tang et al. (2015) and Alexander et al. (2015)), we can compute or estima te the total system travel tim es (68,400 on left, 70,000 on right) and observe that there is a difference in flow on links 3, 4, 5, 7, 8, 9, 10, 11, 12, 14, and 15, but we cannot explain why the state change occurs. Fig. 1. Illustr ation o f a stat e change in th e Nguyen - Dupuis network tha t leads to a change in obse rved flows (link IDs in re d parentheses ). 4 One potential solution for network inference is inverse optimization . Ahuja and Orlin (2001) proposed a class of inverse problems posed as a linear optimization model. This class of problems, called inver se optimiza tion (IO), allow s one to lea rn linear prog ramming p arameters from observed decision variables and prior information such that the desired optimality condition i s maintained. IO models captur e the route choice mechanism if it follows an assignm ent - based mathematical program. The models also have great f lexibility in inferring either demand or system parameters. Nonetheless, inverse optimization in the current literatu re also cannot ef fectively exp lain the state change in Fig. 1 for two primary reasons . First, t he closest model from Güler and Ham acher (2010) stud ies the single commodity capacity inverse minimum cost flo w problem and conclude that it is NP -hard. However, the example in Fig. 1 describes a m ulticommodity flow problem in which there is demand from node 1 and node 4 to node 2 and node 3. Secon d , IO assum es that the data obtained is at the same level of the model, e.g. an inverse of a network flow mod el uses observation of the flow data to learn the parameters of the system model. In many transportation cases, however, this is untrue . T he system model is the whole transportation network, and its parameters dictate how the co ngestion and capacity eff ect s influence travel. However, it is the individual agents that are optimizing their own network models (e.g. shortest path) as a behavioral mechanism and learning from their e xperiences. Without addressing this d iscrepancy, current IO methods need t o estimate variables at the population level, e.g. total link or path flows, which i s statistically ineff icient . If we are able to synchroni ze the route choice behav ioral mechanism at the system level with the behavior of individual agents, we may be able to avoid this costly step. We propose to address this discrepancy w ith a new data - driven methodology that uses inverse optimization with networ k models tha t rely on only lea rning from sam pled heterogeneous agents. The method takes observ ed routes of travelers t o estimate heterogeneou s route preferences and infer network parameters that influence these observed routes, such as the dual prices of the link capacities. Because the method learns the preferences o f multiple agents and relates those preferences to system parameters, it can be used in an online learning environment in which system parameters are updated over time based on only real - ti me sampling of individual agents without having to estimate population link or path flows. The remainder of the paper is organized as follows. Section 2 expands on the review of inverse optimization as a learning methodology, and recent develo pments to handle heterogeneity in observed data. Section 3 presents the proposed methodology i n two stag es: fir st as a bas ic model to infer heterogeneous route preferences of multiple agents using a shared network, and second in inferring the parameters of that network to explain their route choices. Section 4 presents four computational experiments that inc lude a demonstration of online learning in a synthetic network and in a real d ata illustration of a free way network i n Queens, NY. Section 5 concludes. 2 R eview of inverse optimization I nverse optimization can be used to learn network paramete rs from a prio r value. The first inverse optimization model was proposed by Burton and Toint (1992) for the inverse shortest path problem , and it was further generalized by Ahuja and Orlin (2001) . It is defined as follows : for a given prior   of a linear program’ s (LP’s) parameters and observed decision variables   , 5 determine an updated  such that   is optimal while m inimizing its   norm from the prior , as shown in Eq. (1).  is the constraint m atrix and  is the vector of side constraint values. min  |     | :   =  {    :    ,   0 } (1) The   norm minimization is used to regularize what would otherwise be an ill - posed problem with infinite solutions. This ensures that a unique solution can be ob tained given a prio r   . Alternative ly, wei ghts can be added for further calibration. Ahuja and Orlin (2001) showed that Eq. (1) (as well as the   n orm variant) can be reformulated as an LP. This is done by introducing two non - negative decision v ariable vectors  and  such that th eir differenc e is equal to     :     =    . T he problem can then be reformulated using strong duality and dua l feasibility conditions, as shown in Eq. (2) to Eq. (5). min  ,  ,   +  (   norm minimization) (2) subject to         +  (dual feasib ility) (3)    = (     +  )    (strong duality) (4)  ,  ,   0 (non- negativity) (5) where  is a dual variable of the original LP. There have been a num ber of advances and app lications in inverse op timization, with an ear ly survey on combinatorial IO by Heuberger (2004). Table 1 pro vides a summary of these advances. For example, Wang (2009) proposed a cutting plane m ethod to solve the inverse mixed integer linear progra mming (InvMILP) p roblem. G üler and Hamacher (2010) proposed an IO model for estimating link capacities. Zhang and Zhang (2010) proposed an inverse quadratic program. Chow and Recker (2012) proposed an inverse vehicle routing problem (VRP) that includes side constraint (goal arrival times) esti mation. Bertsimas et al. (2015) proposed an inver se va riational in equality to estimate the parameters tha t would lead to observed p atterns under equilib rium. There are also many applications for IO. Day et al. (2002) used IO as a tool for model calibration for railroad networks. Burkard et al. (2004) applied IO to facility location problems. Agarwal and Ergun (2008) used IO as a m echanism design approach to select a solution from the core for a multicomm odity flow game betw een different service networks seek ing an alliance. Brucker and Shakhlevich (2009) studied applications in inverse scheduling. Bertsimas et al. (2012) applied IO to estimate the Black - Litt erma n m odel in portfo lio optimizatio n in the f inancial industry. Birge et al. (2014) applied IO to reveal the electricity market structure based on observed choices made by competitors. Chow et al. (2014) proposed an inverse traffic assignment problem for cali brating impedances at freight facilities, and You et al. (2016) used truck GPS data to infer parameters to forecast u rban truck delivery patte rns. Hong et al. (2017) proposed using inverse optimization to infer the heterogeneous route- level parameters of a mixed logit model of route choice, and tested that method with transit smart card data taken from 50,000 trips in the Seoul metro system. They show empirically that the fixed point method proposed in Chow and Recker (2012) is able to discern heterogeneous parameter dist ributions for a utility function composed of transit time, transfer, and crowding . 6 Table 1. Ove rview of inverse optimization advances and ap plicati ons Methodol ogical a dvances New a pplication s Burton a nd Toi nt (1992) Inverse shortest path Day et al. (2002) Network calibration Ahuja and Orlin (2001) Inverse linear programming Burkard e t al. ( 2004) Inverse median problem Wang (2009) Inverse MILP Agarwal and Ergun (2008) Mechanism design Güler and Hamacher (2010) Link capacities in inverse minim um cost fl ow proble m Brucker and Shakhle vich (200 9) Inverse sche duling Zhang an d Zhang (2010) Nonlinear inverse o ptimization Bertsimas et al. (2012) Financial portfo lio management Chow an d Recker (2012) Multi - agent in verse opt imiz ation, inverse VRP with sid e constraint estimation Birge et al. (2014) Electricity market structur e Aswani et al. (2015) Noisy data Chow et al. (2014) Inverse traffic assignment Bertsimas et al. (2015) Inverse variatio nal inequality You et al. (2016) Urban tr uck forecasting Esfahani et al. (201 5) Incom plete inf ormati on Hong et al. (2017) Mixed l ogit est imation Despite these developments, IO methods involve a singl e model and observations of decision variable outputs of that model. In man y instances, observatio ns are not made at th e aggregate level, but come from individual behavioral agent decisions. This dichotomy between a system model and learning through agent observations is an important distinction. First, IO methods require system observations, but obtaining that information from individual agents result in heterogeneous inputs. The result is that addition al estimation of system aggregation from agent observations is needed, which leads to increased inef ficiencies. Second, the resulting information from agent observations may not be consistent with the system model when inferring system param eters. This lim itation is evident in th e IO literatur e, where most ef forts have fo cused on system s with learning from the same system level observations (e.g. ob served company choices and thei r electricity pricing decisions, or an individual’s perceived costs of a network from their own observed path choices). Recent studies have tried to address these point by using noisy observations. Aswani et al. (2015) set up a bilevel problem to estim ate from noisy data, and Esfahani et al. (2015) modeled the noisy information problem as a robust optimization model. B oth approaches explain the heterogeneity with stochastic distributions to a llow subopti mal observations . However, thi s leads to discrepancies with mechanistic assump tions in the prevailing system model (i.e. it may lead to an observation that is not opt imal with resp ect to t heir own pref erences). The lim itation o f current IO methods to using system observations is also in computational efficiency. For example, Güler and Hamacher (2010) proposed an IO model to infer l ink capacities for a minimum cost flow problem from observed flows. They conclude that the model is NP-hard. Chow and Recker (2012) proposed a multi - agent frame work for IO where a sam ple of individuals ’ trip s cheduling data is obtained and used to infer parameters of individual acti vity scheduling (varia nt of vehi cle routi ng from Recker, 1995) . In this case, parameters of multiple individuals are estim ated such that the mean of their parameters i s a fixed point. This leads to a learning process for heterog eneous parameters of a system model, where individually calibrated optimization models correspond to observations as optimal solutions. Chow and Djavadian (2015) showed how sampled heterog eneous parameters can be fit into a mixed logit representation of constrained act ivity schedule choice . Hong et al. (2017) empirically proved th e effectiveness of 7 using invers e optimizatio n for mixed logit ro ute choice m odel parameter estimation with trans it smart card data. W e formalize the multi - agent appro ach for network route choice such that observations are taken from agents, estim ated parameters are heterogeneous and consistent with tho se agent observations, and further used to infer system - level network para meters tha t impact those observations. 3 Proposed methodolog y We firs t formalize th e multi - agent IO framew ork for general networks with route assignment , and then expand that framework to a methodology to infer network parameters. 3.1 Basic m ulti - agent in verse transportation problem framework Consider a network  (  ,  ) that receives observations from a population  of agents behaviorally seeking to travel from an origin     to a destination     ,    a ccording to a shortest path in terms of additive l ink costs . Each agent    has a perception of network parameters in a subnetwork     ; these varying perception s are reflected in heterogeneous parameters at the system level. In the basic mu lti - agent inverse transportation problem framework , let us assume there are no congestion or capacity effects, and only heterogeneous link costs are present. In othe r words, each link cost   ,    , is described by a distribution over  such that an agent’s perceived values of   ,  j ustify their r e vealed route choice    . Parameter learning is achieved with a set of in verse shortest path problems   (   ,   ,    ) , one for each agent, con strained to have an invariant common prior, as illustrated in Fig. 2 and in Eq. (6) and Eq. (7). Fig. 2. Illustration of multi - agent invers e optimi zation as a fixe d point problem. min   ,   {|      | :    =     (   ,   )} ,    (6) Subject to   = 1 |  |      (7)   (   ,   ,    )    |  | ,   ,  |  |   …      |  | 1 |  |     Fixed point 8 Since  is a sh ortest path problem , which is an LP, each agent’s inverse L P (InvLP) can r each a unique LP solution under   norm (Ahuja and Orlin, 2001) for a given initial prior   . The constraint in Eq. (7) interprets the prior as a common prior (Feinberg, 2000; Chow and Recker, 2012) among the population. G iven an initial guess of a common prior, a convergent iterative algorithm (e.g. Method of Successive Average s (MSA)) would reach a unique and stat istically consistent fi xed point with respect to that guess ( see Chow and Recker, 2012). If we change the starting poin t (for exam ple shifting al l   values fro m all 1 to all 1000), it would lead to a different fixed point solution set. This is similar to how the estim ated para meters in random utility models are also only unique relative to each other, but the overall values can be scaled up or down . An MSA -based algorithm is shown here. Algorithm 1: MSA - based algorithm to solve Eq. (6) to Eq. (7) 0. Given an initial common prior    (e.g. previous update), and set  = 1 . 1. For each agent    , solve a n inverse shortest path problem    =   (   ,    ,    ) . 2. Set average to   =  |  |      . 3. Update common prior:    =      +     . Set  =  + 1 and go to step 1 if s t opping criterion not reached. If a sample    of the popul ation i s used to infer the fixed point and distribution of the heterogeneous parameters, Chow and Djavadian (2015) show that a constrained mixe d multinom ial logit utility function as shown in Eq. (8) can aggregate agent observations. In a discrete choice m odel of route choice, each individual selec ts a route that m aximizes their utility. The utility function can correspond to the objective function estimated for the individuals’ sho rtest path problems. Consider Eq. (8) as a random utility function representing a whole population.   =     +   (8) where   is the utility for alternative (route)  by agent  ,   is a random coefficient vec tor that is normally distrib uted,   is an attribut e of the alternative, and   is a Gum bel - distributed error. To compute the probability from the normal distribution of the random coefficients, one can simulate a set of  draws such that a simulated probability can be computed: Pr   ,    =  |  |  Pr   ,     . For the  agents, t he objective function of each agent is equivalent to a “random draw” of the random coef ficients   of the utility function in the mixed logit model if we assume  =  . While the discussion here focuse s on link travel costs only, the framework is applicable to other types of network flow parameters. For exam ple, in Chow and Djavadian (2015), activity routing models capture parking duration, schedule delay, schedule makespan, and number of trip chains in the objective function. Hong et al. (2017) estimate the transit time, transfer, and crowding parameters in their route choice model. 3.2 Network parameter inference v ia decomposition The methodology is further expanded to directly consider network parameters. Consider the capacitated multicommodity flow problem in Eq. (9) to Eq. (12), where  is the set of commodities and  = {   } is a vector of capacity constraints for a subset of links in the network . 9 min        (cost minimization) (9) subject to   =   ,    (flow conservation for each commodity  ) (10)       (bundled capacity constraints) (11)    0,    (non- negativity) (12) The inverse problem to infer the values of  from observed   and other network parameters i s NP - hard (Güler and Hamacher, 2010). Instead of tackling this inverse problem directly, we seek dual prices  corresponding to the constraint set (11). A value of   = 0 means that a link    is not operating a t capacity   , while   > 0 reflects the impact of a binding capacity on agents’ route choices. In other words, we do no t concern ourselves with finding capacity, but instead with finding the effects of the capa city and its intera ction with the agents. In this problem, we assume link costs are not heterogeneous and known in advance. Instead, each agent has a perceiv ed value of the dual p rice of the capacitated links. The capacitated problem can be decomposed into a master problem for determining optimal dual prices and unconstrained sub- problems for each commodity. The dual pri ce is reflected within ea ch agent’s shortest path problem through the Partial Dualization Theorem (Ahuja et al., 1993) : th e  correspondi ng to Eq. (11) in the m ulticommodity f low problem is equivalent to a  for the uncapacitated shortest path problem of each agent    as shown in Eq. (13). min   (  +  )    (13) By relying on this relations hip, we introduc e a multi - agen t inverse transp ortation pro blem to infer the network dual prices. E ach agent solves an IO where ther e is a common prior dual price vector   . We define two non- negative decision variables   and   for each agent such th at      =      , and solve Eq. (14) to Eq. (18) for each agent, subject to Eq. (19) for all agents. In other words, route dependencies are captured by bundle constraints such as capacity (Eq. 11). With decomposition, the original problem is decomposed into individual shortest path problems where the costs in the objec tive are updated to r eflect the dual price obtained from the restric ted master problem (Eq. 13). In the inverse problem, the requirement for a common prior (Eq . 19) ensures the solution will fit the bundling constraints. min   ,   ,     +   ,    (   norm minimization for each agent    ) (14) subject to       +      +   ,    (dual feasib ility) (15)     = (  +      +   )     ,    (strong duality) (16) 10      +    0,    (dual price non -negativity) (17)   ,   ,    0,    (non- negativity) (18)   = 1 |  |      (common prior) (19) T he formulation in Eq. (14) to Eq. (18) refer s to an inverse of a generic LP in standard form as expressed in Eq. (1). In the case of taking the inverse shortest path, ther e are equality constraints so the dual v ariables for th at problem are unbounded. The f ollowing three assertions are made. Proposition 1 . Eq. (14) to Eq. (19) has a unique solution in a common prior dual price vector for all capacitated links, and this vector is the same for all agents, i.e.   =      . Proof. A multicommodity flow problem solution has a unique set of dual prices (Ahu ja et al., 1993). This homogeneity occurs because the dual price is a lower bound threshold for each individual, and the highest value price is the one kept. This can be illustrated with two agents A and B sharing a link  . Suppose agent A would leave lin k  if the dual price was   . This means any value of     would incentivize agent A to leave link  . Now suppose agent B has a dual price of   >   . Any common prior price      <   would not be fixed, because agent B would perturb up towards   while agent A w ould be indifferent, until the common prior and final prices become fixed at   =   , and both agent A and B share the same   .  Proposition 2. The unique inverse optimal parameters to Eq. (14) to Eq. (19) can be reached by starting with an initial guess at    = 0 and t hen following a basic iterative update of      |  |      . Proof . Since    = 0 represents the lower boundary, in each iteration  the updated average of    would always be increasing due to the lower threshold con dition explained in the Proposition 1 proof. This means a basic iter ative update of letting      |  |       is monotonically increasing. Therefore, i t is guaranteed to reach th e unique solution.  The algorithm is explicitly shown here. Algorithm 2 : Itera tive algorithm to solve Eq. (14) to Eq. (19) 0. Given an initial common prior    (e.g. previous update), and  = 1 . 1. For each agent    , solve an inverse shortest path problem with augmented link costs in Eq. (13),    =   (   ,    ,    ) . 2. Update common prior:    =  |  |      . Set  =  + 1 and go to step 1 if        . Proposition 3. The unique inverse optimal parameters to Eq. (14) to E q. (19) can be reached in polynomial time using the basic iterative update from Proposition 2. Proof . Each run of the agent IO problem is an LP which is polynomial time solvable. The number of iterations o f the iterative up date is finite. This can be shown in a worst case scenario; supp ose 11 out of |  | agents, |  |  1 of them all exhibit d ual price of 0 for a particular link wh ile one agent  has a dual price of   > 0 . In this case, in each iterat ion all the |  |  1 agents would keep setting the  ’s to 0 and agent  ’s to   . This means in the wor st case the average will always be increasing by   |  | as a finite step size until the optim um is reached .  These properties of the methodology signify the effectiveness of using agent observations to learn network param eters. W e illustrate the methodology for two ite ration s . Consider three link flows observed in the network in Fig. 3,  = { 100 , 200 , 100 } . We can assume there are three groups of homogeneous agents, agent group 1 ch oosing link 1, agent group 2 choosing link 2, and agent group 3 choosing l ink 3. Each agent group seeks a dual price to explain their link choice , resulting in nine values of  (for each agent and each link), an d three values of   . Fig. 3. Toy network use d for illustra ting methodolo gy. The algorith m is initiated by setting    = { 0,0,0 } . An inverse shortest path problem is run for each agent. For agent group 1,    = { 0,0,0 } because they are already traveling on the shortest path with dual prices at zero. For agent group 2 to choose link 2, a value of    = { 1,0,0 } is needed. Lastly for agent group 3 to c hoose link 3, a value of    = { 3,2,0 } is needed. At the end of this iteration, the weighted average of the three agents is taken as the new prior:    = 󰇥     ,   , 0 󰇦 = 󰇥   ,   , 0 󰇦 . If this is advanced a sec ond iteration, we would get    = 󰇥   ,   , 0 󰇦 ,    = 󰇥   ,   , 0 󰇦 , and    = { 3,2,0 } . These would lead to a new prior    = 󰇥 (      )  ,       , 0 󰇦 = 󰇥   ,   , 0 󰇦 . By inspection we can see tha t the dual prices will ap proach    =    =    =    = { 3,2,0 } . 3.3 Online learning Suppose we have a system that receives agent routes as the y are revealed in real tim e. The multi - agent IO model works in that sett ing without having to estimate pop ulation lev el parameters . It is assumed that each time a traveler updates t heir route decision , that inform ation is sent to a system that learns from the observation to update link capacity dual prices. This information can then be used to m onitor how system changes are affecting traveler decisions in r eal time. We augment Algorithm 2 to this setting as Algorithm 3. Algorithm 3 : online learning algorithm to update system 0. Given: an initi al common prior (obtained fro m a s ystem)    . 1. For newly arrived agent    , solve an inverse shortest path problem with augmented link costs in Eq. (13),    =   (   ,    ,    ) . 2. Update common prior:    =     .   = 3   = 4   = 6 12 4 Numerica l experim ents Four experiments are conducted. The f irst is perform ed on a small netw ork to evalua te the proposed method without capacity eff ects . The second and third tests are conducted on the Nguyen- Dupuis network with capacity effects . W e perform a parameter recovery test to see whether hidden dual prices can be recovered using the methodology. W e also verify that the method can be applied in an online multi - agent learning setting. In the fourth test, the online learning is demon strated u sing real data from a freeway network in Queens, New York City, a nd Google Maps real - tim e shortest path queries over a 3 - hour period. All the data sets generated for these tests are publicly ac cessible on https://g ithub.com /BUILTNYU/N etwork - learning -via- m ulti - agent-inverse-transportation-problems . 4.1 Verification of meth od to estimate h ete rogeneous link cost parame ters 4.1.1 Experiment 1 design T his experiment has two primary objectives. The first is to illust rate the capability o f the proposed method to capture heterogeneity of users’ preferences at one network level ( li nk costs) even when observations are made at another lev el ( rout e cho ice). Th is objective is achieved by using a simple network with enumerated paths, and simulated link costs that vary across the population. These link cost variations reflect dif ferent traff ic and environm ental conditions (e.g. weather and road surface conditions) present during each user’s trip, while observable route choices may be obtain ed from GPS, p hone, or transit sm art card data (to vary ing degrees). L ink cost heterogene ity is reflected in distributions of the link costs across the population. The second objective is to demonstrate how the proposed m ethod can better handle structural changes in the underlying network. This is accomplished by applying the estimated models on a scenario where one of the links is removed. To give the results more co ntext, we estimate two discrete choice models: a n aggregate multinom ial logit model for route choice , and a mixed multinomial logit model that allows distribution s in the path cost taste pa rameter. In total, the following scenarios shown in Table 2 are evaluated. Parameter estim ation is run for the first six scenar ios. Table 2. Scenarios evaluated in Experiment 1 No. Scenario Model 1 Baseline, independent links Multinomial logit 2 Mixed multinomial logit 3 Shortest path pr oblems calibrated w ith Algorithm 1 4 Baseline, correlated links Multinomial logit 5 Mixed multinomial logit 6 Shortest path pr oblems calibrated w ith Algorithm 1 7 Link 3 re moved, i ndepende nt li nks Multinomial logit 8 Shortest path problems calibrated with Algorithm 1 9 Link 3 re moved, c orrelat ed links Multinomial logit 10 Shortest path problems calibrated with Algorithm 1 4.1.2 Experiment 1 data C onsider a network as shown in Fig. 4 with five links identified in blue, where t here are 500 agents traveling from node 1 to node 4 . For the independent links s cenario , the perceived link costs of the 500 agents are randomly s imulated resulting in mean link costs of   = ( 0. 49 , 0. 50 , 0. 50 , 0. 48 , 0. 49 ) and standard deviations of   = ( 0. 29 ,0. 28 ,0. 29 ,0. 29 ,0. 28 ) . The 13 data is available on the GitHub site noted in Section 4 as Test Set 1 . There are only three paths in their choice set represented by the following link sequences: ( 1,4 ) , ( 2,5 ) , ( 1,3,5 ) , where their average path costs are 0.97, 0.99, and 1.48 , respectively . Based on the sim ulated perceived costs and assumption that the travelers choose shortest routes, 48% choose ( 1,4 ) , 48% choose ( 2,5 ) , and 4% choose ( 1,3,5 ) . Fig. 4. Tes t network f or Section 4.1 experiment with node ( in black) and link IDs (in blue ). For the network scenario with correlated link costs, link 3 and link 5 are simulated to have positive correlation of 0.35. The simulated average link costs across the population of 500 agents are   = ( 1. 72 , 2. 03 , 0. 71 , 1. 48 , 1. 10 ) , while the standard devia tions are   = ( 0. 40 ,0. 41 ,0. 23 ,0. 42 ,0. 31 ) . The a verage path costs in this correlated networ k are 3.20, 3.13, and 3.53, corresponding to paths 1, 2, and 3. The data are available on the GitHu b site noted in Section 4 as Test S et 2 . Based on the simulated perceived costs and sam e assumption as abo ve, 43% choose ( 1,4 ) , 48% choose ( 2,5 ) , and 9% choose ( 1,3,5 ) . For the proposed method, an initial common prior of   = 0.5 is assumed for all links. Algorithm 1 is employed to obtain an estimate of link costs for each of the 500 a gents such that their posterior m ean values are within a tolerance of 0.001 of the prior values. Since there are three routes, the re are only tw o degrees of freedom for link costs to vary, so we do not exp ect estimated distributions to reflect more than two alternative options. For the aggregate multinomial and mixed logit models, the utility functions are based on route costs to be consistent w i th the route choices. This is b y design to contrast the outcom es of the proposed method. For the logit models, the average path costs are assumed to be known as the explanatory path cost variable   for each alternative  .   =     +   is an aggre gate utility function that is dependent only on the same average path cost variables for everyone .   i s set to be the utility of path 2 ( 2,5 ) relative to path 1 ( 1,4 ) :   =   +         , while   is the utility of path 3 ( 1,3,5 ) relative to path 1:   =   +      . In the mixed logit,   is normally distributed. 4.1.3 R esults : Heterogeneity? We first estim ate the parameters for th e independent and correlated networks u sing the proposed method. Algorithm 1 is employed with the convergence shown in Fig. 5 for the independent network . Based on a tolerance of 0.001 , the alg orithm term inated after 2 2 iterations for the independent network data set and 19 iterations for the correlat ed network data set . 1 2 3 4 1 2 3 4 5 14 Fig. 5 . Convergence of algorithm 1 on t est network for the independent li nk costs scenario . The results confirm our hypothesis. Fig. 6 illustrate s how the multi - agent inverse optim ization outputs a distribution of link costs across the population ba sed on observation of their route choices and the reliance on the normative route choice behavior in the inverse transportation problem. The values are {0. 489 , 0. 498 , 0 . 009 , (0. 490 , 0. 493 ), (0. 481 , 0. 484 )} on the independent network corresponding to links 1 to 5, and {( 1. 689 ,1. 693 ) , ( 2. 061 ,2. 065 ) , 0 . 371 ,1. 476 ,1. 104 } on t he correlated n etwork. The link costs end up being homogeneous for the first three links in the independent network (latter three links in the correlated network) , a nd are split over two diff erent values for the remaining two links (the firs t two link s in the correlated network) . This reflects how even a network with only two degrees of freedom in information can lead to an estimation of heterogeneous link costs. Fig. 6 . Output distributio n of posterior l ink costs acr oss the populati on of 500 simul ated agents f or link 1 (top) to link 5 (bottom), f or (a) independen t link network and (b) correl ated link netw ork. In summa ry, the tests in these sc enarios verif y that the m ulti - agent inv erse transportatio n problems can estim ate heterogeneous link costs based only on observed route cho ices regardless (a) (b) 15 of whether the links are independent or correlated . By comparison, the study by Hong et al. (2017) looks at route level attributes only, and Cho w and Recker (2012) stick to same - level l ink observations and link costs in activity routing. For context, the route choices are modeled using multino mial and mixe d logit model s in  using the average route costs as the e xplanatory variables . The estimated multin omial logit m odels have log - likelihood values of  417 . 24 for the independent network and  469 . 47 for the correlated network. Th e McFadden R^2 values are 0. 240 and 0. 145 , respectively. For the mixed logit model, a sampling of 100 simulated draws is used to obtain the results . U sing the Broyden - Fletcher - Goldfarb - Shanno (BFGS) method, the algorithm converges to an estimate af ter 4 iterations. F or the mixed logit model ,  =  417 . 23 and   = 0. 240 for the network with independent link costs. For the correlated network, the LL=  469 . 47 and   = 0. 145 . The estimated coeff icients are shown in Table 3. Since the two networks do not have the same link cost distributions, a direc t comparison of the results is not expected. However, the resu lts clarify the value of the multi - agen t inverse transportation problems when interpreted alongside one another. • Whil e the proposed method endogenously obtained the average link costs, the statistical models required prior information about the average path costs in order to be estimate d. • The statistical m odels clearly do not provide esti mates of link - level para meters, much less link - level hetero geneity. • The estimated results suggest that the standard d eviations of the mixed logit mod els (and hence the distribution assumption for taste variation in path costs) for both the indepe ndent and correlated networks ar e statisti cally insign ificant (t - stats of 0.0081 and 0.0029, respectively). Despite the re being pat h level vari ation in per ceived costs , it is diff icult to cap ture this heterogeneity using the mixed logit model for this example. • The proposed method correctly fits each individual’s route choice to obtain 100% fit t o the data of 48% for path 1, 48% for path 2, and 4% for path 3. On the other hand, t he estimated shares from MNL are 50 . 2% for path 1, 45 . 7% for path 2, and 4. 1% for path 3. Similarly, in the correlated netw ork, the observed shares are 43% for path 1, 47 . 8% for path 2, and 9. 2% for path 3 , while estimated shares f rom MNL are 38 . 8% , 51 .3% , and 9. 9% . Table 3 . Estimated parameters and significance tests for multinomial and mixed multinomial logit model Multino mial Logit , indepe ndent ne twork Variable Estimate Standard error t- statistic   4. 93040 0. 44997  10 . 957 *** Mixed Logi t, indepe ndent net work Variable Estimate Standard error t- statistic   4. 93848 2. 08812  2. 3650 *  .  0. 18917 23 . 22305 0. 0081 Multinomial Logit, correlated link netwo rk Variable Estimate Standard error t- statistic   4. 05413 0. 38268  10 . 594 *** Mixed Logit, correlated link network Variable Estimate Standard error t- statistic   4. 054811 0. 656032  6. 1808 *  .  0. 070375 24 . 292289 0. 0029 Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 16 4.1.4 Results: what happens when a link breaks down and the network changes? To illustrate the method’s ability to evaluate significant structural changes in the network, we consider a scenario where one of the links fail . Scenarios 7 to 1 0 deal with closing link 3 for both the independent and correlated networks . Under the new scenarios, the e stimated models are applied to validate their accuracy i n terms of total route shares. Whe n link 3 is closed, the alternative path 3 no longer exists, and there are only two routes choices left. Under these scenarios, the simulated observed routes show th at 50% of the travelers ta ke path 1 in the independent network, while 47.6% take path 1 in the correlated network. The shortest path assignment using the link costs estimated with the multi - agent inver se optimization indicate with 100% fit the optim ality of the observed choices. For context, the statistical models show som e error as reported in Table 4. Since the m ixed logit estimation w as a poor fit with statisticall y insignificant sta ndard deviatio ns, that m odel is not applied in these scenarios. Table 4. Estimated shares (MNL) vs. actual shares of route choices when link 3 is closed (scenarios 7 – 10) Multino mial Logi t, inde pendent network Alternatives Estimated Shares Actual Shares Error Path 1 0. 524 0.5 0. 024 Path 2 0. 476 0.5 0. 024 Multinomial Logit, correlated link netwo rk Alternatives Estimated Shares Actual Shares Error Path 1 0. 431 0. 476 0. 045 Path 2 0. 569 0. 524 0. 045 4.2 Link capacity dual price es timation parameter reco very test 4.2.1 Experiment 2 design and data inputs In the second and third experiments, the Nguyen -Dupuis (1984) net work shown in Fig. 7 is used. In th e second experiment, the goal is to conduct a param eter recovery test. Based o n Proposition 1, the dual prices are unique and homogeneous across the population of agents. It should therefore be possible to assume link capacities on the network, solve a multicommodity flow problem to simulate the “observed” flows, and then apply Algor ithm 2 to recover the dual prices. The standard demand and link cost parameters from the Nguyen - Dupuis network is assumed: 400 travelers for OD ( 1,2 ) , 600 travelers for OD (4,2) , 800 travelers for OD ( 1,3 ) , and 200 travelers f or OD ( 4,3 ) . By design, the paths in the Nguyen - Dupuis network can be easily enumerated. These are s orted by length and shown in Table 5 with the corresponding path IDs. Initial capacities of 400 at link 1 and 800 at link 7 are assumed to simulate the observed flows. For simulating the path sampling, each of the path s is randomly drawn with probability equal to the pe rcent flow on that path from the s olution to the multicomm odity flow problem . A summary of 100 sampled paths is provided in Fig . 8, and the data set is fully accessible on the GitHub site as Test Set 3 . Although the m ulticommodity flow problem may require an integer solution, in this case an LP - relaxed soluti on is obtained revealing dual prices of    = 7 and    = 5 . The solution of the flow assignment under the hidden link capacities is used t o represent the simulated observation, as shown in Fig. 1. The use of paths is dictated by ascending order of cos ts. For example, if an agent for OD ( 1,2 ) chooses to take path 2, it is because the dual price of path 1 has an effective value o f 4 or more. 17 Fig. 7 . Nguyen - Dupuis (1984) network. Table 5 . Enumer ated pat hs for each of the four OD pairs, sorted by length in ascen ding order 9 7 9 12 9 3 7 14 5 5 13 10 9 9 9 9 8 11 6 (1) (7) (6) (5) (4) (3) (2) (13) (12) (11) (10) (9) (8) (17) (16) (15) (14) (19) (18) OD Pa i r Pa th N o de s e que nc e L i nk s e que nc e L e ngth OD Pa ir Pa t h N o de s e que nc e L i nk s e que nc e L e ngth (1, 2) 1 1- 5- 6- 7- 8- 2 (1)- (5 )- (7)- (9 )- (11 ) 29 (1, 3) 14 1- 5- 6- 7- 11- 3 (1)- (5 )- (7)- (1 0)- (1 6) 32 2 1- 5- 6- 7- 11- 2 (1)- (5 )- (7)- (1 0)- (1 5) 33 15 1- 5- 6- 10- 11- 3 (1)- (5 )- (8)- (1 4)- (1 6) 37 3 1- 5- 6- 10- 11- 2 (1)- (5 )- (8 )- (1 4 )- (15 ) 38 16 1- 5- 9- 10- 11- 3 (1)- (6 )- (12 )- (14 )- (16 ) 40 4 1- 5- 9- 10- 11- 2 (1)- (6 )- (12 )- (14 )- (15 ) 41 17 1- 5- 9- 13- 3 (1)- (6 )- (13 )- (19 ) 36 5 1- 12- 6- 7- 8- 2 (2)- (1 7)- (7 )- (9)- (1 1) 35 18 1- 12- 6- 7- 11- 3 (2)- (1 7)- (7 )- (10 )- (16 ) 38 6 1- 12- 6- 7- 11- 2 (2)- (1 7)- (7 )- (10 )- (15 ) 39 19 1- 12- 6- 10- 11- 3 (2)- (1 7)- (8 )- (14 )- (16 ) 43 7 1- 12- 6- 10- 11- 2 (2)- (1 7)- (8 )- (14 )- (15 ) 44 8 1- 12- 8- 2 (2)- (1 8)- (1 1) 32 (4, 2) 9 4- 5- 6- 7- 8- 2 (3)- (5 )- (7)- (9 )- (11 ) 31 (4, 3) 20 4- 5- 6- 7- 11- 3 (3)- (5 )- (7)- (1 0)- (1 6) 34 10 4- 5- 6- 7- 11- 2 (3)- (5 )- (7)- (1 0)- (1 5) 35 21 4- 5- 6- 10- 11- 3 (3)- (5 )- (8)- (1 4)- (1 6) 39 11 4- 5- 6- 10- 11- 2 (3)- (5 )- (8 )- (1 4 )- (15 ) 40 22 4- 5- 9- 10- 11- 3 (3)- (6 )- (12 )- (14 )- (16 ) 42 12 4- 5- 9- 10- 11- 2 (3)- (6 )- (1 2)- (1 4)- (1 5) 43 23 4- 5- 9- 13- 3 (3)- (6)- (1 3 )- (1 9 ) 38 13 4- 9- 10- 11- 2 (4)- (1 2)- (1 4)- (1 5 ) 37 24 4- 9- 10- 11- 3 (4)- (12 )- (1 4)- (1 6) 36 25 4- 9- 13- 3 (4)- (1 3)- (1 9) 32 18 Fig. 8 . Histogram of simulated route observations in Experiment 2 . Based on this observatio n, we assume there are six dist inct agent groups, where al l members of the group are homogeneous since no additional information is available in this experiment. Assuming that we know there are capacities at link 1 and link 7 but their values are unknown, the inverse shortest path problem   (   ,    ,    ) is illu strated belo w for an agen t (path 8 in Table 5) going from node 1 to node 2 from a prior of    = [ 0,0 ] . The  values are the dual variables of the original sho rtest path p ro blem. Because the shortest path co nstraints are equal ity constraints, th e dual prices here are unbounded. min   =   ,   +   ,   +   ,   +   ,    .  .   +    7 +   ,      ,   +   ,     +    9   +    9   +    12   +    3   +    9   +    5 +   ,      ,   +   ,     +    13   +    5   +    9 +       9   +    10   +    9   +    6 +       9       8       7       14       11 0 5 10 15 20 25 30 35 8 9 17 18 19 25 Frequency Path ID 19   ,      ,      ,     ,      ,      ,    400   + 400   = 400 ( 9 + 9 + 14 )  ,   0 4.2.2 Results W e run Algorithm 2 to seek the corresponding dual prices that led to this flow observation. The convergence of the link capacity dual prices is shown in Fig. 9. Fig. 9 . Convergence of dual prices u sing Algorithm 2. This test indicates tha t it is indeed possible to use the proposed method to update network parameters that influen ce the agents’ route choices. In addition, all the agents end up with the same dual price values as the priors. This test shows how our proposed method can use individual agent learning to inf er the value of shared system resources. 4.3 Verificatio n of method in an online multi - agent learning environment 4.3.1 Experiment 3 design and data inputs In th e third experiment, we wish to verify the applicability of the proposed method as a network monitoring tool. It is assumed that data is received in real time from one agent at a time. After each agent observation, an update is conducted to learn of any changes in the dual prices in the network . The experiment is des igned as follows. W e change link 7 from a capacity of 800 to a capacity of 500 and once again solve the capacita ted assignment problem . In this state, the dual prices are found to be    = 7 and    = 6 . Next, we randomly draw observations from the two states: the first 100 sequential samples are drawn from observations under the initial 800 capacity sta te, followed by 100 sequential samples un der the 500 capacity state represent ing the capacity drop in link 7, and finall y another 100 sequential samples under the 800 capaci ty state representing a return to initial st ate. The data is sum marized in Fig. 10 and accessible on GitHu b as Test Set 4 . The time of e ach arrival is assumed to b e const antly distributed to be one unit of time. 0 1 2 3 4 5 6 7 8 0 5 10 15 20 25 30 35 40 45 50 Dual price Iterations Link1Price (iter) Link7Price (iter) 20 Fig. 10 . Trajectory of simulate d routes ob served in Experiment 3 with regime changes m arked by blue vertical lines . Fig. 10 shows there are key paths that directly affect the dual prices when they are observed. For example, path 13 only appears when the syste m is operating under the 500 capacity regime, while path 18 only appears in the 800 capacity regime. We test to see if the methodology, when operated in an online learning environment, is sensitive to these regim e changes. 4.3.1 Results The online updating approach in Algorithm 3 is employed. E ach update uses the posterior of the previous update as its prior. We get the following trajectory of t he posterior dual prices shown in Fig. 11 as an example of how the monitoring occurs over 300 sequential observati ons. The result shows that the proposed m ethod is indeed sensitive to regime changes in this example, even as there is a learning period af ter each state change. The learning rate de pends on the likelihood of the right observation that comes along to reveal t he need for a change. For example, the change to 500 capacity state does not impact the monitoring of the dual price immediately . It is not until a new route observation of path 1 3, indicating a detour in route beca use of the decreased capacity , does the dual price shift. As a result, the sampling rate is important. T he routes are also important. In this case, the monitoring system is able to de tect a shift back and forth because t he 500 capacity sta te leads to a diff erent set of routes than the 800 capacity state. If the routes remain the sam e, no change may be detected. 0 5 10 15 20 25 30 1 10 19 28 37 46 55 64 73 82 91 100 109 118 127 136 145 154 163 172 181 190 199 208 217 226 235 244 253 262 271 280 289 298 Path ID 300 Sample Draws 21 Fig. 11 . Dual price trajectories based on 300 simulated agent arrivals operating in three separate states. As designed, the monitoring system does not currently allow the dual prices to deflate to zero. If, for example, a link that was initially operating at ca pacity but is now no longer at capacity, the system would not be able to detect a lack of detour flows. One possible solution is to build in a time value component, so that sampled data will also include their inter - arrival times. Longer periods of time of inactivity would result in di scounting of the dual prices back to zero. However, this would require proper calibration based on demand densities and sampling rates. We will look into this issue in future research. 4.4 Illustration of online netw ork learning u sing real data fro m Queens, New York 4.4.1. Data and experimental design In this fina l experimen t, we illustr ate network lea rning using a real data example. A highway network from Queens in New York City is shown in Fi g. 12 overlaid upon a Google Maps image. The link free flow travel tim es (“FF time”) are presented in Tab le 6. The network is designed to have two entries/exits for each of the four cardinal directions. On June 5, 2017, we queried a series of shortest paths from Google Maps API based on Google’s real - time travel times. The queried data, along with the network information and network learning code, are all located in the GitHub site. The following steps are taken to obtain this data. 1. Initiate with dual prices equal to zero for all links in the Queens freeway network. 2. Starting at 6:30AM, and every 5 minutes thereafter until 9:30AM, a. Randomly choose one cardinal direction as the origin and one as the destination. b. Sample the four real - time shortest paths f or each of the possible entry/exit pairs. For example, if origin is North and destination is South, there a re four sh ortest paths: N1-S1, N1-S2, N2-S1, N2-S2. c. Keep the one that is sho rtest among these as the sim ulated observation . d. Run Algorithm 3 to update the link dual prices based on the observation. 0 1 2 3 4 5 6 7 8 0 50 100 150 200 250 300 Dual Price Online agent arrivals link 1 link 7 800 capacity state 500 capacity state 800 capacity state 22 A s congestion occurs in the network, the effects of the capacity on shifting routes should be recognized by the network learning algorithm. The dual prices shou ld reflect links that become more congested with binding capacity effects that resul t in route diversions. The magnitudes of the dual prices should give a relati ve measure of the insufficient capacity in the link with respect to other links. W e take snapshots of the shortest paths found in Google Maps (where congested links are typically red to black in color) so that qualitative com parisons can be made , as a comp rehensive quantitative com parison is not possible with these la tent variables . Fig. 12 . Queens freeway network . Table 6. Link attributes f or the Queens freeway network. Link_id start_node end_node FF time (s) Link_id start_node end_node FF time (s) 1 W1 1 211 21 9 7 233 2 1 W1 211 22 7 9 233 3 1 2 77 23 9 2 99 4 2 1 77 24 2 9 99 5 2 3 133 25 1 8 95 6 3 2 133 26 8 1 95 7 3 N1 39 27 8 S1 180 8 N1 3 39 28 S1 8 180 9 3 4 113 29 9 S1 180 10 4 3 113 30 S1 9 180 11 4 N2 50 31 8 9 36 12 N2 4 50 32 9 8 36 13 4 5 228 33 W2 8 178 23 14 5 4 228 34 8 W2 178 15 5 E1 54 35 6 5 60 16 E1 5 54 36 5 6 60 17 7 5 109 37 6 S2 101 18 5 7 109 38 S2 6 101 19 7 4 206 39 6 E2 57 20 4 7 206 40 E2 6 57 4.4.2. Queens freeway network exper imental results Fig. 15 shows the trajectory of the link dual prices (the ones that became binding) as they evolve from one new sample update to the next. The figure illustrates the sensitivity of the method to changes in the network parameters over time, despite being ba sed on only 3 7 r andomly sampled individual route choices. W e provide a snapshot of the dual prices for the links as they change every half hour throughout the 3 - hour study period as shown in Fig. 1 4. For comparison, we include Fig. 1 5 to show screenshot s of the Google Maps real - time shortest path s found at the sam e time s as the dual price snapshot s. T he screenshots provide a qualitative validation by indicating the presence of congestion that occurs at similar segm ents and similar time fram es. Fig. 13 . Trajectories of link d ual prices as estimated using Algorithm 3 fo r Queens freeway network over a 3 - hour period. 24 Fig. 14 . Snapshot of mul ti - agent IO ou tput dua l prices at every ha lf - hour wit h non - zero prices represented by heavier arrows. A number of co nclusions are dr awn from th is illustrative ex periment. • Network system attributes like link dual prices can be updated using only sa mples of individual route observations, without need to estimate total link or path flows. This demonstrates t he significance of this methodology in being able to cheaply monitor a transportation network’s system performance over t ime. • The changes show that t he inference m odel is indeed sensitive to ch anges in the sys tem. As traffic increases from 6:30AM to 9:30A M in the study period resulting in m ore spillbacks and incidents impacting link capacities, the set of dual prices st eadily increases on average as shown in Fig. 13. • The accuracy of the inference ca nnot be established quantitatively. However, a visual comp arison between Fig. 14 and Fig. 15 indicate similarities in positive dual prices where congestion occurs. For example, the 7:00AM screenshot shows that the segment between nodes 4 and 5 is highly congested, and that is interpreted correctly in Fig. 14. The 7:30AM screenshot reveals the alternative path traversing the link between nodes 8 and S1 is c ongested, which is captured correct ly in the inference model. The 8:30AM screensho t indicates congestion between nodes 5 and 7, which is also captured by the mod el. T his delay ling ers through 9:30AM, and is properly captured as well by the inference model. 25 Fig. 15 . Screenshots of Google Maps real - t ime short est path i nput data at co rrespon ding half - hour in terva ls that reveal congested segments. 5 Conclusion In this study, we derived a new class of inverse opt imization model s such that shared network resources can be quantified from agent observations. This class is fundam entally different from the classic inve rse optimization model, which requires more statistica lly ineffici ent estimation of aggregate system parameters like population link or path flows. The proposed m ulti - agent inverse optimization model class captures heterogeneity in agents us ing shared syst em resources , and also infer s such system parameters as link capacity du al prices all without having to estimate population flows . T his contributi on make s it possible to cheaply apply IO techniques to many data - driven transportation problems in which observations are obtained from only samples of selfish agents or in an online learning setting. To elaborate, we fo rmalize a multi - agent inverse optimization modeling approach using a fixed point common prior to capture heterogeneity , relate that approach to link capacity dual prices through decomposition properties, and propose three algorithms to support these models. We further prove that the method can obtain unique dual prices for a network shared by the agent population in polynomial time (depending on LP algorithm used) . The methodology is tested in four experiments: 6:30 7:00 7:30 8:00 8:30 9:00 9:30 26 1) For a path - enumerated 4-node network, we verify that the methodology can indeed obtain heterogeneous estimates of link cost parameters, even when the re is not enough structural information f or meaningf ul interpret ation with a purely descriptive method like a mixed l ogit model. 2) For t he Nguyen - Dupuis network, we conduct a pa rameter recovery test to verify that the proposed method works in inferring shared system resources through agent informat ion. Th is test illustrates how m uch easier it is to infer impacts of link capacitie s than to try and estim ate them directly as Güler and Hamacher (2010) tried to do as an NP-hard problem. 3) We constr uct an online learning example to demonstrate how the method can work in t his setting. L ink 7 in the network is set to experience a capacity drop before returning to its origina l state to mimic an incident. The results show that t he online learning is able to pick up on t hat drop through an increase in the dual pr ice observed from updated sequences of agent r oute choices . 4) The onli ne learning is further illustrated using real data from a freeway network in Queens, New York, based on sampled real - time Google Maps shortest path queries. Through these queries, we are a ble t o estimate the link c apacity inte ractions wit h the travelers under congestion, and monitor this evolution over a three-hour period. This work dif fers from other da ta - driven meth ods in the lite rature . L earning is made through the use of a normative be havioral mechanism so t hat online monitoring and strategic plannin g scenario evaluation are possible . The research also has implications for autom ated systems and artificially i ntelligent ne tworks in the co ntext of autonom ous fleets (Guo et al., 2017) and smart cities. By structuring the learning in the same environment as the design and opera tion, it makes it easier in future research to design integrated learning and opti mization strategies in networks. For example, som e recent research is looking at ways to optimize resources to sense and learn from a network (e.g. Ryzhov and Powell, 2011). A next step i n this evolution will be to operate a system that jointly co nsiders resource allocation to optimally serv e users and learn from them . Multi - agent IO is on e way to approach such a problem . Other extensions of this research include: conducting studies using multiple sensor sources (e.g. loop detectors, video cameras, taxi GPS data) and integrating the multi - agent IO approach; considering Bayesian techniques like Markov chain Monte Carlo methods for online learning through sampling (Tebaldi and West, 1998) ; and designing more sophist icated online learning systems that incorporate time value of observations and deterioration rate of dual prices. For example, a new observation that shows dual price is 5 instead of 0 can ha ve different meanings if the observation arrives 1 minute later versus 1 hour later. This temporal component needs to be studied. Other aspects of real applications also need to be considered: d ata can be noisy (e.g. perceived link capacity dual price for agents may diff er) and may require stochastic assignment consideration (Ashok and Ben - Akiva, 2002) , only fragm ents of actual paths m ay be available (e.g. transit fare s mart card data) , or travel ers m ay choose to stay at home . Aggregation methods, while discussed in Chow and Djavadian (2015), can be further expanded upon in this gen eralized route inference setting. 27 Acknowledgments Jia Xu and Joseph Chow were partially supported by an NSF CARE ER grant no. CMMI -1652735 , which is gratefully acknowledged. Preliminary res ults from this work w ere presented at TRISTAN IX in Aruba. Helpful feedback from conference attendees and three anonymous referees is muc h appreciated. Reference s Agar wal, R., Ergun, Ö. , 2 008. Me chanism design f or a mul ticomm odity fl ow game in service network a llianc es. Operations Research Letters 36(5), 520 - 524. Ahuja, R. K., Magna nti, T. L., Orlin, J. B . , 1993. Network flows: theory, algorithms, and applications , Pear son . Ahuja, R.K ., Orlin, J.B. , 2001. Inver se optimization. Operations Re search 49(5), 771 - 78 3. Airoldi, E. M. , Blocker, A. W. , 2013. Estim ating latent processes on a network from indirect measurements. Journal of the American S tatistical Associa tion 108(501), 149 - 164 . Akamat su, T., 19 96. Cycl ic flows , markov process and stoc hastic t raffic as signm ent. Trans portatio n Researc h Part B 30 (5), 369 – 386. Alexander, L., Jiang , S., Murg a, M. , Gonzál ez, M.C., 2 015. Origi n – destina tion trips by purp ose and t ime of da y inferre d from mobile pho ne data . Transpo rtation Researc h Part C 58, 240 - 250. Ashok, K. Ben - Akiva, M.E., 2002. Estimation and prediction of time - dependent origi n - destination flows with a stochasti c mappi ng to pat h flows a nd lin k flows. Transpor tation Science 36(2), 184 - 198. Aswani, A. , Shen, Z. J . M., Siddi q, A. , 201 5. Invers e Optim ization wi th Noi sy Data. arXiv pre print arXiv:1507 .03266. Baillon, J.B., Cominetti, R., 2008. Markovian traffic equilibriu m. Mathe matical P rogrammi ng 111 (1 – 2), 33 – 56.  Bell, M.G., 1991. The estimation of origin - desti nation m atrices by const rained ge neralise d least squares. Transportation Research Part B 25(1), 13 - 22. Ben - Elia, E., Shift an, Y. , 2010. Which road do I take ? A learning - ba sed m odel of route - c hoice beha vior wi th real - time information. Transporta tion Rese arch Part A 44(4), 249 - 264. Bertsim as, D., Gupta, V., Pas chalidis, I. C . , 2012. Inve rse optim ization: A new perspe ctive on the Black - Litterman model. Operations Resear ch 60(6), 1389 - 1403 . Bertsim as, D., Gupta, V ., Pa schalidi s, I. C. , 2015. Data - driven est imati on in equi librium using i nverse opt imiza tion. Mathematical Programm ing 153(2), 595 - 633. Birge, J. R., H ortaçsu, A., Pavl in, M. , 2014. Inverse optim ization fo r the re covery of m arket st ructure from m arket outcomes: An application to the miso electr icity market. Available at SSRN 2612234. Brucker, P., Sh akhlevich, N. V. , 2009. Inverse scheduling with maximum lateness objective. Jour nal of Sched uling , 12(5), 475 - 488. Burkard, R. E., Pleschi utschnig, C ., Zha ng, J. , 20 04. Inver se m edian probl ems. Discrete Optimization , 1(1), 23 - 39. Burton, D., Toint, P . L. , 1992. On an in stance o f the in verse short est pat hs probl em. Mat hematical P rogram ming , 53(1 - 3), 45 - 61. Cascetta, E. , 1984. Estimation of trip matrices from traffic counts and survey data: a ge neralized least squares estimator. Transport ation Re search P art B 18(4), 289 - 299. Cascetta , E., Nuzzol o, A., Ru sso, F., Vit etta, A. , 19 96. A m odified l ogit rout e choice model overcom ing path overlap ping probl ems: s pecificat ion and s ome cal ibration results for inte rurban ne tworks . In : Proc . ISTTT 13 (pp. 697 - 711). Lyon, Fra nce: Perga mon. Chow, J. Y. J. , Dja vadian, S. , 2015. Activity - bas ed market equilibriu m for capacitated multimodal transport systems. Trans portatio n Researc h Part C 59, 2 - 18. Chow, J. Y. J. , Re cker, W. W. , 2012. Inverse optimization with endogenous arrival time constr aints to calibrate the househol d acti vity patt ern prob lem. Transportation Research Part B 46(3), 463 - 479. Chow, J. Y. J. , Ri tchie, S. G., Jeo ng, K. , 2014. N onlinear i nverse opt imi zation for para meter estim ation of comm odity - vehicle - decouple d freight assignm ent. Transportation Rese arch Part E 67, 71 - 91. Daganz o, C . F., S heffi, Y. , 19 77. On st ochast ic model s of traffi c assignm ent. Tr ansport ation Sci ence 11(3), 253 - 274. 28 Day, J., Nem hauser, G. L., Sokol , J. S. , 2002. Managem ent of railroad impeda nces for shortest path - ba sed routin g. Electronic Notes in Theoretical Computer Sci ence, 66(6), 53 - 65. Dia, H. , 2002. An agent - based approa ch to m odelling d river ro ute choic e behavi our und er the i nfluenc e of real - ti me inform ation. Transportati on Research Part C 10(5), 33 1 - 349. Esfahani, P. M., Shafieezadeh - Abadeh, S., Hanasu santo, G. A., Ku hn, D. , 2015. Data - driven Inve rse Opt imizat ion with Incom plete Inf ormati on. arXiv pre print arXi v:1512. 05489. Feinber g, Y. , 2000. Charact erizi ng comm on priors i n the fo rm of post eriors. J ournal of Econo mic Theo ry , 91(2 ), 127 - 179. Fosgera u , M., Fre jinger, E. , Karlstrom , A. , 2013. A link based net work rout e choi ce model with un restrict ed choice set. Transport ation Rese arch P art B 56, 70 - 80. Frejinger, E., Bierl aire, M. , 200 7. Capt uring co rrelatio n with sub network s in rout e choice model s. Tra nsportati on Research Part B 41(3), 363 - 37 8. Gao, S. , 2012. M odeling strategic route choice and real - time information impacts in stochastic and time - dependent network s. IEEE Tr ansactio ns on Int elligent Transpor tation Systems 13(3), 1298 - 131 1. Güler, Ç., Ham acher, H. W. , 2 010. Capac ity inve rse mi nimum cost flow p roblem . Journal of C ombi natorial O ptimization 19(1), 43 - 59. Guo, Q.W., C how, J.Y. J., Schonfeld, P., 2017. Stochastic dyna mic switching in fixed and flexible transit serv ices as market entry - exit real options . Transpor tation R esearc h Procedi a 23, 380 - 399. Hazelton, M. L., 2015. Network to mography for integer - valued traffic. The Annals of Applied Statistics 9(1), 474 - 506. Herrera, J. C., Wor k, D. B., H erring, R., Ban, X. J., Jacobson, Q., Bayen, A. M. , 2010. Evaluati on of traffi c data obtained via GPS - ena bled m obile pho nes: The Mobile C entury fi eld expe rime nt. Transportation Research Part C 18(4), 56 8 - 583. Heuberger, C . , 2004. Inverse c ombinat orial opt imiza tion: A sur vey on p roblem s, methods, and resul t s. Journal of Combinatorial Op timization , 8(3), 329 - 361. Hong, S. P., Kim , K. M. , Byeon, G., Min, Y. H. , 2017. A method to directly derive taste heterogeneity of travellers’ route c hoice in public t ransport from obse rved r outes. Tr ansportat ion Res earch Par t B 95, 41 - 52. Munizaga, M. A., Palma, C. , 2012. Estimation of a disaggregate multimodal public transp ort Origin – Destination matrix from passive smartcard data from Santiago, Chile. Transportation Research Part C 24, 9 - 18. Nguyen, S., Dupu is, C. , 1984. An efficie nt method for computing traffic equilibria in networks with asymmetric transportation cos ts. Transpo rtation Science 18(2), 185 - 202. Prato, C. G. , 2 009. Rout e choic e modelin g: past , present and futu re resea rch direc tions. Journal of Choice Modelling 2(1), 65 - 100. Recker, W. W. , 1995. The house hold acti vity patt ern probl em: ge neral for mulati on and s olution. Transport ation Research Part B , 29(1), 61 - 77. Ryzhov, I. O., P owell, W. B. , 20 11. Inform ation coll ection on a graph. Operatio ns Research , 59(1), 1 88 - 201 . Srinivasan , K., Mahmass ani, H. S., 2000 . Modeli ng inerti a and com plianc e me chanisms i n route c hoice be havior under real - time information. Transportation Research Record (1725), 45 - 53. Tang, J., Liu, F., Wa ng, Y. , Wang, H., 2015. Uncovering urban human mobility from large scale taxi GPS data. Physica A: Statistical Mechanics and its Applications 438, 140 - 15 3. Tarantola, A. , 2005. Inverse probl em the ory and met hods f or model paramete r estimat ion . SIAM . Tebaldi, C., W est, M. , 1998. Baye sian infe rence on networ k traffic using l ink count data. Jo urnal of the American Statistical Association 93(442), 557 - 573 . Van Zuylen, H .J. , Willumsen, L.G., 1980. The most likely trip matrix estimated from traffic counts. Transportat ion Research Part B 14(3), 281 - 29 3. Vardi, Y. , 1996. Network tomography: Estimatin g source - des tination traffic intensities from link data. Journ al of the American S tatistical A ssociation 91(433), 365 - 377. Vovsha, P., Bekhor, S . , 1998. Link - nested logi t model of route choi ce: overc oming rout e overlap ping proble m. Transportation Research Record 1645, 133 - 142. Wang, L. , 2009. Cutting plane algorithm s for the inverse mixed i nteger l inear programm ing proble m. Oper ations Research Letters , 37(2) , 114 - 116. Watling, D . P., Rasm ussen, T. K. , Prato, C. G., Nielse n, O. A. , 2015. Stochastic user equilibrium with equilibrated choice sets: Part I – Model formulations under alternative distributions and res trictions. Transport ation Research Part B 77, 166 - 181. William s, S., White, A., Wai ganjo, P., Or wa, D., Klopp, J. , 2015. The digital matatu project: Us ing cell phones to create an open source data for Nairobi's semi - formal bus system. Journal of Tr ansport G eography 49, 39 - 51. 29 Yang, H., Sasa ki, T., Iida, Y. Asa k ura, Y., 1 992. Estim ation of or igin - destination matrices from link traffic counts on congested netw orks. Transport ation Rese arch P art B 26(6), 417 - 434. You, S. I. , Chow, J. Y. J. , Ritchie, S. G. , 2016. Inverse v ehicle routing for a ctivity - based urba n frei ght forecast modeling and city logistics. Transportmetrica A 12(7), 650 - 673 . Zhan, X., Hasa n, S., Ukkusuri , S. V., Kamga, C. , 2013. Urban link travel time estimation using large - scale taxi data with partial info rmation. Transport ation Res earch P art C 33, 3 7- 49. Zhang, J. , Zhang , L. , 201 0. An augm ented L agrangia n method for a cl ass of in verse qua dratic program ming problem s. Appli ed Mathe matics a nd Optimi zation , 61(1), 57 - 83.