RedQueen: An Online Algorithm for Smart Broadcasting in Social Networks

RedQueen : An Online Algorithm for Smart Broadcasting in So cial Net w orks Ali Zarezade ∗ 1 , Utk arsh Upadh y a y ∗ 2 , Hamid R. Rabiee 1 , and Man uel Gomez-Ro driguez 2 1 Sharif Univ ersity , zarezade@ce.sharif.edu, rabiee@sharif.edu 2 Max Planc k Institute for Softw are Systems, utk arsh u@mpi-sws.org, man uelgr@mpi-sws.org Abstract Users in so cial net w orks whose p osts sta y at the top of their follow ers’ feeds the longest time are more lik ely to be notic e d . Can w e design an online algorithm to help them decide when to p ost to stay at the top? In this pap er, we address this question as a no vel optimal con trol problem for jump sto c hastic differen tial equations. F or a wide v ariety of feed dynamics, we show that the optimal broadcasting in tensity for an y user is surprisingly simple – it is given b y the p osition of her most recen t p ost on each of her follow er’s feeds. As a consequence, we are able to develop a simple and highly efficient online algorithm, RedQueen , to sample the optimal times for the user to p ost. Exp erimen ts on b oth synthetic and real data gathered from T witter show that our algorithm is able to consistently mak e a user’s p osts more visible ov er time, is robust to volume changes on her follo wers’ feeds, and significan tly outp erforms the state of the art. 1 In tro duction Whenev er a user in an online so cial netw ork decides to share a new story with her follow ers, she is often comp eting for attention with dozens, if not hundreds, of stories simultane ously shared by other users that their follow ers follow [ 2 , 12 ]. In this context, recent empirical studies hav e shown that stories at the top of their follow ers’ feed are more likely to b e notic e d and consequently liked or shared [ 15 , 16 , 21 ]. Can w e find an algorithm that helps a user decide when to p ost to increase her chances to stay at the top? The “when-to-p ost” problem was first studied by Spaso jevic et al. [ 25 ], who p erformed a large empirical study on the b est times to p ost in T witter and F aceb ook, measuring attention a user elicits by means of the n umber of resp onses to her p osts. Moreov er, they designed several heuristics to pinp oin t at the times that elicit the greatest attention in a training set and show ed that these times also lead to more resp onses in a held-out set. Since then, algorithmic approaches to the “when-to-p ost” problem with prov able guarantees ha ve b een largely lacking. Only v ery recen tly , Karimi et al. [ 17 ] introduced a conv ex optimization framew ork to find optimal broadcasting strategies, measuring attention a user elicits as the time that at least one of her p osts is among the k most recent stories received in her follow ers’ feed. Ho wev er, their algorithm requires exp ensiv e data pre-pro cessing, it do es not adapt to changes in the users’ feeds dynamics, and in practice, it is less effectiv e than our prop osed algorithm, as shown in Section 5. In this pap er, we design a no vel online algorithm for the when-to-p ost problem, where we measure visibility of a broadcaster as the p osition of her most recent post on her follow ers’ feeds ov er time. A des irable prop ert y of this visibility measure is that it can b e easily extracted from real data without actual in terven tions — giv en any particular broadcasting strategy for a user, one can alwa ys measure its visibility using a separate held-out set of the user’s follow ers’ feeds [17]. In contrast, measures based on users’ reactions ( e.g. , n umber ∗ Authors contributed equally . This work w as done during Ali Zarezade’s internship at Max Planc k Institute for Softw are Systems. 1 of likes, shares and replies) are difficult to estimate from real data, due to the presence of other confounding factors suc h as users’ influence, con tent and wording [7, 8, 20]. More precisely , we represent users’ p osts and feeds using the framework of temp oral p oint pro cesses, which c haracterizes the contin uous time interv al b et ween p osts using conditional intensit y functions [ 1 ]. Under this representation, finding the optimal broadcasting (or p osting) strategy for a user reduces to finding its asso ciated conditional broadcasting intensit y [ 17 ]. Then, for a large family of in tensity functions, which includes Ha wkes [ 14 ] and Poisson [ 19 ] as particular instances, w e find “when-to-p ost” b y solving a no vel optimal control problem for a system of jump sto chastic differential equations (SDEs) [ 13 ]. Our problem form ulation differs from previous literature in tw o key technical asp ects, whic h are of indep endent interest: I. The control signal is a conditional (broadcasting) intensit y , which is used to sample sto c hastic even ts ( i.e. , stories to p ost). As a consequence, the problem formulation requires another lay er of sto c hasticit y . In previous work, the control signal is a time-v arying real vector. I I. The (broadcasting) in tensities are sto c hastic Marko v pro cesses and thus the dynamics are doubly sto c hastic. This requires us to redefine the cost-to-go to incorp orate the instantaneous v alue of these in tensities as additional arguments. Previous work has typically considered constant in tensities and only v ery recently time-v arying deterministic intensities [27]. These technical asp ects hav e implications b ey ond the smart broadcasting problem since they enable us to establish a previously unexplored connection b et w een optimal control of jump SDEs and double sto c hastic temp oral point pro cesses ( e.g. , Ha wkes pro cesses), which hav e been increasingly used to model social activit y [10, 11, 28]. Moreo ver, we find that the solution to the ab o v e optimal control problem is surprisingly simple: the optimal broadcasting in tensity for a user is given by the p osition of her most recen t post on each of her follo wer’s feeds. This solution allows for a simple and highly efficien t online pro cedure to sample the optimal times for a user to broadcast, which can b e implemented in a few lines of co de and do es not require fitting a mo del for the feeds’ in tensities. Finally , we p erformed exp eriments on b oth syn thetic and real data gathered from T witter and show that our algorithm is able to consistently make a use r’s p osts stay at the top of her follo wers’ feeds, is robust to c hanges on the dynamics (or v olume) of her follow ers’ feeds, and significantly outp erforms the state of the art [17]. F urther related work. In addition to the paucity of work on the when-to-p ost problem [ 17 , 25 ], discussed previously , our work also relates to: (i) empirical studies on attention and information ov erload in so cial and information net works [ 2 , 12 , 15 , 22 ], which inv estigate whether there is a limit on the amount of ties ( e.g. , friends, follow ees or phone contacts) p eople can main tain, how p eople distribute attention across them, and how attention influences the propagation of information; and (ii) the influence maximization problem [ 6 , 9 , 10 , 18 , 24 ], which aims to find a set of no des in a so cial netw ork whose initial adoption of a certain idea or pro duct can trigger the largest exp ected num b er of follow-ups. In contrast, we fo cus on optimizing a so cial media user’s broadcasting strategy to capture the greatest attention from the follow ers. 2 Preliminaries W e first revisit the framew ork of temp oral p oin t pro cesses [ 1 ] and then use it to represent broadcasters and feeds in so cial and information netw orks. T emp oral p oin t pro cesses. A temp oral p oint pro cess is a sto chastic pro cess whose realization consists of a sequence of discrete even ts lo calized in time, H = { t i ∈ R + | i ∈ N + , t i < t i +1 } . In recent years, they ha ve b een used to represent man y different types of even t data pro duced in online so cial netw orks and the web, suc h as the times of t weets [10], retw eets [28], or links [11]. A temp oral p oint pro cess can also b e represented as a counting pro cess N ( t ) , whic h is the num b er of ev ents up to time t . Moreov er, giv en H ( t ) = { t i ∈ H | t i < t } , the history of even t times up to but not including time t , we can characterize the counting pro cess using the conditional in tensity function λ ∗ ( t ) , whic h is the conditional probabilit y of observing an ev ent in an infinitesimal windo w [ t, t + dt ) given the history H ( t ) , i.e. , λ ∗ ( t ) dt = P { ev ent in [ t, t + dt ) |H ( t ) } = E [ dN ( t ) |H ( t )] , 2 where dN ( t ) ∈ { 0 , 1 } and the sign ∗ means that the intensit y may dep end on the history H ( t ) . The functional form for the in tensity is often chosen to capture the phenomena of interest. F or example, in the context of mo deling so cial activity , retw eets ha ve b een mo deled using Hawk es pro cesses [ 10 , 28 ] and daily and weekly v ariations on the volume of p osted tw eets hav e b een captured using Poisson pro cesses [ 23 , 17 ]. In this work, we consider the follo wing general functional form, whic h includes Hawk es and Poisson as particular instances: λ ∗ ( t ) = λ 0 ( t ) + α Z t 0 g ( t − s ) dN ( s ) , (1) where λ 0 ( t ) ≥ 0 is a time-v arying function, which mo dels the publication of messages by users on their own initiativ e, the second term, with α ≥ 0 , mo dels the publication of additional messages ( e.g. , replies, shares) b y the users due to the influence that previous messages (their own as well as the ones p osted by others) ha ve on their intensit y , and g ( t ) denotes an exp onential triggering kernel e − wt I ( t ≥ 0) . The second term mak es the intensit y dep enden t on history and a sto c hastic pro cess by itself. Finally , the following alternativ e represen tation will b e useful to design our sto c hastic optimal control algorithm for smart broadcasting (prov en in the App endix): Prop osition 1 L et N ( t ) b e a c ounting pr o c ess with an asso ciate d intensity λ ∗ ( t ) given by Eq. 1. Then the tuple ( N ( t ) , λ ∗ ( t )) is a doubly sto chastic Markov pr o c ess, whose dynamics c an b e define d by the fol lowing jump SDE: dλ ∗ ( t ) = [ λ 0 0 ( t ) + w λ 0 ( t ) − w λ ∗ ( t )] dt + αdN ( t ) , (2) with initial c ondition λ ∗ (0) = λ 0 (0) . In the remainder of the pap er, to simplify the notation, w e drop the sign ∗ from the intensities. Represen tation of broadcasters and feeds. Giv en a directed netw ork G = ( V , E ) with |V | = n users, w e assume any user can b e a broadcaster, a follow er or b oth, each broadcaster can b e follow ed by multiple follo wers, and each follow er can follow multiple broadcasters. Then, we represent the broadcasting times of the users as a collection of counting pro cesses denoted by a vector N ( t ) , in which the i -th dimension, N i ( t ) , is the num b er of messages or stories broadcasted by user i up to time t . Here, we denote the history of times of the stories broadcasted by user i b y time t as H i ( t ) , the entire history of times as H ( t ) = ∪ i ∈V H i ( t ) , and c haracterize these counting pro cesses using their corresp onding intensities, i.e. , E [ d N ( t ) |H ( t )] = µ ( t ) dt . Giv en the adjacency matrix A ∈ { 0 , 1 } n × n , where A ij = 1 indicates that user j follo ws user i , w e can represen t the times of the stories users receive in their feeds from the broadcasters they follow as a sum of coun ting pro cesses, A T N ( t ) , and calculate the corresp onding conditional in tensities as γ ( t ) = A T µ ( t ) . Here, w e denote the history of times of the stories received by user j b y time t as F j ( t ) := ∪ i ∈N ( j ) H i ( t ) , where N ( j ) is the set of users that j follo ws. Finally , from the p erspective of a broadcaster i , it is useful to define the coun ting pro cesses M \ i ( t ) = A T N ( t ) − A i N i ( t ) , in which the j -th dimension, M j \ i ( t ) , represents the times of the stories user j receiv es due to other broadcasters she follows, and A i is the i -th row of the adjacency matrix A . Moreov er, for each of these counting pro cesses, the conditional intensit y is given by γ j \ i ( t ) = γ j ( t ) − µ i ( t ) and the history is giv en by F j \ i ( t ) := F j ( t ) \H i ( t ) . 3 Problem F orm ulation In this section, we first define our visibility measure, r ( t ) , then derive a jump sto c hastic differential equation that links our measure to the counting pro cesses asso ciated to a broadcaster and her follow ers, and conclude with a statement of the when-to-p ost problem for our visibility measure. Definition of visibility . Giv en a broadcaster i and one of her follow ers j , we define the visibilit y function r ij ( t ) as the p osition or r ank of the most recent story p osted by i in j ’s feed by time t , which clearly dep ends on the feed ranking mechanism in the corresp onding so cial netw ork. Here, for simplicity , we assume each 3 Broadcaster Other bro adcasters User’s feed u ( t ) λ ( t ) M ( t ) N ( t ) t t r ( t ) = 1 Figure 1: The dynamics of visibility . A broadcaste r i p osts N i ( t ) = N ( t ) messages with intensit y µ i ( t ) = u ( t ) . Her messages accumulate in her follo wer j ’s feed, comp eting for attention with M j \ i ( t ) = M ( t ) other messages, p osted by other broadcasters j follo ws with intensit y γ j \ i ( t ) = λ ( t ) . The visibility function r ij ( t ) = r ( t ) is the p osition or rank of the most recent story p osted by broadcaster i in the follow er j ’s feed by time t. user’s feed ranks stories in inv e rse chronological order 1 . How ever, our framework can b e easily extended to an y feed ranking mechanisms, as long as its rank dynamics can b e expressed as a jump SDE 2 . Under the inv erse chronological ordering assumption, p osition is simply the num b er of stories that others broadcasters p osted in j ’s feed from the time of the most recent story p osted by i un til t . Then, when a new story arrives to a user’s feed, it app ears at the top of the feed and the other stories are shifted do wn by one. If we identify the time of the most recent message p osted by i b y time t as τ i ( t ) = max { t k ∈ H i ( t ) } , then the visibilit y is formally defined as: r ij ( t ) = M j \ i ( t ) − M j \ i ( τ i ( t )) , (3) Note that, if the last story p osted by i is at the top of j ’s feed at time t , then r ij ( t ) = 0 . Dynamics of visibility . Giv en a broadcaster i with broadcasting counting pro cess N i ( t ) and one of her follo wers j with feed coun ting pro cess due to other broadcasters M j \ i ( t ) , the rank of i in j ’s feed r ij ( t ) satisfies the follo wing equation: r ij ( t + dt ) = ( r ij ( t ) + 1) dM j \ i ( t )(1 − dN i ( t )) | {z } 1. Increases by one + 0 |{z} 2. Becomes zero + r ij ( t )(1 − dM j \ i ( t ))(1 − dN i ( t )) | {z } 3. Remains the same , where eac h term mo dels one of the three p ossible situations: 1. The other broadcasters p ost a story in ( t, t + dt ] , dM j \ i ( t ) = 1 , and broadcaster i do es not p ost, dN i ( t ) = 0 . The p osition of the last story p osted by i in j ’s feed steps down by one, i.e. , r ij ( t + dt ) = r ij ( t ) + 1 . 2. Broadcaster i p osts a story in ( t, t + dt ] , dN i ( t ) = 1 , and the other broadcasters do not, dM j \ i ( t ) = 0 . No matter what the previous rank w as, the new rank is r ij ( t + dt ) = 0 since the newly p osted story app ears at the top of j ’s feed. 3. No one p osts any story in ( t, t + dt ] , dN i ( t ) = 0 and dM j \ i ( t ) = 0 . The rank remains the same, i.e. , r ij ( t + dt ) = r ij ( t ) W e skip the case in whic h M j \ i ( t ) = 1 and dN i ( t ) = 1 in the same time interv al ( t, t + dt ] b ecause, b y the Blumenthal zero-one law [ 3 ], it has zero probability . No w, b y rearranging terms and using that dN i ( t ) dM j \ i ( t ) = 0 , w e uncov er the following jump SDE for the visibility (or rank) dynamics: dr ij ( t ) = − r ij ( t ) dN i ( t ) + dM j \ i ( t ) . (4) 1 At the time of writing, T witter and W eib o rank stories in inv erse chronological order by default and F acebo ok allows choosing such an ordering. 2 This would require either having access to the corresp onding feed ranking mechanism or reverse engineering it, which is out of the scope of this work. 4 where dr ij ( t ) = r ij ( t + dt ) − r ij ( t ) . Figure 1 illustrates the concept of visibility for one broadcaster and one follo wer. The when-to-p ost problem. Giv en a broadcaster i and her follo wers N ( i ) , our goal is to find the optimal conditional in tensity µ i ( t ) = u ( t ) that minimizes the exp ected v alue of a particular nondecreasing conv ex loss function ` ( r ( t ) , u ( t )) of the broadcaster’s visibility on each of her follow er’s feed, r ( t ) = [ r ij ( t )] j ∈N ( i ) , and the in tensity itself, u ( t ) , ov er a time window ( t 0 , t f ] , i.e. , minimize u ( t 0 ,t f ] E ( N i , M \ i )( t 0 ,t f ]  φ ( r ( t f )) + Z t f t 0 ` ( r ( τ ) , u ( τ )) dτ  sub ject to u ( t ) ≥ 0 ∀ t ∈ ( t 0 , t f ] , (5) where u ( t 0 , t f ] denotes user i ’s intensit y from t 0 to t f , the exp ectation is taken ov er all p ossible realizations of the counting processes asso ciated to user i and all other broadcasters from t 0 to t f , denoted as ( N i , M \ i )( t 0 , t f ] , and φ ( r ( t f )) is an arbitrary p enalt y function 3 . Here, b y considering a nondecreasing loss, we p enalize times when the p osition of the most recen t story on each of the follow er’s feeds is high ( i.e. , the most recent story do es not stay at the top ) and w e limit the num ber of stories the broadcaster can p ost. Finally , note that the optimal intensit y u ( t ) for broadcaster i at time t ma y dep end on the visibility r ( t ) with resp ect to each of her follo wers and thus the asso ciated counting pro cess N i ( t ) ma y b e doubly sto c hastic. 4 Sto c hastic Optimal Con trol Algorithm In this section, w e tackle the when-to-post problem defined b y Eq. 5 from the p erspective of stochastic optimal control of jump SDEs [ 13 ]. More sp ecifically , we first derive a solution to the problem considering only one follow er, pro vide an efficient practical implementation of the solution and then generalize it to the case of multiple follow ers. W e conclude this section b y deriving a solution to the problem given an (idealized) oracle that knows the times of all stories in the follow ers’ feeds a priori , which we will use as baseline. Optimizing for one follo wer. Giv en a broadcaster i with N i ( t ) = N ( t ) and µ i ( t ) = u ( t ) and only one of her follow ers j with M j \ i ( t ) = M ( t ) and γ j \ i ( t ) = λ ( t ) , we can rewrite the when-to-p ost problem defined by Eq. 5 as minimize u ( t 0 ,t f ] E ( N ,M )( t 0 ,t f ]  φ ( r ( t f )) + Z t f t 0 ` ( r ( τ ) , u ( τ )) dτ  sub ject to u ( t ) ≥ 0 ∀ t ∈ ( t 0 , t f ] , (6) where, using Eq. 2 and Eq. 4, the dynamics of M ( t ) and r ( t ) are given by the following t wo coupled jump SDEs: dr ( t ) = − r ( t ) dN ( t ) + dM ( t ) dλ ( t ) = [ λ 0 0 ( t ) + w λ 0 ( t ) − w λ ( t )] dt + α dM ( t ) , with initial conditions r ( t 0 ) = r 0 and λ ( t 0 ) = λ 0 , and the dynamics of N ( t ) are given by the intensit y u ( t ) that we aim to optimize. The ab o ve sto c hastic optimal con trol problem differs from previous literature in tw o k ey technical asp ects, which require careful reasoning: (i) The control signal u ( t ) is a conditional intensit y , whic h controls the dynamics of the counting pro cess N ( t ) ( i.e. , n um b er of stories broadcasted b y user i b y time t ). As a consequence, the problem form ulation needs to account for another la yer of sto chasticit y . Previous w ork assumes the control signal to b e a time-v arying real vector. (ii) The dynamics of the counting pro cess M ( t ) ( i.e. , num b er of stories broadcasted by other users that j follo ws by time t ) are doubly sto c hastic Marko v. Previous work has typically considered memoryless P oisson pro cesses and, only very recently , inhomogeneous Poisson [27]. 3 The final p enalt y function φ ( r ( t f )) is necessary to derive the optimal intensit y u ∗ ( t ) in Section 4. Ho wev er, the actual optimal intensit y u ∗ ( t ) do es not dep end on the particular choice of terminal condition. 5 Algorithm 1: RedQueen for fixed s , q and one follow er. Input: Parameters q and s Output: Returns time for the next p ost t ← ∞ ; τ ← other sN extP ost ( ) while τ < t do ∆ ∼ exp( p s/q ) t ← min( t, τ + ∆) τ ← other sN extP ost ( ) end return t Next, we will define a nov el optimal cost-to-go function that accoun ts for the ab o ve unique aspects of our problem, showing that the Bellman’s principle of optimality still follows, and finally find the optimal solution using the corresp onding Hamilton-Jacobi-Bellman (HJB) equation. Definition 2 The optimal c ost-to-go J ( r ( t ) , λ ( t ) , t ) is define d as the minimum of the exp e cte d value of the c ost of going fr om state r ( t ) with intensity λ ( t ) at time t to final state at time t f , i.e. , min u ( t,t f ] E ( N ,M )( t,t f ]  φ ( r ( t f )) + Z t f t ` ( r ( τ ) , u ( τ )) dτ  , (7) wher e the exp e ctation is taken over al l tr aje ctories of the c ontr ol and noise jump pr o c ess, N and M , in the ( t, t f ] interval, given the initial values of r ( t ) , λ ( t ) and u ( t ) . T o find the optimal con trol u ( t, t f ] and cost-to-go J , w e break the problem in to smaller subproblems, using the Bellman’s principle of optimality , which the ab o ve definition allows (prov en in App endix): Lemma 3 (Bellman’s Principle of Optimality) The optimal c ost satisfies the fol lowing r e cursive e qua- tion: J ( r ( t ) , λ ( t ) , t ) = min u ( t,t + dt ] E [ J ( r ( t + dt ) , λ ( t + dt ) , t + dt )] + ` ( r ( t ) , u ( t )) dt. (8) where the exp ectation is taken o ver all tra jectories of the control and noise jump pro cesses, N and M , in ( t, t + dt ] . Then, we use the Bellman’s principle of optimality to derive a partial differen tial equation on J , often called the Hamilton-Jacobi-Bellman (HJB) equation [ 13 ]. T o do so, we first assume J is contin uous and then rewrite Eq. 8 as J ( r ( t ) , λ ( t ) , t ) = min u ( t,t + dt ] E [ J ( r ( t ) , λ ( t ) , t ) + dJ ( r ( t ) , λ ( t ) , t )] + ` ( r ( t ) , u ( t )) dt 0 = min u ( t,t + dt ] E [ dJ ( r ( t ) , λ ( t ) , t )] + ` ( r ( t ) , u ( t )) dt. (9) Then, we differentiate J with resp ect to time t , r ( t ) and λ ( t ) using Lemma 6 (refer to App endix). Sp ecifically , consider x ( t ) = r ( t ) , y ( t ) = λ ( t ) and F = J in the ab ov e men tioned lemma, then, dJ ( r ( t ) , λ ( t ) , t ) = J t ( r ( t ) , λ ( t ) , t ) dt + [ λ 0 0 ( t ) + w λ 0 ( t ) − w λ ( t )] J λ ( r ( t ) , λ ( t ) , t ) dt + [ J (0 , λ ( t ) , t ) − J ( r ( t ) , λ ( t ) , t )] dN ( t ) + [ J ( r ( t ) + 1 , λ ( t ) + α, t ) − J ( r ( t ) , λ ( t ) , t )] dM ( t ) Next, if we plug in the ab o ve equation in Eq. 9, it follows that 0 = min u ( t,t + dt ] n J t ( r ( t ) , λ ( t ) , t ) dt + [ λ 0 0 ( t ) + w λ 0 ( t ) − w λ ( t )] J λ ( r ( t ) , λ ( t ) , t ) dt + [ J (0 , λ ( t ) , t ) − J ( r ( t ) , λ ( t ) , t )] E [ dN ( t )] + [ J ( r ( t ) + 1 , λ ( t ) + α, t ) − J ( r ( t ) , λ ( t ) , t )] E [ dM ( t )] + ` ( r ( t ) , u ( t )) dt o . (10) 6 Algorithm 2: Optimal p osting times with an oracle. Input: Initial state r 0 , interv al widths w 1 , . . . , w m +1 , parameter q and significance s ( t ) = s Output: Overall cost J ( r 0 , 0) , optimal control u ∗ 0 , . . . , u ∗ m for r ← r 0 + m to 0 do J ( r, m + 1) ← 1 2 r 2 end for k ← m to 0 do for r ← r 0 + k − 1 to 0 do J ( r, k ) = min { 1 2 q + J (0 , k + 1) , 1 2 sw k +1 ( r + 1) 2 + J ( r + 1 , k + 1) } end end for k ← 0 to m do if 1 2 q + J (0 , k + 1) < 1 2 sw k +1 ( r k + 1) 2 + J ( r k + 1 , k + 1) then u ∗ k ← 1; r k +1 ← 0 else u ∗ k ← 0; r k +1 ← r k + 1 end end return J ( r 0 , 0) , u ∗ 0 , . . . , u ∗ m No w, using E [ dN ( t )] = u ( t ) dt and E [ dM ( t )] = λ ( t ) dt , and rearranging terms, the HJB equation follows: 0 = J t ( r ( t ) , λ ( t ) , t ) + [ λ 0 0 ( t ) + w λ 0 ( t ) − w λ ( t )] J λ ( r ( t ) , λ ( t ) , t ) + [ J ( r ( t ) + 1 , λ ( t ) + α, t ) − J ( r ( t ) , λ ( t ) , t )] λ ( t ) + min u ( t,t + dt ] ` ( r ( t ) , u ( t )) + [ J (0 , λ ( t ) , t ) − J ( r ( t ) , λ ( t ) , t )] u ( t ) . (11) T o b e able to con tinue further, w e need to define the loss ` and the p enalty φ . F ollowing the literature on sto c hastic optimal control [ 13 ], we consider the following quadratic forms, which will turn out to b e a tractable c hoice 4 : φ ( r ( t f )) = 1 2 r 2 ( t f ) and ` ( r ( t ) , u ( t )) = 1 2 s ( t ) r 2 ( t ) + 1 2 q u 2 ( t ) , where s ( t ) is a time significance function s ( t ) ≥ 0 , which fav ors some perio ds of times ( e.g. , times in which the follo wer is online 5 ), and q is a given parameter, whic h trade-offs visibilit y and num b er of broadcasted p osts. Under these definitions, we take the deriv ative with respect to u ( t ) of Eq. 11 and uncov er the relationship b et w een the optimal intensit y and the optimal cost: u ∗ ( t ) = q − 1 [ J ( r ( t ) , λ ( t ) , t ) − J (0 , λ ( t ) , t )] . (12) Finally , we substitute the ab o ve expression in Eq. 11 and find that the optimal cost J needs to satisfy the follo wing nonlinear differential equation: 0 = J t ( r ( t ) , λ ( t ) , t ) + [ λ 0 0 ( t ) + w λ 0 ( t ) − w λ ( t )] J λ ( r ( t ) , λ ( t ) , t ) + [ J ( r ( t ) + 1 , λ ( t ) + α, t ) − J ( r ( t ) , λ ( t ) , t )] λ ( t ) + 1 2 s ( t ) r 2 ( t ) − 1 2 q − 1 [ J ( r ( t ) , λ ( t ) , t ) − J (0 , λ ( t ) , t )] 2 (13) with J ( r ( t f ) , λ ( t f ) , t f ) = φ ( r ( t f )) as the terminal condition. The following lemma pro vides us with a solution to the ab o ve equation (prov en in App endix): 4 Considering other losses with a sp ecific semantic meaning ( e.g. , I ( r ( t ) ≤ k ) ) is a challenging direction for future work. 5 Such information may b e hidden but one can use the follow ers’ posting activity or geographic lo cation as a proxy [17]. 7 Lemma 4 A ny solution to the nonline ar differ ential e quation given by Eq. 13 c an b e appr oximate d as closely as desir e d by J ( r ( t ) , λ ( t ) , t ) = f ( t ) + p s ( t ) /q r ( t ) + m X j =1 g j ( t ) λ j ( t ) , wher e f ( t ) and g j ( t ) ar e time-varying functions, and m c ontr ols for the appr oximation guar ante e. Giv en the ab ov e Lemma and Eq. 12, the optimal in tensity is readily given by following theorem: Theorem 5 The optimal intensity for the when-to-p ost pr oblem define d by Eq. 6 with quadr atic loss and p enalty function is given by u ∗ ( t ) = p s ( t ) /q r ( t ) . The optimal intensit y only dep ends on the p osition of the most recent p ost by user i in her follow er’s feed and th us allows for a very efficient pro cedure to sample p osting times, whic h exploits the sup erposition theorem [ 19 ]. The k ey idea is as follows: at any given time t , we can view the pro cess defined by the optimal in tensity as a sup erposition of r ( t ) inhomogeneous p oisson pro cesses with in tensity p s ( t ) /q r ( t ) which starts at jumps of the rank r ( t ) , and find the next sample by computing the minimum across all samples from these pro cesses. Algorithm 1 summarizes our (sampling) metho d, whic h we name RedQueen [ 4 ]. Within the algorithm, other sN extP ost ( ) returns the time of the next even t b y other broadcasters in the follow ers’ feeds, once the even ts happ ens. In practice, w e only need to know if the even t happ ens b efore we p ost. Remark ably , it only needs to sample M ( t f ) times from a (exp onen tial) distribution (if significance is constan t) and requires O (1) space. Optimizing for multiple follow ers. Giv en a broadcaster i with N i ( t ) = N ( t ) and µ i ( t ) = u ( t ) and her follo wers N ( i ) with M \ i ( t ) = M ( t ) and γ \ i ( t ) = λ ( t ) , the dynamics of M ( t ) and r ( t ) , whic h we need to solv e Eq. 5, are given by: d r ( t ) = − r ( t ) dN ( t ) + d M ( t ) d λ ( t ) = [ λ 0 0 ( t ) + w  λ 0 ( t ) − w  λ ( t )] dt + α  d M ( t ) , where  is the elemen t-wise pro duct and α = [ α 1 , · · · , α n ] T and w = [ w 1 , · · · , w n ] T are the parameters defining eac h of the follow ers’ feed dynamics, and n = |N ( i ) | is the num b er of follow ers. Consider the following quadratic forms for the loss ` and the p enalt y φ : φ ( r ( t f )) = n X i =1 1 2 r 2 i ( t f ) ` ( r ( t ) , u ( t ) , t ) = n X i =1 1 2 s i ( t ) r 2 i ( t ) + 1 2 q u 2 ( t ) . where s i ( t ) is the time significance function for follow er i , as defined ab o v e, and q is a given parameter. Then, pro ceeding similarly as in the case of one follow er, we can show that: u ∗ ( t ) = n X i =1 p s i ( t ) /q r i ( t ) , (14) whic h only dep ends on the p osition of the most recent p ost by user i in her follow ers’ feeds. Finally , we can readily adapt RedQueen (Algorithm 1) to efficiently sample the p osting times using the ab o ve intensit y – it only needs to sample | ∪ j ∈N ( i ) F j \ i ( t f ) | v alues and requires O ( |N ( i ) | ) space. Optimizing with an oracle. In this section, w e consider a broadcaster i with N i ( t ) = N ( t ) and µ i ( t ) = u ( t ) , only one of her follow ers j with M j \ i ( t ) = M ( t ) , and a constant significance s ( t ) = s . The deriv ation can b e easily adapted to the case of multiple follow ers and time-v arying significance. 8 R E D Q U E E N Oracle Karimi 10 1 10 2 10 3 Budget 10 − 1 10 0 10 1 10 2 (a) Position ov er time 10 1 10 2 10 3 Budget 0 50 100 (b) Time at the top Figure 2: Optimizing for one follow er. Performance of RedQueen in comparison with the oracle and the metho d by Karimi et al. [ 17 ] against num ber of broadcasted even ts. The feeds coun ting pro cesses M ( t ) due to other broadcasters are Hawk es pro cesses with λ 0 = 10 , α = 1 and w = 10 . In all cases, the time horizon t f − t 0 is c hosen such that the num b er of stories p osted by other broadcasters is ∼ 1000 . Error bars are to o small to b e seen. Supp ose there is an (idealized) oracle that rev eals M ( t ) from t 0 to t f , i.e. , the history F j \ i ( t f ) = F ( t f ) is giv en, and M ( t f ) = |F ( t f ) | = m . Then, we can rewrite Eq. 5 as minimize u ( t 0 ,t f ] E N ( t 0 ,t f ]  φ ( r ( t f )) + Z t f t 0 ` ( r ( τ ) , u ( τ )) dτ  sub ject to u ( t ) ≥ 0 ∀ t ∈ ( t 0 , t f ] , where the expectation is only taken ov er all possible realizations of the coun ting pro cess N ( t 0 , t f ] since M ( t 0 , t f ] is revealed by the oracle and thus deterministic. Similar to the previous sections, assume the loss ` and p enalt y φ are quadratic. It is easy to realize that the b est times for user i to p ost will alwa ys coincide with one of the times in F ( t f ) . More sp ecifically , given a p osting time τ i ∈ ( t k , t k +1 ) , where t k , t k +1 ∈ F ( t f ) , one can reduce the cost by (1 / 2) q ( τ i − t k ) r 2 ( t k ) by c ho osing instead to p ost at t k . As a consequence, we can discretize the dynamics of r ( t ) in times F ( t f ) , and write r k +1 = r k + 1 − ( r k + 1) u k , where r k = r ( t − k ) , u k = u ( t + k ) ∈ { 0 , 1 } , t k ∈ F ( t f ) . W e can easily see that r k is b ounded by 0 ≤ r k < r 0 + m . Similarly , we can derive the optimal cost-to-go in discrete-time as: J ( r k , k ) = min u k ,...,u m 1 2 r 2 m +1 + m X i = k 1 2 q w i +1 r 2 i +1 + 1 2 s u 2 i , where w i = t i − t i − 1 . Next, we can break the minimization and use Bellman’s principle of optimality , J ( r k , k ) = min u k 1 2 q w k +1 r 2 k +1 + 1 2 s u 2 k + J ( r k +1 , k + 1) , and, since u k ∈ { 0 , 1 } , the ab o ve recursive equation can b e written as J ( r k , k ) = min  1 2 s + J (0 , k + 1) , 1 2 q w k +1 ( r k + 1) 2 + J ( r k + 1 , k + 1)  . Finally , we can find the optimal control u ∗ k , k = 0 , . . . , m and cost J ( r 0 , 0) b y backtrac king from the terminal condition J ( r m +1 , m + 1) = r 2 m +1 / 2 to the initial state r 0 , as summarized in Algorithm 2, whic h can b e adapted to multiple follow ers. Note that, in this case, the optimal strategy is not sto c hastic and consists of a set of optimal p osting times, as one could hav e guessed. How ever, for m ultiple follow ers, the complexit y of the algorithm is O ( m 2 ) , where m = | ∪ j ∈N ( i ) F j \ i ( t f ) | . 9 R E D Q U E E N Karimi 0 2 4 6 8 10 12 14 500 1000 1500 2000 0 2 4 6 8 10 12 14 9 10 11 12 13 (a) P osition ov er time (b) Time at the top Figure 3: Optimizing for m ultiple follow ers. P erformance of RedQueen in comparison with the metho d by Karimi et al. [ 17 ] against num ber of follo wers. The feeds counting pro cesses M ( t ) due to other broadcasters follo w piecewise constan t intensities, where the intensit y of each follow er remains constant within each piece, it v aries as a half-sinusoid across pieces and it starts with a random initial phase. The p erformance of b oth metho ds stays constant up on addition of more follow ers. 5 Exp erimen ts 5.1 Exp erimen ts on synthetic data Exp erimen tal setup. W e ev aluate the p erformance via tw o quality measures: p osition ov er time, R T 0 r ( t ) dt , and time at the top, R T 0 I ( r ( t ) < 1) dt and compare the p erformance of RedQueen against the oracle, describ ed in Section 4, and the metho d b y Karimi et al. [ 17 ], which, to the b est of our knowledge, is the state of the art. Unless otherwise stated, we set the significance s i ( t ) = 1 , ∀ t, i and use the parameter q to control the n umber of p osts by RedQueen 6 . Optimizing for one follo wer. W e first exp erimen t with one broadcaster and one follow er against an increasing num b er of even ts (or budget). W e generate the counting pro cesses M ( t ) due to other broadcasters using Ha wkes pro cesses, which are particular instances of the general functional form given by Eq. 1. W e p erform 10 indep enden t simulation runs and compute the av erage and standard error (or s tandard deviation) of the quality measures. Fig. 2 summarizes the results, whic h sho w that our metho d: (i) consisten tly outp erforms the metho d by Karimi et al. by large margins; (ii) achiev es at most 3 × higher p osition ov er time than the oracle as long as the budget is < 30 % of the p osted even ts by all other broadcasters; and, (iii) ac hieves > 40 % of the v alue of time at the top that the oracle achiev es. Optimizing for multiple follo wers. Next, we exp erimen t with one broadcaster and multiple follow ers. In this case, we generate the counting pro cesses M ( t ) due to other broadcasters using piece-constant intensit y functions. More sp ecifically , we simulate the feeds of eac h follow er for 1 day , using 24 1 -hour long segmen ts, where the rate of p osts remains constant p er follo wer in each segment and the rate itself v aries as a half-sinusoid ( i.e. , from sin 0 to sin π ), with each follow er starting with a random initial phase. This exp erimen tal setup repro duces volume changes throughout the day across follow ers’ feeds in different time-zones and closely resem bles the settings in previous work [ 17 ]. The total num b er of p osts by the RedQueen broadcaster is k ept nearly constant and is used as the budget for the other baselines. A dditionally , for Karimi’s metho d, we pro vide as input the true empirical rate of t weets p er hour for each user. Here, we do not compare with the oracle since, due to its quadratic complexity , it do es not scale. Figure 3 summarizes the results. In terms of position ov er time, RedQueen outp erforms Karimi’s metho d b y a factor of 2 . In terms of time at the top, RedQueen ac hieves ∼ 18% low er v alues than Karimi’s metho d for 1 - 4 follow ers but ∼ 10% higher v alues for > 5 follow ers. A p oten tial reason for Karimi’s method to p erforms b est in terms of time at the top for a low num ber of follow ers and piecewise constan t intensities is that, while the num b er of follow ers is lo w, there are segments which are clearly fa vorable and thus Karimi’s metho d 6 The exp ected number of p osts by RedQueen are a decreasing function of q . Hence, we can use binary search to guess q and then use averaging ov er multiple simulation runs to estimate the number of p osts made. 10 R E D Q U E E N Karimi 0 . 0 0 . 5 1 . 0 R E D Q U E E N Karimi 1 2 3 (a) P osition ov er time (b) Time at the top Figure 4: Performance of RedQueen and the metho d by Karimi et al. [ 17 ] for 2000 T witter users, pick ed at random. The solid horizontal line (square) shows the median (mean) quality measure, normalized with resp ect to the v alue achiev ed by the users’ actual true p osts, and the b o x limits corresp ond to the 25%-75% p ercen tiles. concen trates posts on those, ho wev er, as the num b er of follow ers increases, there are no clear fav orable segmen ts and thus adv ance planning do es not giv e Karimi’s metho d an y adv antage. On the other hand, RedQueen , due to its online nature, is able to adapt to transient v ariations in the feeds. 5.2 Exp erimen ts on real data Dataset description and exp erimental setup. W e use data gathered from T witter as rep orted in previous w ork [ 5 ], whic h comprises profiles of 52 million users, 1 . 9 billion directed follo w links among these users, and 1 . 7 billion public tw eets p osted by the collected users. The follow link information is based on a snapshot taken at the time of data collection, in September 2009. Here, we fo cus on the tw eets published during a tw o month p erio d, from July 1, 2009 to September 1, 2009, in order to b e able to consider the so cial graph to b e appro ximately static, and sample 2000 users uniformly at random as broadcasters and record all the tw eets they p osted. Then, for each of these broadcasters, we track do wn their follow ers and record all the (re)t weets they p osted as well as reconstruct their timelines b y collecting all the (re)t weets published by the p eople they follo w. W e assign equal significance to eac h follow er but filter out those who follow more than 500 p eople since, otherwise, they w ould dominate the optimal strategy . Finally , w e tune q suc h that the total n umber of tw eets p osted by our metho d is equal to the num b er of tw eets the broadcasters tw eeted during the t wo month p erio d (with a tolerance of 10% ). Solution qualit y . W e only compare the p erformance of our metho d against the method b y Karimi et al. [ 17 ] since the oracle do es not scale to the size of real data. Moreo v er, for the metho d by Karimi et al., w e divide the tw o month p eriod into ten segments of appro ximately one week to fit the piecewise constant in tensities of the follo wers’ timelines, which the metho d requires. Fig. 4 summarizes the results by means of b o x plots, where p osition ov er time and time at the top are normalized with resp ect to the v alue achiev ed b y the broadcasters’ actual true p osts during the tw o month p erio d. That means, if y = 1 , the optimized in tensity achiev es the same p osition ov er time or time at the top as the broadcaster’s true p osts. In terms of p osition ov er time and time at the top, RedQueen consistently outp erforms comp eting metho ds b y large margins and achiev es 0 . 28 × lo wer av erage p osition and 3 . 5 × higher time at the top, in av erage, than the broadcasters’ true p osts – in fact, it ac hieves lo wer position ov er time (higher time at the top) for 100 % ( 99 . 1 %) of the users. Time significance. W e lo ok at the actual broadcasting strategies for one real user and inv estigate the effect of a time v arying significance. W e define s i ( t ) to b e the probability that follow er i is online on that weekda y , estimated empirically using the (re)t weets the follo wer p osted as in Karimi et al. [ 17 ]. Fig. 5 compares the p osition ov er time for the most recent tw eet p osted by a real user against the most recent one p oste d b y a simulation run of RedQueen with and without time v arying significance. W e can see that without significance information, RedQueen p osts at nearly an even pace. How ev er, when we supply empirically estimated significance, RedQueen a voids tw eeting at times the follow ers are unlikely to b e active, i.e. , the w eekends, denoted by the shaded areas in panel (c) of Fig. 5. Due to this, the av erage p osition (maximum 11 01/06 15/06 31/06 0 1500 3000 N ( t ) ¯ r ( t ) 01/06 15/06 31/06 0 1500 3000 N ( t ) ¯ r ( t ) (a) T rue p osts (b) RedQueen (without significance) 1 T R T 0 ¯ r ( t ) dt = 698 . 04 1 T R T 0 ¯ r ( t ) dt = 389 . 45 01/06 15/06 31/06 0 1500 3000 M T W Th F Sa Su 0 100 200 300 400 (c) RedQueen (with significance) (d) F ollo wers’ (re)tw eets p er weekda y 1 T R T 0 ¯ r ( t ) dt = 425 . 25 Figure 5: A broadcaster c hosen from real data. Panels compares the p osition ov er time ¯ r ( t ) = P N i =0 r ( t ) / N (in green; low er is b etter) for th e most recent tw eet p osted by a real user against the most recen t one p oste d b y a simulation run of RedQueen without and with significance. Here, the orange staircases represent the coun ts N ( t ) of the tw eets p osted by the real user and RedQueen ov er time. The shaded area in panel (c) highligh ts week ends. W e can see that RedQueen av oided tw eeting on week ends, when the follow ers are less lik ely to b e active/logged-in, as seen in panel (d). p osition) falls from 389 . 45 ( 1085 . 17 ) to 425 . 25 ( 1431 . 0 ), but is still low er than 698 . 04 ( 2597 . 9 ) obtained by the user’s original p osting schedule. 6 Conclusions In this pap er, we approac hed the when-to-p ost problem from the p erspective of sto c hastic optimal control and sho wed that the optimal broadcasting strategy is surprisingly simple – it is given by the p osition of her most recen t p ost on each of her follow er’s feed. Suc h a strategy can b e implemented using a simple and efficient on-line algorithm. W e exp erimented with synthetic and real-world data gathered from T witter and show ed that our algorithm consistently makes a user’s p osts more visible ov er time and it significantly outp erforms the state of the art. Our work also op ens many ven ues for future work. F or example, in this work, we considered so cial net works that sort stories in the users’ feeds in inv erse chronological order ( e.g. , T witter, W eibo). Extending our metho dology to so cial netw orks that sort stories algorithmically ( e.g. , F acebo ok) is a natural next step. Curren tly , RedQueen optimizes a quadratic loss on the p osition of a broadcaster’s most recen t p ost on her follo wers’ feeds ov er time. Ho wev er, it would b e useful to derive optimal broadcasting intensities for other losses, e.g. , time at the top. Moreov er, we assume that only one broadcaster is using RedQueen . A very in teresting follo w-up w ould b e augmen ting our framework to consider multiple broadcasters under co op erativ e, comp etitiv e and adversarial environmen ts. Finally , the nov el technical asp ects of our problem formulation, e.g. , optimal control of jump SDEs with double sto c hastic temp oral p oin t pro cesses, can b e applied to other con trol problems in so cial and information netw orks suc h as activity shaping [10] and opinion con trol [27]. 12 References [1] O. Aalen, O. Borgan, and H. K. Gjessing. Survival and event history analysis: a pr o c ess p oint of view . Springer, 2008. [2] L. Backstrom, E. Bakshy , J. M. Kleinberg, T. M. Lento, and I. Rosenn. Cen ter of attention: How faceb ook users allo cate attention across friends. ICWSM , 2011. [3] R. Blumenthal. An extended marko v prop ert y . T r ansactions of the Americ an Mathematic al So ciety , 85(1):52–72, 1957. [4] L. Carroll. Thr ough the lo oking glass: A nd what Alic e found ther e . Rand, McNally , 1917. [5] M. Cha, H. Haddadi, F. Beneven uto, and P . K. Gummadi. Measuring User Influence in T witter: The Million F ollo wer F allacy. ICWSM , 2010. [6] W. Chen, C. W ang, and Y. W ang. Scalable influence maximization for prev alen t viral marketing in large-scale so cial netw orks. In Pr o c e e dings of the 16th ACM SIGKDD international c onfer enc e on Know le dge disc overy and data mining , 2010. [7] J. Cheng, L. Adamic, P . Dow, J. Klein b erg, and J. Lesko vec. Can cascades b e predicted? In Pr o c e e dings of the 23r d international c onfer enc e on W orld wide web , 2014. [8] T. Chenhao, L. Lee, and B. Pang. The effect of wording on message propagation: T opic- and author- con trolled natural exp eriments on twitter. In Pr o c e e dings of the 52nd Annual Me eting of the Asso ciation for Computational Linguistics , 2014. [9] N. Du, L. Song, M. Gomez-Ro driguez, and H. Zha. Scalable influence estimation in contin uous-time diffusion net works. In A dvanc es in Neur al Information Pr o c essing Systems , 2013. [10] M. F ara jtabar, N. Du, M. Gomez-Ro driguez, I. V alera, H. Zha, and L. Song. Shaping so cial activity by incen tivizing users. In NIPS , 2014. [11] M. F ara jtabar, Y. W ang, M. Gomez-Ro driguez, S. Li, H. Zha, and L. Song. Coevolv e: A joint p oin t pro cess mo del for information diffusion and netw ork co-evolution. In A dvanc es in Neur al Information Pr o c essing Systems , 2015. [12] M. Gomez-Ro driguez, K. Gummadi, and B. Schölk opf. Quantifying information ov erload in so cial media and its impact on so cial contagions. In 8th International AAAI Confer enc e on W eblo gs and So cial Me dia , 2014. [13] F. B. Hanson. Applie d sto chastic pr o c esses and c ontr ol for Jump-diffusions: mo deling, analysis, and c omputation . SIAM, 2007. [14] A. G. Hawk es. Sp ectra of some self-exciting and m utually exciting point pro cesses. Biometrika , 58(1):83–90, 1971. [15] N. Ho das and K. Lerman. Ho w visibility and divided attention constrain so cial contagion. So cialCom , 2012. [16] J.-H. Kang and K. Lerman. Vip: Incorp orating human cognitive biases in a probabilistic mo del of ret weeting. In So cial Computing, Behavior al-Cultur al Mo deling, and Pr e diction , pages 101–110. Springer, 2015. [17] M. R. Karimi, E. T a v ak oli, M. F ara jtabar, L. Song, and M. Gomez-Ro driguez. Smart broadcasting: Do y ou wan t to b e seen? In Pr o c e e dings of the 22nd ACM SIGKDD International Confer enc e on Know le dge Disc overy and Data Mining , 2016. 13 [18] D. Kemp e, J. Klein b erg, and É. T ardos. Maximizing the spread of influence through a so cial netw ork. In Pr o c e e dings of the ninth ACM SIGKDD international c onfer enc e on Know le dge disc overy and data mining , 2003. [19] J. F. C. Kingman. Poisson pr o c esses . Oxford universit y press, 1992. [20] H. Lakk ara ju, J. J. McAuley , and J. Lesko v ec. What’s in a name? understanding the interpla y b et ween titles, con tent, and communities in so cial media. In ICWSM , 2013. [21] K. Lerman and T. Hogg. Lev eraging p osition bias to improv e p eer recommendation. PloS one , 9(6), 2014. [22] G. Miritello, R. Lara, M. Cebrian, and E. Moro. Limited communication capacity unv eils strategies for h uman interaction. Scientific r ep orts , 3, 2013. [23] N. Na v aroli and P . Sm yth. Modeling resp onse time in digital h uman comm unication. In Ninth International AAAI Confer enc e on W eb and So cial Me dia , 2015. [24] M. Richardson and P . Domingos. Mining kno wledge-sharing sites for viral marketing. In Pr o c e e dings of the eighth ACM SIGKDD international c onfer enc e on Know le dge disc overy and data mining , 2002. [25] N. Spaso jevic, Z. Li, A. Rao, and P . Bhattacharyy a. When-to-p ost on so cial netw orks. In Pr o c e e dings of the 21th ACM SIGKDD International Confer enc e on Know le dge Disc overy and Data Mining , pages 2127–2136. A CM, 2015. [26] M. H. Stone. The generalized weierstrass approximation theorem. Mathematics Magazine , 21(5):237–254, 1948. [27] Y. W ang, E. Theo dorou, A. V erma, and L. Song. Steering opinion dynamics in information diffusion net works. arXiv pr eprint arXiv:1603.09021 , 2016. [28] Q. Zhao, M. Erdogdu, H. He, A. Ra jaraman, and J. Lesko v ec. Seismic: A self-exciting p oin t pro cess mo del for predicting tw eet p opularit y . In 21th A CM SIGKDD International Confer enc e on Know le dge Disc overy and Data Mining , 2015. 14 Pro of of Prop osition 1 Using the left contin uit y of p oisson pro cesses, we hav e that: dλ ∗ ( t ) = λ 0 0 ( t ) dt + α d  Z t 0 g ( t − s ) dN ( s )  . Then, using Ito’s calculus [13], we can rewrite the differential in the second term as Z t + dt 0 g ( t + dt − s ) dN ( s ) − Z t 0 g ( t − s ) dN ( s ) = Z t + dt 0 ( g ( t − s ) + g 0 ( t − s ) dt ) dN ( s ) − Z t 0 g ( t − s ) dN ( s ) = Z t + dt t g ( t − s ) dN ( s ) + dt Z t + dt 0 g 0 ( t − s ) dN ( s ) = g (0) dN ( t ) − w dt Z t + dt 0 g ( t − s ) dN ( s ) = dN ( t ) − w dt Z t 0 g ( t − s ) dN ( s ) = dN ( t ) + w α [ λ 0 ( t ) − λ ∗ ( t )] dt. This completes the pro of. Pro of of Lemma 3 min u ( t,t f ] E ( N ,M )( t,t f ]  φ ( r ( t f )) + Z t f t ` ( r ( τ ) , u ( τ )) dτ  = min u ( t,t f ] E ( N ,M )( t,t f ]  φ ( r ( t f )) + Z t + dt t ` ( r ( τ ) , u ( τ )) dτ + Z t f t + dt ` ( r ( τ ) , u ( τ )) dτ  = min u ( t,t f ] E ( N ,M )( t,t + dt ]  E ( N ,M )( t + dt,t f ] h φ ( r ( t f )) + ` ( t, r, u ) dt + Z t f t + dt ` ( r ( τ ) , u ( τ )) dτ i  = min u ( t,t + dt ] min u ( t + dt,t f ] E ( N ,M )( t,t + dt ]  ` ( r ( t ) , λ ( t ) , t ) dt + E ( N ,M )( t + dt,t f ] h φ ( r ( t f )) + Z t f t + dt ` ( r ( τ ) , u ( τ )) dτ i  = min u ( t,t + dt ] E ( N ,M )( t,t + dt ] [ J ( λ ( t + dt ) , r ( t + dt ) , t + dt )] + ` ( r ( t ) , u ( t )) dt. Pro of of Lemma 4 A ccording to the Stone-W eierstrass theorem, an y contin uous function in a closed interv al can b e approximated as closely as desired by a p olynomial function [ 26 ]. So by assuming the con tinuit y of cost function we consider general form J ( r ( t ) , λ ( t ) , t ) = n X i =0 m X j =0 f ij ( t ) r i ( t ) λ j ( t ) , 15 where m and n are arbitrary large num b ers. Indeed in each time t w e approximate a t wo v ariate function of r ( t ) and λ ( t ) by a p olynomial where the co efficient are defined by the time v arying functions f ij ( t ) . If we substitute this function in to Eq. 13 and simplifying the expression we would hav e 0 = n X i =1 f 0 i 0 r i ( t ) + f i 0 ( r + 1) i λ − f i 0 r i λ + m X j =1 f 0 0 j λ j + j ( λ 0 0 + β λ 0 − β λ ) f 0 j λ j − 1 + f 0 j ( λ + α ) j λ − f 0 j λ j +1 + n X i =1 m X j =1 j ( λ 0 0 + β λ 0 − β λ ) f ij r i λ j − 1 + n X i =1 m X j =1 f ij ( r + 1) i ( λ + α ) j λ − f ij r i λ j +1 − 1 2 s − 1  n X i =1 f i 0 r i + n X i =1 m X j =1 f ij r i λ j  2 + 1 2 q r 2 + f 0 00 where for notational simplicit y we omitted the time argument of functions. T o find the unknown functions f ij ( t ) , we equate the co efficien t of different v ariables. If we consider the co efficien t of r 2 n , we hav e f n 0 ( t ) = 0 . W e can contin ue this argument for n − 1 , n − 2 , · · · , 2 to show that ∀ i ≥ 2; f i 0 ( t ) = 0 . Similar reasoning for co efficien ts of r 2 i λ 2 j sho ws that ∀ j, i ≥ 2; f ij ( t ) = 0 . Finally , the co efficient of r 2 is 1 / 2 q − 1 / 2 s − 1 f 2 10 ( t ) = 0 so f 10 ( t ) = ( sq ) 1 / 2 . If we rename f 0 j ( t ) to g j ( t ) and f 00 ( t ) to f ( t ) , then we hav e J ( r ( t ) , λ ( t ) , t ) = f ( t ) + ( sq ) 1 / 2 r ( t ) + m X j =1 g j ( t ) λ j ( t ) . W e can contin ue the previous metho d to find the remaining coefficients and completely define the cost-to-go function. If we equate the co efficien t of λ j to zero we would hav e a system of first o der differential equation whic h its j ’th row is g 0 j ( t ) + j ( α − β ) g j ( t ) + ( j + 1)  λ 0 0 ( t ) + β λ 0 ( t ) + j 2 α 2  g j +1 ( t ) + m − j X k =2  j + k k + 1  α k +1 g j + k ( t ) = 0 When λ 0 ( t ) = λ 0 , we can express this using matrix differential equation g 0 ( t ) = A g ( t ) . and its solution is g ( t ) = c 1 e ζ 1 t u 1 + c 2 e ζ 2 t u 2 + · · · + c n e ζ n t u n where ζ i and u i are eigen v alue and eigenv ector of matrix A and c i is a constant found using the terminal conditions. Since in triangular matrices diagonal en tries are eigenv alues, w e hav e g ( t ) = P m j =1 c i e j ( β − α ) u i . W e can approximate general time v arying λ 0 ( t ) using piecewise function and rep eat the ab o ve pro cedure for each piece. Lemma 6 Lemma 6 L et x ( t ) and y ( t ) b e two jump-diffusion pr o c esses define d by fol lowing jump SDEs: dx ( t ) = f ( x ( t ) , t ) dt + h ( x ( t ) , t ) dN ( t ) + g ( x ( t ) , t ) dM ( t ) dy ( t ) = m ( y ( t ) , t ) dt + n ( y ( t ) , t ) dM ( t ) , wher e N ( t ) , M ( t ) ar e indep endent jump pr o c esses. If function F ( x, y , t ) is onc e c ontinuously differ entiable in x , y and t , then, dF ( x ( t ) , y ( t ) , t ) = ( F t + f F x + mF y )( x ( t ) , y ( t ) , t ) dt +  F  x ( t ) + h ( x ( t ) , t ) , y ( t ) , t  − F ( x ( t ) , y ( t ) , t )  dN ( t ) +  F  x ( t ) + g ( x ( t ) , t ) , y ( t ) + n ( y ( t ) , t ) , t  − F ( x ( t ) , y ( t ) , t )  dM ( t ) . Pro of A ccording to the definition of differential, dF : = dF ( x ( t ) , y ( t ) , t ) = F ( x ( t + dt ) , y ( t + dt ) , t + dt ) − F ( x ( t ) , y ( t ) , t ) = F  x ( t ) + dx ( t ) , y ( t ) + dy ( t ) , t + dt  − F  x ( t ) , y ( t ) , t  16 where we used the complete notation for F to b e more clear. Using the zero-one law of p oin t pro cesses, w e can write dF = F  x + f dt + h, y + mdt, t + dt  dN ( t ) + F  x + f dt + g , y + mdt + n, t + dt  dM ( t ) + F  x + f dt, y + mdt, t + dt  (1 − dN ( t ))(1 − dM ( t )) − F  x, y , t  where for notational simplicity we drop arguments of all functions except F . Then, we can expand the first three terms in the right hand sides: F  x + f dt + h, y + mdt, t + dt  = F ( x + h, y , t ) + F x ( x + h, y , t ) f dt + F y ( x + h, y , t ) mdt + F t ( x + h, y , t ) dt + F  x + f dt + g , y + mdt + n, t + dt  = F ( x + g , y + n, t ) + F x ( x + g , y + n, t ) f dt + F y ( x + g , y + n, t ) mdt + F t ( x + g , y + n, t ) dt + F  x + f dt, y + mdt, t + dt  = F ( x, y , t ) + F x ( x, y , t ) f dt + F y ( x, y , t ) mdt + F t ( x, y , t ) dt, using that the bilinear differen tial form dt dN ( t ) = 0 [ 13 ] and dN ( t ) dM ( t ) = 0 by the zero-one jump law [ 19 ]. Finally dF = ( f F x + mF y + F t )( x, y , t ) dt +  F ( x + h, y , t ) − F ( x, y , t ))  dN ( t ) +  F ( x + g , y + n, t ) − F ( x, y, t )  dM ( t ) . 17