Time-Staging Enhancement of Hybrid System Falsification

Remke & T ran (Eds.): Symbolic-Numeric Methods for Reasoning about CPS and IoT (SNR’21) EPTCS 361, 2022, pp. 25–43, doi:10.4204/EPTCS.361.5 T ime-Staging Enhancement of Hybrid System F alsification * Gidon Ernst LMU Munich, Germany gidon.ernst@lmu.de Ichiro Hasuo National Institute of Informatics, T okyo, Japan hasuo@nii.ac.jp Zhenya Zhang Kyushu Uni versity , Japan zhang.zhenya.623@m.kyushu-u.ac.jp Sean Sedwards Univ ersity of W aterloo, W aterloo, Canada sean.sedwards@uwaterloo.ca Optimization-based falsification employs stochastic optimization algorithms to search for error input of hybrid systems. In this paper we introduce a simple idea to enhance falsification, namely time staging , that allo ws the time-causal structure of time-dependent signals to be exploited by the op- timizers. T ime staging consists of running a falsification solver multiple times, from one interval to another , incrementally constructing an input signal candidate. Our experiments show that time staging can dramatically increase performance in some realistic examples. W e also present theoreti- cal results that suggest the kinds of models and specifications for which time staging is likely to be effecti ve. 1 Intr oduction Hybrid Systems Quality assurance of cyber -physical systems (CPS) has been recognized as an impor - tant challenge, where many CPS are hybrid systems that combine the discrete dynamics of computers and the continuous dynamics of physical components. Unfortunately , analysis of hybrid systems poses unique challenges, such as the limited applicability of formal verification . In formal verification one aims to gi ve a mathematical proof for a system’ s correctness. This is much harder for hybrid systems than for computer software/hardware, where the presence of continuous dynamics makes many problems more complex or e ven undecidable (e.g. reachability in h ybrid automata). input true false Boolean semantics input more robustly true less so quantitative robust semantics ?? This way to ! climb down :) Figure 1: From Boolean to robust semantics Optimization-Based F alsification Be- cause of these difficulties, an increas- ing number of researchers are turning to optimization-based falsification as a quality assurance measure. It is a testing method rather than that of formal verifi- cation; the problem is formalized as fol- lo ws. • Given: a model M (that takes an input signal u and yields an output signal M ( u ) ), and a specification ϕ (a temporal formula) • Answer: err or input , that is, an input signal u such that the corresponding output M ( u ) violates ϕ u / / M M ( u ) 6| = ϕ ? / / * Presented at SNR’2018. G. Ernst and Z. Zhang were then at the National Institute of Informatics. 26 T ime-Staging Enhancement of Hybrid System Falsification In the optimization-based falsification approach, the above falsification problem is turned into an opti- mization problem. This is possible thanks to r obust semantics of temporal formulas [14]. Instead of the Boolean satisfaction relation v | = ϕ , robust semantics assigns a quantity J v , ϕ K ∈ R ∪ { ∞ , − ∞ } that tells us, not only whether ϕ is true or not (by the sign), b ut also how r ob ustly the formula is true or false. This allo ws one to emplo y hill-climbing optimization (see Fig. 1): we iterati vely generate input signals, in the direction of decreasing robustness, hoping that e ventually we hit negati ve rob ustness. Optimization-based falsification is a subclass of sear ch-based testing : it adapti vely chooses test cases (input signals u ) based on previous observations. One can use stochastic algorithms for optimization (such as simulated annealing), which turn out to be much more scalable than many model checking algorithms that rely on exhaustiv e search. Note also that the system model M can be black box: it is enough to kno w the correspondence between input u and output M ( u ) . An error input u is concrete e vidence for the system’ s need for improvement, and thus has great appeal to practitioners. The approach of optimization-based falsification was initiated in [14] and has been activ ely pursued e ver since [3, 5, 7, 8, 10, 11, 20]. There are now mature tools, such as Breach [8] and S-T aliro [5], which work with industry-standard Simulink models. Contribution W e introduce a simple idea of time staging for enhancement of optimization-based fal- sification. T ime staging consists of running a falsification solver repeatedly , from one input segment to another , incrementally constructing an input signal. In general, in solving a concrete problem C by a metaheuristic H (such as stochastic optimization, e volutionary computation, etc.), a key to success is to communicate as much information as possible in the translation from C to H —that is, to let H exploit structures unique to C . Our idea of time staging follo ws this philosophy . More specifically , via time staging we communicate the time-causal structure of time-dependent signals—a structure that is present in some instances of the falsification problem b ut not in optimization problems in general—to stochastic optimization solvers. Our implementation of time-staged falsification is based on Breach [8]. W e sho w that this simple idea can dramatically enhance its performance in some examples. W e also present some theoretical considerations on the kinds of problem instances where time staging is likely to work, and some results that aid implementation of time staging. Structure of the Paper In §2 we informally outline optimization-based falsification and illustrate the idea of time staging. W e then turn to formal de velopments: in §3 we re view existing falsification works and in §4 we present our algorithm, augmented by some theoretical results that aid its implementation. §5 is de voted to the theoretical consideration of two specific settings in which time staging is guaranteed to work well. These settings will serve as useful “rules of thumb” for practical applications. In §6 we discuss our implementation and experimental results. W e conclude in §8. 2 Schematic Over view: F alsification and Time Staging W e illustrate f alsification and time staging, informally with an example. Example Setting W e take as a system model M a simple automoti ve powertrain whose input signal is 1-dimensional (the throttle u ) and whose output signal is the vehicle speed v . W e assume that M exhibits the follo wing natural behavior: the larger u is, the quicker v grows. Let our specification be ϕ ≡  2 ( v ≤ 120 )  , where ≡ denotes the syntactic equality . T o falsify ϕ the vehicle speed v must e xceed G. Ernst, I. Hasuo, Z. Zhang & S. Sedwards 27 · · · − → i -th sampling time throttle u ( i ) 1 u ( i ) 2 u ( i ) 3 u ( i ) 4 time vehicle speed 120 v ( i ) Choosing u ( i +1) 1 , . . . , u ( i +1) K by optimization − − − − − − − − − − − − − − − → ( i + 1)-th sampling time throttle u ( i +1) 1 u ( i +1) 2 u ( i +1) 3 u ( i +1) 4 time vehicle speed 120 v ( i +1) − → · · · Figure 2: Con ventional optimization-based f alsification (without time staging). first stage time throttle u (1) 1 u (2) 1 u ( n 1 ) 1 . . . . . . . . . optimization time vehicle sp eed 120 v (1) 1 v (2) 1 v ( n 1 ) 1 . . . . . . . . . optimization Choosing the best prefix u ( n 1 ) 1 − − − − − − − − → second stage time throttle u ( n 1 ) 1 u (1) 2 u (2) 2 u ( n 2 ) 2 . . . . . . . . . optimization time vehicle sp eed v ( n 1 ) 1 v (1) 2 . . . . . . . . . v ( n 2 ) 2 optimization − → · · · Figure 3: Falsification with time staging input signal u robustness J M ( u ) , ϕ K u (2) u (1) · · · u ( i ) hill-climbing u ( i +1) ? Figure 4: Hill-climbing optimization in falsification ⇒ ⇒ ⇒ · · · ⇒ Figure 5: Nelder-Mead optimization. Here the in- put space is the two-dimensional square, and the (unkno wn) score function is depicted by contour lines. Figures are from W ikipedia 28 T ime-Staging Enhancement of Hybrid System Falsification 120. From the assumption about M ’ s behavior , we e xpect u to be lar ge in a f alsifying input signal. Note that this is a simplified version of one of our e xperiments in §6. Optimization-Based F alsification Fig. 2 illustrates how a con ventional optimization-based falsifica- tion procedure works. In the i -th sampling one tries an input signal u ( i ) . Following the falsification litera- ture we focus on piecewise constant signals. Thus a signal u ( i ) is represented by a sequence ( u ( i ) 1 , . . . , u ( i ) K ) of real numbers. See the top left of Fig. 2. The corresponding output signal v ( i ) = M ( u ( i ) ) is shown belo w it. Since v ( i ) does not reach the threshold 120, we mov e on to the ( i + 1 ) -th sampling and try a new input signal u ( i + 1 ) = ( u ( i + 1 ) 1 , . . . , u ( i + 1 ) K ) . The choice of u ( i + 1 ) is made by an optimization algorithm. Specifically , the optimization algorithm observ es the results of the pre viously sampled input signals u ( 1 ) , . . . , u ( i ) —especially the robustness v alue J M ( u ( i ) ) , ϕ K that each input u ( i ) achie ves. In the current setting where ϕ ≡  2 ( v ≤ 120 )  , the robustness v alue is simply the difference between 120 and the peak vehicle speed. The optimization algorithm tries to deriv e some general tendency , which it then uses to increase the probability that the next input signal u ( i + 1 ) will make the robustness smaller (i.e. the peak vehicle speed higher). Hill climbing is a prototype of such optimization algorithms. Its use in falsification is illustrated in Fig. 4, where u = ( u 1 , . . . , u K ) is depicted as one-dimensional for clarity . The actual curve for the robustness value J M ( u ) , ϕ K (gray and dashed) is unknown. Still the previous observations under input u ( 1 ) , . . . , u ( i ) suggest that to the right is the climbing down direction. The next candidate u ( i + 1 ) is picked accordingly , to wards negati ve robustness. Another well-known optimization algorithm is the Nelder- Mead algorithm . See Fig. 5, where the input space is two-dimensional and the (unkno wn) robustness function is depicted by contour lines. W e see in the right of Fig. 2 that the new input signal u ( i + 1 ) = ( u ( i + 1 ) 1 , . . . , u ( i + 1 ) K ) leads to a corre- sponding output signal v ( i + 1 ) that reduces the robustness value by achieving a higher peak speed. W e continue this way , u ( i + 2 ) , u ( i + 3 ) , . . . , hoping to ev entually reach a falsifying input signal. Absence of the Time-Causal Inf ormation A closer look at Fig. 2 rev eals room for improv ement. In Fig. 2, the new input signal u ( i + 1 ) indeed achie ves a smaller ov erall robustness J M ( u ) , ϕ K than u ( i ) . Ho wev er , its initial segment u ( i + 1 ) 1 is smaller than u ( i ) 1 ; consequently the vehicle speed v ( i + 1 ) is smaller than v ( i ) in the first few seconds. K eeping the initial segment u ( i ) 1 would hav e achiev ed an ev en greater peak speed. The problem here is that the time-causal structure inherent in the problem is not explicitly commu- nicated to the optimization algorithm. The relev ant structure is more specifically time monotonicity : an input prefix that achieves smaller robustness (i.e. a greater peak speed) is more likely to extend to a full falsifying input signal. Although it is possible that a stochastic optimization algorithm somehow “learns” time monotonicity , it is not guaranteed, because the structure of input spaces (the horizontal axis in Fig. 4 and the squares in Fig. 5) does not explicitly reflect time-causal structures. While the time monotonicity is not shared by all instances of the falsification problems, we find many realistic instances that approximately satisfy the property . W e discuss time monotonicity in §5, as well as in the context of our e xperiments in §6. F alsification with Time Staging Our proposal of time staging consists of incrementally synthesizing a candidate input signal. W e illustrate this in Fig. 3. In the first stage (left), we run a falsification algorithm and try to find an initial input segment that achie ves low robustness (i.e. high peak speed). This first stage G. Ernst, I. Hasuo, Z. Zhang & S. Sedwards 29 comprises running n 1 samplings, as illustrated in Fig. 2. This process will gradually improve candidates for the initial input segment, in the way the arrows ↑ on the left in Fig. 3 designate. Let us assume that the last candidate u ( n 1 ) 1 is the (tentati ve) best, achie ving the smallest robustness. In the second stage (on the right in Fig. 3) we continue u ( n 1 ) 1 and synthesize the second input segment. This is again by running a falsification algorithm, as depicted. Note that, in each stage (a box in Fig. 3), the whole iterated process in Fig. 2 is conducted. In this way we continue to the K -th stage, always starting with the input segment that performed the best in the previous stage, thus exploiting the time- causal structure. While time staging is not difficult to implement, there is a challenge in using it ef fectiv ely . An imme- diate question is whether choosing the single best input segment in each stage is the optimal approach. Our current strategy fa vors e xploitation ov er exploration: it might miss a falsifying signal whose rob ust- ness must decrease slowly in the earlier segments and only quickly in the latter segments. Indeed we are working on an ev olutionary variant of the above time-staged algorithm, where multiple segments are passed ov er from one stage to another , in order to maintain diversity and conduct exploration. That said, ev en under the current simple strategy of picking the best one, we observe significant performance enhancement in some falsification problems. See §6. W e can summarize this trade-of f in terms of the size of search s paces. Let U be the set of candidates for input segments, and K be the number of stages. Then the size of the set of whole input signals is | U | K , choosing one input segment for each stage. In our staged algorithm, in contrast, the search space for each stage is U and ov erall our search space is K · | U | . This reduction comes with the risk of missing some f alsifying input signals. The experimental results in §6 suggest this risk is worth taking. Moreover , in §5 we present some theoretical conditions for the absence of such risk. They help users decide in practical applications when time staging will be ef fectiv e. 3 Optimization-Based F alsification From this section on we turn to the formal description and analysis of our algorithm. This section presents a re view of e xisting works on optimization-based falsification. System Models Let us formalize our system models. Definition 3.1 (time-bounded signal) . Let T ∈ R > 0 be a positiv e real. A (time-bounded) m-dimensional signal with a time horizon T is a function w : [ 0 , T ] → R m . Let w : [ 0 , T ] → R m and w 0 : [ 0 , T 0 ] → R m be (time-bounded) signals. Their concatenation w · w 0 : [ 0 , T + T 0 ] → R m is defined by ( w · w 0 )( t ) : = w ( t ) if t ∈ [ 0 , T ] , and w 0 ( t − T ) if t ∈ ( T , T + T 0 ] . Let T 1 , T 2 ∈ ( 0 , T ] such that T 1 < T 2 . The r estriction w | [ T 1 , T 2 ] : [ 0 , T 2 − T 1 ] → R m of w : [ 0 , T ] → R m to the interv al [ T 1 , T 2 ] is defined by ( w | [ T 1 , T 2 ] )( t ) : = w ( T 1 + t ) . Definition 3.2 (system model M ) . A system model , with M -dimensional input, is a function M that takes an input signal u : [ 0 , T ] → R M and returns M ( u ) : [ 0 , T ] → R N . Here the common time horizon T ∈ R > 0 is arbitrary . Furthermore, we impose the following causality condition on M . For any time-bounded signals u : [ 0 , T ] → R M and u 0 , we require that M ( u · u 0 )   [ 0 , T ] = M ( u ) . Note that M ( u · u 0 ) = M ( u ) · M ( u 0 ) does not hold in general: feeding u can change the internal state of M . This moti vates the follo wing definition. 30 T ime-Staging Enhancement of Hybrid System Falsification Definition 3.3 (continuation M u ) . Let M be a system model and u : [ 0 , T ] → R M be a signal. The continuation of M after u , denoted by M u , is defined as follows. For an input signal u 0 : [ 0 , T 0 ] → R M : M u ( u 0 )( t ) : = M ( u · u 0 )( T + t ) . Signal T emporal Logic and Robust Semantics W e revie w signal temporal logic ( STL ) [21] and its r obust semantics [10, 14]. V ar is the set of variables, and let N : = | V ar | . V ariables stand for physical quantities, control modes, etc. ≡ denotes syntactic equality . Definition 3.4 (syntax) . In STL , atomic pr opositions and formulas are defined as follows, respectiv ely: α :: ≡ f ( x 1 , . . . , x n ) > 0, and ϕ :: ≡ α | ⊥ | ¬ ϕ | ϕ ∧ ϕ | ϕ U I ϕ . Here f is an n -ary function f : R n → R , x 1 , . . . , x n ∈ V ar , and I is a closed non-singular interv al in R ≥ 0 , i.e. I = [ a , b ] or [ a , ∞ ) where a , b ∈ R and a < b . W e omit subscripts I for temporal operators if I = [ 0 , ∞ ) . Other common connecti ves like ∨ , → , > , 2 I (always) and 3 I (e ventually), are introduced as abbreviations: 3 I ϕ ≡ > U I ϕ and 2 I ϕ ≡ ¬ 3 I ¬ ϕ . Atomic formulas like f (  x ) ≤ c , where c ∈ R is a constant, are also accommodated by using ne gation and the function f 0 (  x ) : = f (  x ) − c . Definition 3.5 (robust semantics [9, 10]) . For an unbounded n -dimensional signal w : R ≥ 0 → R n and t ∈ R ≥ 0 , w t denotes the t -shift of w , that is, w t ( t 0 ) : = w ( t + t 0 ) . Let w : R ≥ 0 → R N be a signal (recall N = | V ar | ), and ϕ be an STL formula. W e define the r obustness J w , ϕ K ∈ R ∪ { ∞ , − ∞ } as follo ws, by induction. Here u and t denote infimums and supremums of real numbers, respecti vely . J w , f ( x 1 , · · · , x n ) > 0 K : = f  w ( 0 )( x 1 ) , · · · , w ( 0 )( x n )  J w , ⊥ K : = − ∞ J w , ¬ ϕ K : = − J w , ϕ K J w , ϕ 1 ∧ ϕ 2 K : = J w , ϕ 1 K u J w , ϕ 2 K J w , ϕ 1 U I ϕ 2 K : = F t ∈ I  J w t , ϕ 2 K u d t 0 ∈ [ 0 , t ) J w t 0 , ϕ 1 K  Here are some intuitions and consequences. The rob ustness J w , f (  x ) > c K stands for the vertical margin f (  x ) − c for the signal w at time 0. A negati ve robustness value indicates how far the formula is from being true. The robustness for the ev entually modality is computed by J w , 3 [ a , b ] ( x > 0 ) K = F t ∈ [ a , b ] w ( t )( x ) . The original semantics of STL is Boolean, giv en by a binary relation | = between signals and for- mulas. The robust semantics refines the Boolean one, in the sense that: J w , ϕ K > 0 implies w | = ϕ , and J w , ϕ K < 0 implies w 6| = ϕ . Optimization-based falsification via robust semantics [14] hinges on this refinement. See [10]. Although the definitions so far are for unbounded signals only , we note that the robust semantics J w , ϕ K , as well as the Boolean satisfaction w | = ϕ , allows straightforward adaptation to time-bounded signals (Def. 3.1). See Appendix A. F alsification Solv ers In the ne xt definition, a prototype of a score function ρ is gi ven by the robustness ρ ϕ of a gi ven STL specification ϕ . The generality of allowing other ρ is needed later in §4. ρ ϕ ( v ) : = J v , ϕ K (1) Definition 3.6 (falsification solver) . A falsification solver is a stochastic algorithm F alsify that takes, as input: 1) a system model M (Def. 3.2) with M -dimensional input; 2) a score function ρ that takes an output signal v of M and returns a score ρ ( v ) ∈ R ∪ {− ∞ , ∞ } ; and 3) a time horizon T ∈ R > 0 . The algorithm F alsify returns an M -dimensional signal u : [ 0 , T ] → R M . G. Ernst, I. Hasuo, Z. Zhang & S. Sedwards 31 Algorithm 1 Internal Structure of a Falsification Solv er Falsify ( M , ρ , T ) Require: a system model M , a score function ρ , and T ∈ R > 0 1: U ← ()  the list U collects all the candidates u : [ 0 , T ] → R M 2: while ¬ InitialSamplingDone ( T , U ) do 3: u ← InitialSampling ( T )  u : [ 0 , T ] → R M is sampled follo wing some recipe 4: U ← cons ( U , u ) 5: while ¬ OptimizationSamplingDone ( M , ρ , T , U ) do 6: u ← OptimizationSampling ( M , ρ , T , U ) 7:  u is sampled, so that ρ ( M ( u )) becomes small, based on previous samples in U 8: U ← cons ( U , u ) 9: u ← ar g min u ∈ U ρ ( M ( u )) 10: return u  a trial is successful if ρ ( M ( u )) < 0 Each in v ocation Falsify ( M , ρ , T ) of the solver is called a falsification trial . It is successful if the returned signal u satisfies ρ ( M ( u )) < 0. Note that the returned signal u can differ in ev ery trial, since F alsify is a stochastic algorithm. W e further assume the internal structure of the solver F alsify follows the scheme in Algorithm 1. It consists of two phases. The first initial sampling phase collects some candidates for u : [ 0 , T ] → R M regardless of the system model M or the score function ρ . In the second optimization sampling phase, a stochastic optimization algorithm is employed to sample a candidate u that is likely to make the score ρ ( M ( u )) small. Implementation of Falsification Solvers Both Breach [8] and S-T aliro [5] tak e industry-standard Simulink models as system models. For input signal candidates the tools focus on piecewise constant signals; they are represented by sequences ( u 1 , . . . , u K ) of real numbers, much like in §2. Here K is the number of contr ol points ; in our staged algorithm we use the same K for the number of stages. The tools offer multiple stochastic optimization algorithms for the optimization sampling phase, including CMA-ES [6], global Nelder-Mead and simulated annealing . The initial sampling phase is mostly by random sampling. Additionally , in Breach with global Nelder-Mead, so-called corner samples are added to the list U . The number of corner samples grows exponentially as K grows, i.e. as we have more control points. 4 T ime Staging in Optimization-Based Falsification Definition 4.1 (time-staged deployment of f alsification solv er) . Let M be a system model, ϕ be an STL formula, and T ∈ R > 0 be a time horizon. Let K ∈ N be a parameter; it is the number of time stages. The time-staged deployment of a falsification solver F alsify is the procedure in Algorithm 2. On the line 3, the model M u is the continuation of M after u (Def. 3.3); the score function ∂ v ρ ϕ is defined by ( ∂ v ρ ϕ )( v 0 ) : = ρ ϕ ( v · v 0 ) ( 1 ) = J v · v 0 , ϕ K . (2) The whole procedure is stochastic (since F alsify is); an in v ocation is called a time-staged falsification trial . It is successful if the returned signal u satisfies J M ( u ) , ϕ K < 0. A falsification trial (i.e. an in vocation of Algorithm 1) is an iterati ve process: the more we sample, the more likely we obtain a falsifying input signal. Since we run multiple falsification trials in Algorithm 2 32 T ime-Staging Enhancement of Hybrid System Falsification Algorithm 2 T ime-Staged Deployment of a F alsification Solver Require: a falsification solv er Falsify , a system model M , an STL formula ϕ , T ∈ R > 0 and K ∈ N 1: u ← ()  the input prefix obtained so far . W e start with the empty signal () 2: for j ∈ { 1 , . . . , K } do 3: u 0 ← F alsify ( M u , ∂ M ( u ) ρ ϕ , T K )  synthesizing the j -th input segment 4: u ← u · u 0  concatenate u 0 , after which the length of u is jT K 5: return u  a time-staged falsification trial is successful if J M ( u ) , ϕ K < 0 (one trial for each of the K stages), an important question is ho w we distrib ute a vailable time to dif ferent stages. A simple strategy is to fix the number of samples in each phase of Algorithm 1. Then the predicates InitialSamplingDone ( T , U ) and OptimizationSamplingDone ( M , ρ , T , U ) are giv en by | U | > N init max and | U | > N opt max , where N init max , N opt max are constants. An adaptive strategy , that we also implemented for the optimization sampling phase, is to continue sampling until we stop seeing progress. Here we fix a parameter N stuck max , and we stop after N stuck max con- secuti ve samplings without reducing robustness. A similar strategy of adapti vely choosing the number of samples can be introduced for random sampling in the initial sampling phase (the lines 2 – 4 of Algo- rithm 1). 4.1 T owards Efficient Implementation A key to speedup of Algorithm 2 is in the line 3; more specifically , ho w we handle the previous input prefix u . Here we discuss two directions, one on the model M u and the other on the score function ∂ v ρ ϕ . (W e note that the suggested enhancements are not currently used in our implementation, because of performance reasons. See below .) Continuation of Models Optimization-based falsification has a very wide application domain. Since it only requires a black-box model M , the concrete form of M can vary from a program to a Simulink model and e ven a system with hardw are components ( HILS ). These models can be very big, and usually the bottleneck in falsification lies in simulation , that is, to compute M ( u ) giv en an input signal u . In the line 3 of Algorithm 2, therefore, using the definition M ( u · u 0 )( T + t ) in Def. 3.3 is in principle not a good strategy: it requires simulation of M for the whole prefix u · u 0 , which can be av oided if we can directly simulate the continuation M u . In Simulink this is possible by saving the snapshot of the model after a simulation, via the SaveFinalState model configuration parameter . In our implementation we do not do so, though, because the ov erhead of saving and loading snapshots is currently greater than the cost of simulating. This balance can become different, if we figure out a less expensiv e way to use snapshots, or if we study more complex models. Derivati ve of F ormulas The situation is similar with the score function ∂ M ( u ) ρ ϕ in the line 3 of Algo- rithm 2. Using the presentation ρ ϕ ( M ( u ) · v 0 ) in (2) requires scanning the same prefix M ( u ) repeatedly . Desired here is a syntactic presentation of ∂ M ( u ) ρ ϕ , that will be gi ven as an STL formula ∂ M ( u ) ϕ such that ∂ M ( u ) ρ ϕ = ρ ( ∂ M ( u ) ϕ ) . This would allo w one to utilize av ailable algorithms for computing rob ustness v alues J v , ∂ M ( u ) ϕ K . Definition 4.2 (deriv ati ve of flat STL formulas) . Let T ∈ R > 0 , and v : [ 0 , T ] → R N be a signal. Giv en an STL formula ϕ that is flat in the sense that it does not have nested temporal operators, the derivative G. Ernst, I. Hasuo, Z. Zhang & S. Sedwards 33 ∂ v ϕ by v is defined inducti vely as follo ws. ∂ v  f (  x ) > 0  : ≡ c J v , f (  x ) > 0 K ∂ v ⊥ : ≡ ⊥ ∂ v ( ¬ ϕ ) : ≡ ¬ ∂ v ϕ ∂ v ( ϕ 1 ∧ ϕ 2 ) : ≡ ( ∂ v ϕ 1 ) ∧ ( ∂ v ϕ 2 ) ∂ v ( ϕ 1 U I ϕ 2 ) : ≡ c J v , ϕ 1 U I ϕ 2 K ∨  ( c J v , 2 ϕ 1 K ∧ ϕ 1 ) U I − T ϕ 2  Here the interval I − T is obtained by shifting endpoints, such as [ a , b ] − T = [ a − T , b − T ] . For each r ∈ R , the notation c r abbre viates the atomic formula r > 0, where r is thought of as a constant function. W e use the fact that J w , c r K = r . Until formulas ϕ 1 U I ϕ 2 are split into the ev aluation on the signal prefix v (first disjunct), and a continuation (second disjunct). The constant c J v , 2 ϕ 1 K injects the robustness of ϕ 1 seen so far into the residual formula (recall that both of 2 and ∧ take the infimum). It follo ws that ∂ v  2 I ϕ  ≡ c J v , 2 ϕ K ∧ 2 I − T ϕ and ∂ v  3 ϕ  ≡ c J v , 3 ϕ K ∨ 3 I − T ϕ . Proposition 4.3. Let T ∈ R > 0 , v : [ 0 , T ] → R N be a signal, and ϕ be a flat STL formula. W e have, for any v 0 : [ 0 , T 0 ] → R N , J v 0 , ∂ v ϕ K = J v · v 0 , ϕ K . A proof is in Appendix B. Use of deri vati ves for timed specifications is also found e.g. in [22]. The settings are different, though: Boolean semantics in [22] while our semantics is quantitative. Our restriction to flat formulas comes mainly from this difference, and lifting the flatness restriction seems hard. 5 Sufficient Conditions f or Time Staging W e present some theoretical analyses of the performance of time staging that indicate to which class of systems the time-staged approach can apply . W e giv e some sufficient conditions under which the approach is guaranteed to work. Ho wev er , it should be noted that it is not necessary that a concrete system satisfies these conditions strictly as these are rather restricti ve. Nev ertheless, we belie ve that users with expert domain knowledge can judge whether their models satisfy these conditions approximately . This way our results pro vide those users with “rules of thumb . ” As we discussed in the last paragraph of §2, the potential performance adv antage by time staging comes from the reduction of search spaces from | U | K to K · | U | . Here U is the set of potential input segments for each stage, and K is the number of stages. This advantage comes at the risk of missing out some error input signals. The follo wing basic condition (3), that we call incr emental falsification , ensures that there is no such risk. More precisely , we can decompose the “best” input signal u into a first stage u 1 and its remainder u 2 such that the entire falsification problem (left hand side) is solved by greedy optimization of the initial segment (inner arg min u 1 ), and subsequent optimization of the continuation (outer min u 2 ). For all choices of T 1 , T 2 with ranges u : [ 0 , T 1 + T 2 ] → R m and u i : [ 0 , T i ] → R m : min u J M ( u ) , ϕ K = min u 2 r M   arg min u 1 J M ( u 1 ) , ϕ K  · u 2  , ϕ z (3) Algorithm 2 repeatedly unfolds (3) by picking constant T 1 = T / K where T is the time horizon and K is the number of stages. The rest of this section is dev oted to the search for concrete sufficient conditions for (3). 34 T ime-Staging Enhancement of Hybrid System Falsification Monotone Systems and Ceiling Specifications W e formalize the time monotonicity property in §2. That it implies incremental falsification (3) can be easily pro ved. Definition 5.1 (time-monotone falsification problem) . A system model M and an STL formula ϕ are said to constitute a time-monotone falsification pr oblem if, for any input signals u 1 , u 0 1 : [ 0 , T 1 ] → R m and u 2 : [ 0 , T 2 ] → R m , J M ( u 1 ) , ϕ K ≤ J M ( u 0 1 ) , ϕ K implies J M ( u 1 · u 2 ) , ϕ K ≤ J M ( u 0 1 · u 2 ) , ϕ K . W e in vestigate yet more concrete conditions that ensures time monotonicity . The follo wing condition on system models is assumed in the example of §2. Definition 5.2 (monotone system, ceiling specification) . Let x be a variable (for output). A system model M is said to be monotone in x if, for each u 1 , u 0 1 : [ 0 , T 1 ] → R M and u 2 : [ 0 , T 2 ] → R M , M ( u 1 )( T 1 )( x ) ≤ M ( u 0 1 )( T 1 )( x ) implies M ( u 1 · u 2 )( T 1 + T 2 )( x ) ≤ M ( u 0 1 · u 2 )( T 1 + T 2 )( x ) . An STL formula of the form 2 ( x < c ) , where x is a variable and c ∈ R is a constant, is called a ceiling formula . One can speculate that a monotone system and a ceiling specification 2 ( x < c ) , like those in §2, constitute a time-monotone falsification problem. The speculation is not true, unfortunately; a coun- terexample is easily constructed using a model M whose output signal is not increasing. W e can instead sho w the following weak er property . Definition 5.3 (truncated time monotonicity) . A system model M and an STL formula ϕ constitute a truncated time-monotone falsification pr oblem if, for any input u 1 , u 0 1 : [ 0 , T 1 ] → R m and u 2 : [ 0 , T 2 ] → R m , J M ( u 1 ) , ϕ K ≤ J M ( u 0 1 ) , ϕ K implies existence of T ∈ ( 0 , T 1 ] such that J M (( u 1 | [ 0 , T ] ) · u 2 ) , ϕ K ≤ J M (( u 0 1 | [ 0 , T ] ) · u 2 ) , ϕ K . Proposition 5.4. Let M be a model monotone in x, and ϕ ≡  2 ( x < c )  . Then M and ϕ constitute a truncated time-monotone falsification pr oblem. The proof, in Appendix C.1, constructs a concrete choice of T in Def. 5.3. Specifically it is the instant T ∈ [ 0 , T 1 ] in which the robustness J M ( u 1 | [ 0 , T ] ) , ϕ K is minimum. In the scenario of §2 this is the instant that the vehicle speed is in its peak. Note that truncated time monotonicity does not guarantee incremental falsification as per (3), but it implies that the current rigid time staging at 0 , T K , 2 T K , . . . , ( K − 1 ) T K is not optimal. These theoretical considerations suggest potential improv ement of the staged procedure in Def. 4.1 with adapti ve choice of stages, which is left for future work. Stateless Systems and Reachability Specifications Here is another sufficient condition. Definition 5.5 (stateless system, reachability formula) . A system model M is said to be stateless if, for any input signals u 1 , u 0 1 : [ 0 , T 1 ] → R m and u 2 : [ 0 , T 2 ] → R m , we have M ( u 1 · u 2 ) | ( T 1 , T 2 ] = M ( u 0 1 · u 2 ) | ( T 1 , T 2 ] . An STL formula 3 ψ , where ψ is modality-free, is called a reac hability formula . Note that being stateless is a sufficient but not necessary condition for M ( u 1 · u 2 ) = M ( u 1 ) · M ( u 2 ) . Statelessness requires insensiti vity to previous input prefix es, but a stateless system can still be sensiti ve to time. Proposition 5.6. Let M be a stateless system and ϕ be a reac hability specification ϕ ≡ ( 3 ψ ) . Then M and ϕ satisfy the incr emental falsification pr operty (3). A proof is easy . A typical situation in which we would appeal to Prop. 5.6 is when: the specification is 3 ( x < c 1 ∨ x > c 2 ) (which can be hard to falsify if c 1 < c 2 are close); and the system is already in its stable state (so that its behavior does not depend much on what happened during the transient phase). Our experiments in §6 demonstrate that time staging can drastically impro ve performance in such settings. G. Ernst, I. Hasuo, Z. Zhang & S. Sedwards 35 T able 1: Experimental results. Each column shows how many falsification trials succeeded (out of 20), and the av erage runtime. S1: 2 [ 0 , 30 ] ( v < 120 ) . S2: 2 [ 0 , 30 ] ( g = 3 → v ≥ 30 ) . S3: 3 [ 10 , 30 ] ( v ≤ v min ∨ v ≥ v max ) , where: v min = 50, v max = 60 (easy); v min = 53, v max = 57 (hard). S4: 2 [ 0 , 10 ] ( v < v ) ∨ 3 [ 0 , 30 ] ( ω > ω max ) , where: v min = 80 , ω max = 4500 (easy); v min = 50 , ω max = 2700 (mid); v min = 50 , ω max = 2520 (hard). The specification S for the Abstract Fuel Control model is ¬ ( 3 [ t 1 , t 2 ]  [ 0 , t 0 ] ( AF − AF ref > δ ∗ 14 . 7 )) , where: t 1 = 0, t 2 = 6, t 0 = 3, δ = 0 . 07 (init); t 1 = 6, t 2 = 26, t 0 = 4, δ = 0 . 01 (stable). Starred numbers 0* or 20* indicate that GNM is deterministic so all trials yield the same result. model Automatic T ransmission Abst. Fuel Ctrl. spec. S1 S2 S3 easy S3 hard S4 easy S4 mid S4 hard S init S stable algorithm time #/20 time #/20 time #/20 time #/20 time #/20 time #/20 time #/20 time #/20 time #/20 CMA-ES 27s 20 5s 20 39s 14 57s 0 32s 16 37s 9 59s 0 49s 0 82s 1 +TS 52s 15 15s 16 9s 19 23s 11 15s 14 14s 14 24s 3 30s 0 42s 1 +A-TS 41s 18 15s 17 9s 16 21s 10 26s 14 22s 14 20s 5 26s 0 41s 0 SA 50s 5 43s 7 37s 9 55s 0 35s 6 36s 9 47s 5 51s 0 76s 2 +TS 37s 20 33s 16 11s 19 33s 8 21s 14 25s 13 51s 0 47s 1 54s 7 +A-TS 34s 20 18s 17 9s 18 26s 4 16s 18 21s 11 30s 2 34s 0 42s 5 GNM 6s 20* 61s 0* 56s 0* 55s 0* 43s 0* 46s 0* 53s 0* 50s 0* 86s 0* +TS 42s 20* 15s 20* 13s 20* 25s 20* 11s 20* 45s 0* 52s 0* 30s 20* 20s 20* +A-TS 20s 20* 16s 20* 10s 20* 26s 20* 13s 20* 45s 0* 43s 0* 37s 0* 19s 20* Corner Samples f or Global Nelder -Mead The reduction of search spaces from | U | K to K · | U | has its analogue in the number of corner samples in Breach with global Nelder-Mead (lines 2 – 4 of Algorithm 1, see the last paragraph of §3). Originally the number of corner samples is 2 K · M , where K is the number of control points and M is the number of input v alues. By introducing K time stages, the total number of corner samples is reduced to K · 2 M . 6 Experiments W e compare the success rate and time consumption of the proposed method. The benchmarks here use automoti ve Simulink models that are commonly used in the falsification literature. Specifications are chosen taking the deliberations of §5 into account, namely with ceiling specifications (Def. 5.2, including the example of §2), a reachability specification (Def. 5.5) and a combination thereof. The base line is Algorithm 1 implemented by Breach [8]. The methods proposed in §4 are imple- mented on top of Breach: the time-staged Algorithm 2 (TS), and the adaptive strategy (A-TS, the one described after Def. 4.1). All three algorithms (plain, TS, A-TS) are combined with different optimiza- tion solvers: CMA-ES, simulated annealing (SA), global Nelder-Mead (GNM), obtaining a total of nine configurations. The results in T able 1 indicate that both success rate and runtime performance are significantly im- prov ed by time staging, often finding counterexamples when non-staged Breach fails (e.g. columns S3 har d and S init ). Furthermore, we see that while the adapti ve algorithm (A-TS) does not necessarily lead to a higher success rate in comparison to the time-staged one (TS), it gives yet another runtime perfor- mance improvement. Howe ver , as discussed in detail in §6, there is no overall best algorithm, and time staging af fects the optimization algorithms differently depending on the problem. Benchmarks Automatic T ransmission is a Simulink model that was proposed as a benchmark for fal- sification in [15]. It has input v alues thr ottle ∈ [ 0 , 100 ] and brake ∈ [ 0 , 325 ] , and outputs the car’ s speed v , 36 T ime-Staging Enhancement of Hybrid System Falsification the engine rotation ω , and the selected gear g . W ith this model we consider fiv e specifications S1–5 . The first two are ceiling ones. Specification S1 2 [ 0 , 30 ] ( v < 120 ) (cf. the example in §2) states the speed be always below a threshold. This property is easily falsified with thr ottle = 100. Specification S2 2 [ 0 , 30 ] ( g = 3 → v ≥ 30 ) states that it is not possible to dri ve slo wly in a high gear . A falsifying trajectory first has to speed up to reach this gear and subsequently roll out until speed falls below the threshold. This latter part of the trajectory can again be seen as a ceiling specification. Note that this property is interesting because the robustnes does not provide an y guidance unless gear 3 has been entered by the system. Specification S3 is a reachability problem, 3 [ 10 , 30 ] ( v ≤ v min ∨ v ≥ v max ) , that encodes the search for a trajectory that keeps the speed between a lo wer and upper bound. The falsification problem consists of two sub-challenges: 1) hitting this speed interval precisely after an initial acceleration up to 10s simulated time; and 2) maintaining a correct speed till the time horizon. This suggests that a natural decomposition of the problem can indeed be achie ved by separating these two aspects in time. Specification S4 2 [ 0 , 10 ] ( v min < v ) ∨ 3 [ 0 , 30 ] ( ω > ω max ) expresses that speed v min can only be reached with an engine rotation e xceeding a threshold ω max . This specification is mentioned in [15] and e valuated in e.g., [3, 4], too. T o f alsify , a trajectory must be found that reaches speed v early with an engine rotation lo wer than ω max . The difficulty increases with higher v min and lower ω max , respectiv ely . The formula represents the mixture of ceiling and reachability specifications. The second system model is Abstract Fuel Contr ol from [17]. It takes two input values, pedal angle (from [ 0 , 61 . 1 ] ) and engine speed (from [ 0 , 1100 ] ); it outputs air-fuel ratio AF , which influences fuel ef ficiency and performance of the car . The value of AF is expected to be close to a reference v alue AF ref . According to [17], this setting corresponds to the so-called normal mode , where AF ref = 14 . 7 is constant. W e used the specification ¬ ( 3 [ t 1 , t 2 ]  [ 0 , t 0 ] ( AF − AF ref > δ ∗ 14 . 7 )) : the air-fuel ratio does not de viate from an acceptable range for more than t 0 seconds. W e ev aluated this specification with two parameter sets: the initial period with a larger error mar gin, and the stable period with a smaller margin. See T able 1 for parameter v alues. Experimental Setup and Results For the experiments with the Automatic T ransmission model, the input signals were piecewise constant with 5 control points. The time horizon was T = 30. The param- eters outlined in §4 were as follows: the maximum number of samplings for each plain (non-staged) falsification trial was 150 (initial and optimization samplings combined). In the time-staged (TS) trials, we make the number of stages coincide with that of control points. Analogously , the sampling budget per stage was set to 30 for K = 5 stages, resulting in overall 150 samplings. The adaptive algorithm (A-TS) ran with the threshold N stall max = 30 / 2 = 15 per each of fi ve stages. The e xperiments with the Abstract Fuel Control model were run up to the time horizon T = t 2 + t 0 where t 2 and t 0 are as in T able 1. W e used three and fiv e stages, respectiv ely , for the initial and stable specifications. These again coincide with the number of control points. The TS algorithms conducted 30 samplings in each stage. The experiments ran Breach version 1.2.9 and MA TLAB R2017b on an Amazon EC2 c4.8xlarge instance with a 36 core Intel(R) Xeon(R) CPU (2.90GHz) and 58G of main memory . Ho wever , we did not use the opportunity to parallelize, and the time reported is in the same order of magnitude as that of a modern desktop workstation. The results are sho wn in T able 1. They are grouped by the underlying stochastic optimization algo- rithm: CMA-ES, simulated annealing (SA) and global Nelder -Mead (GNM). In each group, we compare plain (unstaged) Breach to the time-staged (TS) and the adaptive time-staged (A-TS) ones. W e compare av erage runtimes (lower is better) and the success rate (higher is better), aggregated over 20 falsifica- G. Ernst, I. Hasuo, Z. Zhang & S. Sedwards 37 tion trials for each configuration. Those good results which deserve attention are highlighted in bold. Note that the implementation of the global Nelder -Mead algorithm in Breach uses a deterministic source of quasi-randomness (Halton sequences), which implies that whether GNM finds a counterexample is consistent across all trials (marked with an asterisk ∗ ). Discussion Focusing on the Automatic T ransmission model first, we see that CMA-ES works well for S1, although GNM performs e ven better (6s, supposedly because it uses corner samples, see §3). T ime staging introduces ov erhead to CMA-ES and GNM, because each stage is optimized individually . In contrast, simulated annealing (SA) benefits from time staging for the two ceiling specifications S1–2. W e presume that since SA emphasizes exploration, it benefits from exploitation added by time staging (cf. §2). The second specification S2 is slightly more complex: before gear 3 is reached, there is no guidance from the robustness semantics, because J , g = 3 K = − ∞ masks any quantitativ e information on v . Hence, falsifying this property needs some luck during the collection of initial samples in Algorithm 1. CMA- ES apparently exploits this, see top of column S2 of T able 1. Considering the other algorithms, SA and GNM, both benefit from time staging: exploitation of time causality prevents these good trajectory prefixes from being discarded accidentally once the required gear is reached (cf. Fig 2). The results for S3 are ev aluated with two different choices of parameters. The harder instance was falsified by the time-staged algorithms only , which can likely be attributed to the flattening of the search space from size | U | K to K · | U | (§5). S4 is evidently harder than the pre vious ones. Time staging improves performance in a general tendency b ut not in all cases (SA for S4 hard). The results for the Abstract Fuel Control model (the last two columns in T able 1) sho w that the time-staged algorithms boost the ability to falsify some rare e vents. The specification for the initial stage where AF is still unstable (S init) can be considered a rare e vent since all the three non-staged algorithms failed to falsify it. T ime-staged SA and time-staged GNM managed to find error inputs. In the last column (stable period) is remarkable, too, where success rate and run time of SA and GNM significantly improv ed. Overall, while the performance of the non-staged algorithms suffers from tightening the bounds, the time-staged v ersions are able to find f alsifying trajectories with good success rates while at the same time exhibiting significantly shorter runtimes. 7 Related W ork Falsification is a special case of search-based testing, so considerable research efforts hav e been made to wards cover age [3, 7, 20]. The benefits of co verage in falsification guarantees are tw ofold. Firstly , they indicate confidence for correctness in case no counterexamples are found. Paired with sound robustness estimates for simulations, one can cover a an infinite parameter space with finitely many simulations. C2CE [13] is a recent tool that computes approximations of reachable states using such an approach. Secondly , coverage can be utilized for a better balance between explor ation and exploitation : stochastic optimization algorithms can be called in an interleav ed manner , in which cov erage guides further explo- ration. The approach based on Rapidly-Exploring Random T rees [11] puts an emphasis on exploration by achieving high coverage of the state space. In their algorithm, robustness-guided hill-climbing opti- mization plays a supplementary role. Compared to these works, our current results go in an orthogonal direction, by utilizing time causality to enhance exploitation. The so-called multiple-shooting approach to falsification [24] can be seen of a generalization of RR Ts. It consists of: an upper layer that searches 38 T ime-Staging Enhancement of Hybrid System Falsification for an abstract error trace given by a succession of cells; and a lower layer where an abstract error trace is concretized to an actual error trace by picking points from cells. The approach can discover f alsifying traces by backwards search from a goal region, but needs to concatenate partial traces with potential gaps, which can fail. Furthermore it is unclear ho w to extend it to general STL specifications. A survey of simulation based approaches has been done by Kapinski et al. [18]. Monotonicity has been exploited in different ways for falsification. Robust Neighborhood Descent [1, 2] (RED) searches for trajectories incrementally , restarting the search from points of low robustness. Descent computation of RED assumes explicit deriv ati ves of the dynamics to guarantee con vergence to (local) minima. It is the same principle underlying Prop. 5.4) and our experiments indicate that this principle is useful for black-box optimization, too. In [2], RED is paired with simulated annealing to combine local and global search and to account for more exploration. Doing so for our present work remains to be done in the future. In [16], Hoxha et al. mine parameters θ under which specifications φ [ θ ] are satisfied or falsified by the system. They show that the robust semantics of formulas is monotone in θ and use that fact to tighten such parameters. This is orthogonal to this work as it does not use monotonicity of the system itself. Kim et al. [19] use an idea similar to [16] to partition specifications into upper bounds and lower ceilings. Howe ver , instead of robustness-guided optimization they use exhausti ve exploration of the input space in a way that in turn requires that the system dynamics is monotone in the choice of each input at each time point. This is different from our Def 5.1 of time- monotonicity that aims at incrementally composing good partial choices. The recent work [12] introduces a compositional falsification frame work, focusing on those systems which include machine-learning (ML) components that perform tasks such as image recognition. While the current work aims at the orthogonal direction of finding rare counterexamples, we are interested in its combination with the results in [12], gi ven the increasingly important roles of ML algorithms in CPS. 8 Conclusions and Futur e W ork W e have introduced and ev aluated the idea of time staging to enhance falsification for hybrid systems. The proposed method emphasizes exploitation over exploration as part of stochastic optimization. As there is no single algorithm that fits ev ery problem (as a consequence of having no free lunch [23]), having a v ariety of methods at disposal permits the user of a system to choose the one suitable for the problem at hand. W e hav e shown that the proposed approach is a good fit for problems that suitable exhibit time-causal structures, where it significantly outperforms non-staged algorithms. T wo obvious directions for future work have been pointed out already . Instead of just picking the best trajectory for each stage, it might be beneficial to retain a few , potentially div erse ones in the spirit of e volutionary algorithms (§2). For example, it would be interesting to explore the space between this work on one hand and co verage-dri ven rapidly-exploring random trees. Another idea is to discover time stages adaptiv ely (§5, the discussion after Prop. 5.4). For the ex- periments presented here, we chose to set uniformly fixed stages, which runs the risk of either being too coarse grained (missing some falsifying input), or being too fine grained (wasting analysis time). Finally , another future direction is to explore variations of rob ust semantics to mitigate discrete propositions like g = 3 (§6), for example using a veraging modalities [4]. Other t-norms than min/max for the semantics of conjunction/disjunction could preserve more information from dif ferent subformulas. Acknowledgement. This work is supported by ERA T O HASUO Metamathematics for Systems Design Project (No. JPMJER1603), Japan Science and T echnology Agency . G. Ernst, I. Hasuo, Z. Zhang & S. Sedwards 39 Refer ences [1] Houssam Abbas, Andrew K. W inn, Georgios E. Fainekos & A. Agung Julius (2014): Functional gradient descent method for Metric T emporal Logic specifications . In: American Control Conference, A CC 2014, Portland, OR, USA, June 4-6, 2014 , IEEE, pp. 2312–2317, doi:10.1109/A CC.2014.6859453. [2] Houssam Y Abbas (2015): T est-based falsification and conformance testing for cyber-physical systems . Ph.D. thesis, Arizona State Univ ersity . [3] Arvind S. Adimoolam, Thao Dang, Ale xandre Donz ´ e, James Kapinski & Xiaoqing Jin (2017): Classification and Covera ge-Based F alsification for Embedded Contr ol Systems . In Rupak Majumdar & V iktor Kuncak, editors: Computer Aided V erification - 29th Int. Conf., CA V 2017 , LNCS 10426, Springer , pp. 483–503, doi:10.1007/978-3-319-63387-9 24. [4] T akumi Akazaki & Ichiro Hasuo (2015): T ime Robustness in MTL and Expr essivity in Hybrid System F alsi- fication . In Daniel Kroening & Corina S. Pasareanu, editors: Computer Aided V erification - 27th Int. Conf., CA V 2015 , LNCS 9207, Springer , pp. 356–374, doi:10.1007/978-3-319-21668-3 21. [5] Y ashwanth Annpureddy , Che Liu, Georgios E. Fainekos & Sriram Sankaranarayanan (2011): S-T aLiRo: A T ool for T emporal Logic F alsification for Hybrid Systems . In Parosh Aziz Abdulla & K. Rustan M. Leino, editors: T ools and Algorithms for the Construction and Analysis of Systems - 17th Int. Conf., T ACAS 2011 , LNCS 6605, Springer , pp. 254–257, doi:10.1007/978-3-642-19835-9 21. [6] Anne Auger & Nikolaus Hansen (2005): A restart CMA evolution strate gy with increasing population size . In: Proceedings of the IEEE Congress on Evolutionary Computation, CEC 2005 , IEEE, pp. 1769–1776, doi:10.1109/CEC.2005.1554902. [7] Jyotirmoy V . Deshmukh, Xiaoqing Jin, James Kapinski & Oded Maler (2015): Stochastic Local Searc h for F alsification of Hybrid Systems . In Bernd Finkbeiner , Geguang Pu & Lijun Zhang, editors: Automated T echnology for V erification and Analysis - 13th Int. Symp., A TV A 2015 , LNCS 9364, Springer , pp. 500–517, doi:10.1007/978-3-319-24953-7 35. [8] Alexandre Donz ´ e (2010): Br each, A T oolbox for V erification and P arameter Synthesis of Hybrid Systems . In T ayssir T ouili, Byron Cook & Paul B. Jackson, editors: Computer Aided V erification, 22nd Int. Conf., CA V 2010 , LNCS 6174, Springer , pp. 167–170, doi:10.1007/978-3-642-14295-6 17. [9] Alexandre Donz ´ e, Thomas Ferr ` ere & Oded Maler (2013): Efficient Robust Monitoring for STL . In Natasha Sharygina & Helmut V eith, editors: Computer Aided V erification - 25th Int. Conf., CA V 2013 , LNCS 8044, Springer , pp. 264–279, doi:10.1007/978-3-642-39799-8 19. [10] Alexandre Donz ´ e & Oded Maler (2010): Robust Satisfaction of T emporal Logic over Real-V alued Signals . In Krishnendu Chatterjee & Thomas A. Henzinger , editors: Formal Modeling and Analysis of Timed Systems - 8th Int. Conf., FORMA TS 2010 , LNCS 6246, Springer , pp. 92–106, doi:10.1007/978-3-642-15297-9 9. [11] T ommaso Dreossi, Thao Dang, Ale xandre Donz ´ e, James Kapinski, Xiaoqing Jin & Jyotirmoy V . Desh- mukh (2015): Efficient Guiding Strate gies for T esting of T emporal Properties of Hybrid Systems . In Klaus Hav elund, Gerard J. Holzmann & Rajeev Joshi, editors: N ASA Formal Methods - 7th Int. Symp., NFM 2015 , LNCS 9058, Springer , pp. 127–142, doi:10.1007/978-3-319-17524-9 10. [12] T ommaso Dreossi, Alexandre Donz ´ e & Sanjit A. Seshia (2017): Compositional F alsification of Cyber- Physical Systems with Mac hine Learning Components . In Clark Barrett, Misty Davies & T emesghen Kahsai, editors: N ASA Formal Methods - 9th Int. Symp., NFM 2017 , LNCS 10227, pp. 357–372, doi:10.1007/978- 3-319-57288-8 26. [13] Parasara Sridhar Duggirala, Sayan Mitra, Mahesh V iswanathan & Matthew Potok (2015): C2E2: A V erifi- cation T ool for Stateflow Models . In Christel Baier & Cesare Tinelli, editors: T ools and Algorithms for the Construction and Analysis of Systems - 21st Int. Conf., T ACAS 2015 , LNCS 9035, Springer, pp. 68–82, doi:10.1007/978-3-662-46681-0 5. [14] Georgios E. Fainek os & George J. Pappas (2009): Robustness of temporal logic specifications for continuous- time signals . Theor . Comput. Sci. 410(42), pp. 4262–4291, doi:10.1016/j.tcs.2009.06.021. 40 T ime-Staging Enhancement of Hybrid System Falsification [15] Bardh Hoxha, Houssam Abbas & Georgios E. Fainekos (2014): Benchmarks for T emporal Logic Require- ments for Automotive Systems . In Goran Frehse & Matthias Althoff, editors: 1st and 2nd Int. W orkshops on Applied veRification for Continuous and Hybrid Systems, ARCH@CPSW eek 2014 and 2105 , EPiC Series in Computing 34, EasyChair , pp. 25–30, doi:10.29007/xwrs. [16] Bardh Hoxha, Adel Dokhanchi & Geor gios E. Fainekos (2018): Mining parametric temporal logic pr operties in model-based design for cyber -physical systems . STTT 20(1), pp. 79–93, doi:10.1007/s10009-017-0447-4. [17] Xiaoqing Jin, Jyotirmoy V . Deshmukh, James Kapinski, Koichi Ueda & Kenneth R. Butts (2014): P owertrain contr ol verification benchmark . In Martin Fr ¨ anzle & John L ygeros, editors: 17th International Conference on Hybrid Systems: Computation and Control (part of CPS W eek), HSCC’14, Berlin, Germany , April 15-17, 2014 , A CM, pp. 253–262, doi:10.1145/2562059.2562140. [18] James Kapinski, Jyotirmoy V Deshmukh, Xiaoqing Jin, Hisahiro Ito & K en Butts (2016): Simulation- based approac hes for verification of embedded contr ol systems: an overview of traditional and ad- vanced modeling, testing, and verification techniques . IEEE Control Systems 36(6), pp. 45–64, doi:10.1109/MCS.2016.2602089. [19] Eric S. Kim, Murat Arcak & Sanjit A. Seshia (2016): Dir ected Specifications and Assumption Mining for Monotone Dynamical Systems . In Alessandro Abate & Georgios E. Fainekos, editors: Proceedings of the 19th International Conference on Hybrid Systems: Computation and Control, HSCC 2016, V ienna, Austria, April 12-14, 2016 , A CM, pp. 21–30, doi:10.1145/2883817.2883833. [20] Jan Kur ´ atko & Stefan Ratschan (2014): Combined Global and Local Searc h for the F alsification of Hybrid Systems . In Axel Legay & Marius Bozga, editors: Formal Modeling and Analysis of T imed Systems - 12th Int. Conf., FORMA TS 2014 , LNCS 8711, Springer , pp. 146–160, doi:10.1007/978-3-319-10512-3 11. [21] Oded Maler & Dejan Nickovic (2004): Monitoring T emporal Pr operties of Continuous Signals . In Y assine Lakhnech & Sergio Y ovine, editors: Formal T echniques, Modelling and Analysis of Timed and Fault-T olerant Systems, Joint Int. Confs. on Formal Modelling and Analysis of Timed Systems, FORMA TS 2004 and Formal T echniques in Real-T ime and Fault-T olerant Systems, FTR TFT 2004 , LNCS 3253, Springer , pp. 152–166, doi:10.1007/978-3-540-30206-3 12. [22] Dogan Ulus, Thomas Ferr ` ere, Eugene Asarin & Oded Maler (2016): Online T imed P attern Matching Us- ing Derivatives . In Marsha Chechik & Jean-Franc ¸ ois Raskin, editors: T ools and Algorithms for the Con- struction and Analysis of Systems - 22nd Int. Conf., T A CAS 2016 , LNCS 9636, Springer, pp. 736–751, doi:10.1007/978-3-662-49674-9 47. [23] David W olpert & W illiam G. Macready (1997): No fr ee lunch theorems for optimization . IEEE T rans. Evolutionary Computation 1(1), pp. 67–82, doi:10.1109/4235.585893. [24] Aditya Zutshi, Jyotirmoy V . Deshmukh, Sriram Sankaranarayanan & James Kapinski (2014): Multiple shoot- ing, CEGAR-based falsification for hybrid systems . In T ulika Mitra & Jan Reineke, editors: 2014 Interna- tional Conference on Embedded Software, EMSOFT 2014, New Delhi, India, October 12-17, 2014 , ACM, pp. 5:1–5:10, doi:10.1145/2656045.2656061. G. Ernst, I. Hasuo, Z. Zhang & S. Sedwards 41 A STL Semantics f or Time-Bounded Signals Definition A.1 (robust semantics for time-bounded signals) . Let T ∈ R > 0 , w : [ 0 , T ] → R N be a time- bounded signal, and ϕ be an STL formula. W e define the r obustness J w , ϕ K T ∈ R ∪ { ∞ , − ∞ } of w with respect to ϕ as follows, by induction. Here the superscript T is an annotation that designates the time horizon. J w , f ( x 1 , . . . , x n ) > 0 K T : = f  w ( 0 )( x 1 ) , . . . , w ( 0 )( x n )  J w , ⊥ K T : = − ∞ J w , ¬ ϕ K T : = − J w , ϕ K T J w , ϕ 1 ∧ ϕ 2 K T : = J w , ϕ 1 K T u J w , ϕ 2 K T J w , ϕ 1 U I ϕ 2 K T : = G t ∈ I ∩ [ 0 , T ]  J w t , ϕ 2 K T − t u l t 0 ∈ [ 0 , t ) J w t 0 , ϕ 1 K T − t 0  The Boolean semantics | = , found e.g. in [10], allows a similar adaptation to time-bounded signals, too. B Brzozowski Deri vativ e of Flat STL F ormulas In the time-staged falsification procedure we often encounter the following situation: an STL formula ϕ and a signal v : [ 0 , T ] → R N are fixed; and we hav e to compute robustness J v · v 0 , ϕ K for a number of dif ferent signals v 0 : [ 0 , T 0 ] → R N . T o aid such computation, a natural idea is to use a syntactic construct ∂ v ϕ of (Brzozowski) derivative . It should be compatible with robust semantics in the sense that J v · v 0 , ϕ K = J v 0 , ∂ v ϕ K , reducing the computation of the LHS to that of the RHS. Similar use of deri vati ves is found e.g. in [22]. The settings are dif ferent, though: Boolean semantics is used in [22] while we use quantitativ e robust semantics. In fact, the definition of deriv ati ves in this section focuses on flat formulas (i.e. free from nested modalities). This restriction is mandated by the quantitati ve semantics, as our proof later suggests. Anyway , the definitions and results in this section are ne w to the best of our knowledge. W e need the following e xtension of STL syntax. Definition B.1 (extended STL ) . W e extend the syntax of STL (Def. 3.4) by atomic propositions c r for each r ∈ R . The robust semantics in Def. 3.5 (and that in Def. A.1 in Appendix A) is extended accordingly: J w , c r K : = r . Intuiti vely c r is an atomic proposition that constantly returns the robustness v alue r . Definition B.2 (deri vati ve) . Let T ∈ R > 0 , and v : [ 0 , T ] → R N be a time-bounded signal. For each extended STL formula ϕ , we define its derivative ∂ v ϕ by v by the follo wing induction. ∂ v  f (  x ) > 0  : ≡ c J v , f (  x ) > 0 K ∂ v c r : ≡ c r ∂ v ⊥ : ≡ ⊥ ∂ v ( ¬ ϕ ) : ≡ ¬ ∂ v ϕ ∂ v ( ϕ 1 ∧ ϕ 2 ) : ≡ ( ∂ v ϕ 1 ) ∧ ( ∂ v ϕ 2 ) ∂ v ( ϕ 1 U I ϕ 2 ) : ≡ c J v , ϕ 1 U I ϕ 2 K ∨  ( c J v , 2 ϕ 1 K ∧ ϕ 1 ) U I − T ϕ 2  Here the interval I − T is obtained from I by shifting both of its endpoints earlier by T , such as [ a , b ] − T = [ a − T , b − T ] . Definition B.3 (flat STL formula) . An STL formula ϕ is flat if it does not have nested temporal modal operators. This means: if ϕ 1 U I ϕ 2 is a subformula of ϕ , then neither ϕ 1 nor ϕ 2 contains U . Proposition B.4. Let T ∈ R > 0 , v : [ 0 , T ] → R N be a signal, and ϕ be a flat STL formula. W e have, for each T 0 ∈ R > 0 and v 0 : [ 0 , T 0 ] → R N , J v 0 , ∂ v ϕ K = J v · v 0 , ϕ K . 42 T ime-Staging Enhancement of Hybrid System Falsification Pr oof. By induction on the construction of ϕ . Most equalities belo w follow from the definition of ∂ and that of J K . J v 0 , ∂ v  f (  x ) > 0  K = J v 0 , c J v , f (  x ) > 0 K K = J v , f (  x ) > 0 K = J v · v 0 , f (  x ) > 0 K J v 0 , ∂ v c r K = J v 0 , c r K = r = J v · v 0 , c r K J v 0 , ∂ v ⊥ K = J v 0 , ⊥ K = − ∞ = J v · v 0 , ⊥ K J v 0 , ∂ v ( ¬ ϕ ) K = J v 0 , ¬ ∂ v ϕ K = − J v 0 , ∂ v ϕ K I.H. = − J v · v 0 , ϕ K = J v · v 0 , ¬ ϕ K J v 0 , ∂ v ( ϕ 1 ∧ ϕ 2 ) K = J v 0 , ( ∂ v ϕ 1 ) ∧ ( ∂ v ϕ 2 ) K = J v 0 , ∂ v ϕ 1 K u J v 0 , ∂ v ϕ 2 K I.H. = J v · v 0 , ϕ 1 K u J v · v 0 , ϕ 2 K = J v · v 0 , ϕ 1 ∧ ϕ 2 K Here is a nontri vial case. J v 0 , ∂ v ( ϕ 1 U I ϕ 2 ) K = J v 0 , c J v , ϕ 1 U I ϕ 2 K K t J v 0 ,  ( c J v , 2 ϕ 1 K ∧ ϕ 1 ) U I − T ϕ 2  K = J v , ϕ 1 U I ϕ 2 K t G t ∈ ( I − T ) ∩ [ 0 , T 0 ]  J v 0 t , ϕ 2 K u J v , 2 ϕ 1 K u l t 0 ∈ [ 0 , t ) J v 0 t 0 , ϕ 1 K  = G t ∈ I ∩ [ 0 , T ]  J v t , ϕ 2 K u l t 0 ∈ [ 0 , t ) J v t 0 , ϕ 1 K  t G t ∈ ( I − T ) ∩ [ 0 , T 0 ]  J v 0 t , ϕ 2 K u  l t 0 ∈ [ 0 , T ] J v t 0 , ϕ 1 K  u l t 0 ∈ [ T , T + t ) J ( v · v 0 ) t 0 , ϕ 1 K  ( ∗ ) = G t ∈ I ∩ [ 0 , T ]  J ( v · v 0 ) t , ϕ 2 K u l t 0 ∈ [ 0 , t ) J ( v · v 0 ) t 0 , ϕ 1 K  t G t 00 ∈ I ∩ [ T , T + T 0 ]  J ( v · v 0 ) t 00 , ϕ 2 K u  l t 0 ∈ [ 0 , T ] J ( v · v 0 ) t 0 , ϕ 1 K  u l t 0 ∈ [ T , t 00 ) J ( v · v 0 ) t 0 , ϕ 1 K  = G t ∈ I ∩ [ 0 , T ]  J ( v · v 0 ) t , ϕ 2 K u l t 0 ∈ [ 0 , t ) J ( v · v 0 ) t 0 , ϕ 1 K  t G t 00 ∈ I ∩ [ T , T + T 0 ]  J ( v · v 0 ) t 00 , ϕ 2 K u l t 0 ∈ [ 0 , t 00 ) J ( v · v 0 ) t 0 , ϕ 1 K  = G t ∈ I ∩ [ 0 , T + T 00 ]  J ( v · v 0 ) t , ϕ 2 K u l t 0 ∈ [ 0 , t ) J ( v · v 0 ) t 0 , ϕ 1 K  = J v · v 0 , ϕ 1 U I ϕ 2 K In ( ∗ ) we used the follo wing facts. Firstly , for a formula ψ without temporal operators, we ha ve J v , ψ K = J v · v 0 , ψ K . Secondly , if v ’ s domain is [ 0 , T ] and t ∈ [ 0 , T ] , then v t · v 0 = ( v · v 0 ) t . Note that the flatness assumption on ϕ is crucially used in the proof step. Modifying Def. 4.2 in order to accommodate nested modalities seems hard, after analyzing the proof step ( ∗ ) . G. Ernst, I. Hasuo, Z. Zhang & S. Sedwards 43 C Omitted Pr oofs C.1 Proof of Pr op. 5.4 Pr oof. By the definitions we ha ve, for each input signal u : [ 0 , T ] → R M , J M ( u ) , 2 ( x < c ) K = l t ∈ [ 0 , T ] c − M ( u )( t )( x ) . Therefore the assumption J M ( u 1 ) , ϕ K ≤ J M ( u 0 1 ) , ϕ K expands to l t ∈ [ 0 , T 1 ] c − M ( u 1 )( t )( x ) ≤ l t ∈ [ 0 , T 1 ] c − M ( u 0 1 )( t )( x ) . (4) The first infimum in the above is taken ov er a compact domain [ 0 , T 1 ] ; therefore there exists T ∈ [ 0 , T 1 ] that achie ves the infimum. Let T be such a real number . The following is ob vious. l t ∈ [ 0 , T 1 ] c − M ( u 1 )( t )( x ) = l t ∈ [ 0 , T ] c − M ( u 1 )( t )( x ) = c − M ( u 1 )( T )( x ) ≤ l t ∈ [ 0 , T 1 ] c − M ( u 0 1 )( t )( x ) ≤ l t ∈ [ 0 , T ] c − M ( u 0 1 )( t )( x ) . (5) Another immediate consequence, deri ved using the causality of M (Def. 3.2), is c − M ( u 1 | [ 0 , T ] )( T )( x ) ≤ c − M ( u 0 1 | [ 0 , T ] )( T )( x ) . (6) Our goal is to sho w J M ( u 1 | [ 0 , T ] · u 2 ) , ϕ K ≤ J M ( u 0 1 | [ 0 , T ] · u 2 ) , ϕ K . J M ( u 0 1 | [ 0 , T ] · u 2 ) , 2 ( x < c ) K = l t ∈ [ 0 , T + T 2 ] c − M ( u 0 1 | [ 0 , T ] · u 2 )( t )( x ) = l t ∈ [ 0 , T ] c − M ( u 0 1 )( t )( x ) u l t ∈ ( T , T + T 2 ] c − M ( u 0 1 | [ 0 , T ] · u 2 | [ 0 , t − T ] )( t )( x ) ( ∗ ) ≥ l t ∈ [ 0 , T ] c − M ( u 1 )( t )( x ) u l t ∈ ( T , T + T 2 ] c − M ( u 0 1 | [ 0 , T ] · u 2 | [ 0 , t − T ] )( t )( x ) by (4) ≥ l t ∈ [ 0 , T ] c − M ( u 1 )( t )( x ) u l t ∈ ( T , T + T 2 ] c − M ( u 1 | [ 0 , T ] · u 2 | [ 0 , t − T ] )( t )( x ) ( † ) = · · · = J M ( u 1 | [ 0 , T ] · u 2 ) , 2 ( x < c ) K . In the above we heavily used the causality of M (Def. 3.2). For e xample, in the step ( ∗ ) above, causality is used in deri ving M ( u 0 1 | [ 0 , T ] · u 2 )( t ) = M ( u 0 1 )( t ) . In the step ( † ) we applied the monotonicity of M to the signals u 1 | [ 0 , T ] , u 0 1 | [ 0 , T ] and u 2 | [ 0 , t − T ] . Note that (6) allows to do so.