Protecting Actuators in Safety-Critical IoT Systems from Control Spoofing Attacks

Protecting Actuators in Safety-Critical Io T Systems from Contr ol Spoofing Aacks Monowar Hasan Dept. of Computer Science, University of Illinois mhasan11@illinois.edu Sibin Mohan Dept. of Computer Science, University of Illinois sibin@illinois.edu ABSTRA CT In this paper , w e propose a framework called Contego- TEE to secure Internet-of- Things (Io T) edge devices with timing requirements from control spoong attacks where an adversary sends malicious control signals to the actuators. W e use a trusted computing base available in commodity processors (such as ARM T rustZone) and propose an invariant checking mechanism to ensure the security and safety of the physical system. A working prototype of Contego- TEE was developed using embedde d Linux kernel. W e demonstrate the feasibility of our approach for a robotic vehicle running on an ARM-based platform. 1 IN TRODUCTION T oday’s embedded and cyb er-physical systems are ubiquitous. A large number of critical cyber-physical systems ( e.g., autonomous cars, drones, manufacturing systems, power grids, industrial control systems, etc. ) have real-time (RT) properties ( e.g., strict timing and safety requirements). The curr ent trend is to connect embedded RT devices to the Internet ( e.g., remote surveillance over wired/wireless network, connected vehicles through cellular wireless networks, etc. ) and this gives rise to the real-time Internet-of- Things (RT - Io T) [ 1 ]. RT -Io T systems ar e intended to provide better user experience through stronger connectivity and better use of next- generation embedde d devices, albeit with safety-critical properties. RT -Io T systems are also increasingly becoming targets for cyber- attacks. A number of high-prole attacks on RT -Io T systems, e.g., denial-of-service (DoS) attacks mounted from Io T devices [ 2 ], Stuxnet [ 3 ], attack demonstrations by researchers on medical devices [ 4 ] and automobiles [ 5 ] have shown that the threat is real. Successful cyb er attacks against such systems could lead to problems more serious than just loss of data or availability because of their critical nature [ 1 ]. Enabling security in RT -Io T , however , is often more challenging than generic IoT due to additional timing/safety constraints imposed by RT -enabled systems. Since RT -Io T systems ar e largely based on sensing and actuation, any false/spoofed command to the actuators can disrupt the normal operation of the physical plant. Commonly used open-source RT - Io T development stacks (such as Linux) do not provide explicit control over actuation signals. For instance , if the application task obtains permission (say , root or other privileged user access) to the peripheral interface ( e.g., I2C [ 6 ]), it is possible to send arbitrary signals to the actuators. Let us consider an industrial r obotic arm (running an embedde d variant of Linux in an ARM Cortex- A53 platform [ 7 ]) that periodically opens and closes the grip to drop o and pick up objects in an assembly line. The mov ement of the grip is contr olled by a servo. W e use an open-source implementation [ 8 ] for this robotic arm where each operation is represented by a pulse value x (where x = 577 for grip_open() and x = 420 for grip_close() ) 0 100 200 Contego-TEE disabled 0 100 200 300 400 0 100 200 Contego-TEE enabled 0.0 0.2 0.4 0.6 0.8 1.0 Access Sequence Number 0.0 0.2 0.4 0.6 0.8 1.0 Pulse Value Figure 1: Demonstration of a control spoong attack on a robotic control arm running emb edded Linux. and each pulse sends the following four 1 byte command se quences to the ser vo registers: 0 & 0xFF , 0 > > 8 , x & 0xFF , x > > 8 . An example of a sp oong attack for this control arm is presented in Fig. 1 (x-axis is the ser vo access sequence number and y-axis is the corresponding pulse value). Without any actuation command validation, it is possible to send arbitrar y (high) pulses to the servo registers that prevents the grip from picking up/dropping objects (showing in the shaded region, see the top gure) that is not otherwise p ossible when our scheme (called Contego- TEE, see §3 for details) is enabled (b ottom gure). Our proposed framework, Contego- TEE, prev ents the sending of malicious/undesired commands to physical actuators and ensures safety of the system. Specically , we use the concept of Trusted Execution Envir onments (TEEs) [ 9 ] available in commodity processors ( e.g., ARM TrustZone [ 10 ], Intel SGX [ 11 ]) to ensure that our protection mechanisms can not b e disabled even if the host OS is compromised. W e develop a rule-based invariant checking and access control mechanism as w ell as design-time (schedulability ) tests to ensure timing and safety requirements of the system. Contego- TEE specically designed for legacy systems developed with Commodity-O- The-Shelf (COTS) components and does not require any mo dication to the application code/logic . In this paper we present the following contributions. • A new framework called Contego- TEE to secure COTS-based RT -Io T systems against attacks that spoof control signals (§3.1). • A runtime, rule-base d invariant checking mechanism as well as design-time analysis to ensure security ( and safety) of the physical plant (§3.2). • An open source implementation and patch to the (embedded) Linux kernel that includes the Contego- TEE functionality (§4.1). W e use ARM TrustZone as a TEE and implemement our solution in an ARM Cortex- A53 board ( i.e., Raspb erry Pi [ 7 ]). W e also 1 demonstrate the viability of our approach using a COTS rover platform (§4.2). 2 SYSTEM AND AD VERSARY MODEL In the following, we rst present background on RT -Io T systems and give an overview of a TEE-based architecture ( e.g., ARM TrustZone). W e then introduce our system model (§2.2) and describe our assumptions on the adversarial capabilities (§2.3). 2.1 Preliminaries 2.1.1 RT -Io T Systems: RT -IoT systems comprise Io T e dge devices with RT capabilities. RT systems are those that, apart from a requirement for functional correctness, require that temp oral properties b e met as well. These temporal properties are often presented in the form of deadlines . The usefulness of results (say the performance of the actuators) produced by the system drops after the passage of a deadline. Some of the common properties and assumptions related to RT systems include [ 1 ]: (i) periodic/sp oradic execution of set of tasks 1 , (ii) strict timing and safety requirements, (iii) well-characterized execution time ( e.g., execution times in the worst-case are known for all loops), (iv) limited resources ( e.g., memory , processing power and energy). RT -Io T systems are often designed base d on the periodic task model [ 12 ], i.e., each task τ i is characterized by a tuple: ( C i , T i , D i ) where C i is the W orst Case Execution Time (WCET), T i is the perio d ( e.g., inter-invocation time) and D i is the deadline. Schedulability tests [ 13 , 14 ] are used to determine if all tasks in the system meet their respective deadlines. If they do, then the taskset is deemed to be ‘schedulable’ and the system is considered safe . 2.1.2 TEE and ARM T rustZone: TEE is a set of hardware and software-based security extensions where the processors maintain a separate d subsystem in addition to the traditional OS components. TEE te chnology has b een implemented on commercial secure hardware such as ARM T rustZone [ 10 ] and Intel SGX [ 11 ]. In this work we consider TrustZone as the building blo ck of Contego- TEE due to wide acceptability of ARM processors for embedded Io T systems – although our framework can be ported into other TEE platforms without loss of generality . ARM partnered with GlobalPlatform and has dened new TEE APIs [ 15 ]. TrustZone encompasses the following major features [ 16 ]: (a) safe and secure boot (to ensure all software components are in a trusted state b efore launching the OS); (b) isolated execution of critical applications ( i.e., in a se cure enclave) and (c) protection for trusted applications data (in terms of integrity and condentiality). ARM TrustZone contains two dierent privilege blocks: (i) Normal W orld (N W) and (ii) Secure W orld (SW). The N W is the untrusted environment running a commodity untrusted OS where SW is a protected computing blo ck that only runs privileged instructions. SW in TrustZone denes the memor y regions that can only be accessed by privileged instructions and the code that runs in the SW has higher privilege than the N W. Hardware logic ensures that the resources in the secure world can not be accessed from the normal world ( e.g., if the co de running in the N W tries to access protected memory regions, TrustZone thr ows a hardware 1 The ‘task’ in RT -IoT systems can trivially be mapped with the concept of process or thread in general-purpose OSes. RT Ta s k s Ope ra ti n g S y s te m A c tu a ti o n Sens ing P h y s ic a l P la n t Figure 2: High-level schematic of a RT control system. exception). The SW instructions are triggered when a specic ag in the processor e.g., Non-secure (NS) bit in the Secure Conguration Register (SCR) is not set. These two worlds bridge via a software module referred to as Secure Monitor . The context switch between the NW and SW is p erformed through a Secure Monitor Call (SMC). In this work we use the T rustZone functionality to prevent the malicious commands from being sent to the actuators (See §3 for details). W e now pr esent our system and adversar y model. 2.2 System Model In Fig. 2 we present a high-level illustration of a RT control system. W e consider a set of periodic RT control tasks Γ = { τ i , τ 2 , · · · , τ N } that execute on single processor 2 . The physical system consists of a set of M actuators ( e.g., ser vo, motor , buzzer ): { π 1 , π 2 , · · · , π M } . RT tasks periodically issue commands to the actuators to control physical entities ( e .g., wheel, propeller , alarm, r ob otic grip, etc. ). W e assume that each task is allow ed to access a subset of peripherals. W e r epresent this access permission as an N × M Boolean matrix A = [ a i k ] where a i k = 1 represents task τ i can send commands to actuator π k . W e also assume that the RT tasks nish computation before their deadline, e.g., the tasks are schedulable 3 . 2.3 Adversary Model W e consider the following adversarial capabilities: (a) Integrity Violation – an adversary may insert a malicious task (that respects the RT guarante es) and/or modify exiting control logic to manipulate actuator commands and control system b ehavior in undesirable ways; (b) Denial of Service (DoS) – the attacker may take control of the RT task(s) and destabilize the physical plant e.g., by sending multiple control requests in a burst that may result in a malfunctioning actuator , or worse, damage the actual hardware/actuator and ev en threaten the safety of the system. The attacker can gain privilege d ( e.g., root) access to perform adversarial actions ( e.g., to spo of control signals). W e do not make any assumptions as to how the compromised tasks enter the device. For instance, bad software engine ering practices leave vulnerabilities in the systems [ 17 ]. When the system is developed using a multi-vendor model [ 18 ] (where its components are manufactured and integrated by dierent vendors) a malicious code logic may be injected (say by a less-truste d vendor ) during deployment. The adversary may also induce end-users to download the modied source code, say by using social engineering 2 Since majority of the RT -Io T e dge devices still use single core chips due to simplicity and determinism. 3 In the Appendix we present formal expressions to determine schedulability of the tasks. 2 User space Kernel space RT Tasks Actuation TEE - enabled SoC Normal World (NW) Secure World (SW) Enclave Client SMC Invariant Checker Allow/Deny Actuation Peripheral Subsystem I2C SPI … 6 7 8 2 5 Physical Plant … Actuators Sensing 1 State Observation Access Matrix 4 … Sensors 3 Read sensors Normal Wo rld (NW) Secure World (SW) /* task initialization */ while ( true ): /* task code */ . . . /* send actuation request*/ actuation_request() /* rest of task code */ . . wait_for_next_period() invariant_checking () Switch to SW Return Context Figure 3: O verview of Contego- TEE system design (left) and high-level control ow of RT tasks in Contego- TEE (right). tactics [ 19 ]. W e also assume that the attackers do not have any physical access ( e.g., they can not physically control/turn o/damage the actuators). 3 A CT U A TION MONI TORING FRAMEWORK In the following we rst introduce the Contego- TEE framework (§3.1). W e then present me chanisms to detect any abnormal control commands issued by (rogue) tasks and analyze schedulability conditions that ensures our (invariant) che cking techniques can b e enforced at runtime (§3.2). 3.1 Overview and Architecture As mentioned earlier , to secure RT -IoT platforms we propose a TEE-based architecture that monitors actuation commands send to the physical entities. At the high-level, our design is based on the Simplex architecture [ 20 ]. Researchers use Simplex-based architecture for time-critical cyber-physical systems to provide fault-tolerance [ 21 , 22 ] and recently , security [ 23 – 25 ]. A Simplex system consists of the following main comp onents: (a) under normal operating conditions a High-Performance (complex) Controller actuates the physical plant (such a contr oller may be unveriable due to its complexity , yet it must actuate a safety-critical system); (b) if, during operation the system state becomes unstable ( e.g., it is in danger of violating a safety condition), a Safety Controller takes over and (c) the exact switching behavior is implemente d by a Decision Module that de cides which controller output will driv e the plant. In our context, we use a trusted ( and veried) computing module (this is analogous to the safety controller) execute d in a secure enclave ( viz., SW) and ensures that even if the (potentially untrusted) N W RT tasks are compromised, an adversary can not send false signals to the physical actuators. In Fig. 3 we illustrate the high-level overview of Contego- TEE design and control ow of the RT tasks. Contego- TEE contains the following essential components: (a) a TEE-enabled SoC (System- on-Chip) such as those supported by ARM T rustZone [ 10 ] (block 2 ○ in the gure); (b) an Enclave Client ( block 7 ○ ) that is used to communicate between N W and SW and (c) an Invariant Che cker (block 8 ○ ) that is used to monitor (and validate) the actuation commands. The physical plant ( 1 ○ ) is connected with sensors ( 2 ○ ) and actuators ( 3 ○ ) and contr olle d by the (potentially vulnerable) RT tasks ( 5 ○ ). RT tasks execute in untrusted NW and issue system calls ( e.g., read() , write() , ioctl() ) to access the sensors/actuators using spe cic interface such as I2C [ 6 ] and/or SPI [ 26 ]. Contego- TEE ensures that RT tasks cannot dir ectly send any actuation commands ( e.g., it breaks the bridge b etween 6 ○ , 2 ○ and 4 ○ ). W e do this by placing a dispatcher ( e.g., enclave client) b etween the peripheral subsystem and actual hardware . As a result, before issuing any command to the physical actuators, it will be validated by our trusted application ( e.g., invariant checker) running inside the secure enclave ( i.e., in the SW). In particular , when a RT task τ i sends an actuation command x t i k to any peripheral π k at time t , enclave client traps those r equest and forwards the command to the invariant checker using SMC. Dep ending on the access permission matrix A and current system state S ( t ) , invariant checker then decides whether the given command x t i k can be issued to the actuator π k (refer to §3.2 for details). In Contego- TEE, b oth the enclave client and invariant checker operate in the privileged mode ( e.g., kernel space) so that it can directly control low-level hardware. By using the enclave client (to invoke context switching) and invariant checking mechanisms, Contego-TEE ensures that ev en if the NW RT tasks are compromised, an adversary can not send false signals to the actuators. W e note that unlike NW RT tasks that may perform other computation, the invariant che cker contains a small, veried, code blocks that is used to monitor only actuation r equests. W e also note that Contego- TEE does not require any application- level modications , e.g., developers can e xecute unmodied, existing legacy RT tasks, using our Contego- TEE enabled OS-kernel (refer to §4.1 for implementation/porting details). 3.2 Invariant Checking and Timing Analysis 3.2.1 Invariant Checking: In order to validate each actuation command invoked by the RT tasks, Contego- TEE performs various actions. One obvious access control me chanism is to ensure that a task τ i can access a given actuator π k only if the task has the required permission ( e.g., a i k = 1 ). Contego- TEE therefore denies all the actuation commands from tasks if the corresponding access ag is zero . Howev er , if the attacker can compromise a task with legitimate access (to a given set of actuators) then the (victim) 3 T able 1: Applicability of Contego- TEE for V arious RT -Io T P latforms Platform Application Domain Actuators Possible Invariant Conditions * Response Remarks W ater/air monitoring system Home/industrial automation Buzzer , display (a) Send high pulse to buzzer only if water-level is high/air quality abnormal/detect smoke; (b) do not display alert if the system state is normal IGNORE Ignore all commands that fail invariant checking Surveillance system Home/industrial automation Servo, buzzer (a) Trigger alarm only if there is an impact/object detected in camera; (b) rotate camera (using servos) only within allowable pan/tilt angle IGNORE Ignore all commands that fail invariant checking Infusion/syringe pump Health-care Motor , display (a) Drive the motor only to allowable positions/rates (b) display only the amount of uid infused ( e.g., obtained from motor encoders) IGNORE Ignore actuation when the task tries to infuse wrong amount of uid Robotic arm Manufacturing Servo, buzzer (a) Che ck the servo pulse sequences matches with the desired (design-time) sequence; (b) do not raise alarm if the pulse sequence is normal IGNORE, F AIL-SAFE If mismatch, use the predened sequence; ignore other pulses using rate-control rule Robotic vehicle (aerial/ground) Manufacturing, surveillance, agriculture Servo, motor (a) Che ck if the robot is following the mission; (b) allow only predened number of actuation commands per period IGNORE, F AIL-SAFE Ignore command using rate-control rule. If it deviates from the mission, use predened command and/or state- observations * W e omit mathematical expressions for readability . task may send arbitrary commands to the actuators. Therefor e in addition to checking access matrix A , Contego- TEE also performs checking of system invariants and monitors the numb er of actuation commands for a given time interval as we discuss below . State Invariant Checking: Invariant checking [ 27 ] is useful to detect control spoong attacks. For a given RT -Io T platform we do this by considering the availability of an invariant checking function CheckInv ( τ i , π k ) that predicts the actuation signal and only allows access if the output of the function matches that of the requested command. In particular , if a task τ i sends actuation command x t i k at time t to any peripheral π k and the task has the required permission ( i.e., a i k = 1 ), CheckInv ( τ i , π k ) rst obtains system state S ( t ) by obser ving a set of signals S i = { s 1 , s 2 , · · · , s L i } and decides whether x t i k is valid for curr ent state S ( t ) . For example, consider a warehouse water monitoring system where an alarm is triggered only if the water level of the tank (measured by the sensor s W L ) is higher that a predened threshold ( θ W L ) and/or the water temperature ( s W T ) is not in expected range ( i.e., [ θ t 1 W T , θ t 2 W T ] ). W e represent this as the following invariant rule: INV W :: ( s W L > θ W L ) ∨ ( s W T < [ θ t 1 W T , θ t 2 W T ]) → x = ON : x = OFF , e .g., Contego- TEE will only allow the sending of the high pulse ( i.e., x = ON ) to the alarm system (say a buzzer) only if the invariant conditions are satised. W e note that since Contego- TEE operates at the kernel- level, it can dir ectly access raw signals without any interaction of NW RT tasks or other (user space) libraries. Rate Control: Note that since RT systems are deterministic by design, the (worst-case) numb er of actuation requests can b e bounded at design time [ 28 ]. Therefore , if a task τ i tries to access actuator(s) more than e xp ected within a given time interval ( e.g., T i ), it may b e indication of a possible attack. In such cases Contego- TEE will limit subsequent access requests from τ i and prevent the sending of actuation commands to the hardwar e. W e enforce rate control using the following invariant rule: INV R C i k :: ∆ i k ( w ) < b θ i k → Che ckInv ( τ i , π k ) : ignore , i.e., Contego- TEE ignores further actuation commands if the numb er of requests ∆ i k (·) from any job of τ i within the (relative) time window w ∈ [ 0 , T i ] is exceeded a design-time threshold b θ i k . Such a rate control me chanism is specially useful to defend against DoS attacks where an attacker sends multiple actuation commands in a burst (say to quickly change the speed of whe els/propellers in robotic ground/aerial vehicles, abruptly move robotic arms, falsely toggles buzzers, etc. ) to disrupt normal operations of the system. 3.2.2 Response Mechanisms: When there exists a mismatch between output of the CheckInv (·) and the requested actuation commands, Contego- TEE makes use of the following strategies to keep the physical system safe. • IGNORE: this strategy pr events the execution of any actuation commands requested by RT tasks. Hence, actuators will not receive any signals from Contego- TEE and will continue to operate using the last known (uncompromised) commands. Contego- TEE will also ignore commands if the task makes multiple requests in a short time window ( e.g., by using rate control rule). • F AIL-SAFE: while the IGNORE strategy ensures that actuators will not get any abnormal signals, ignoring actuation commands (for a long time) may not b e acceptable for highly dynamic systems such as unmanned ground/aerial vehicles ( e.g., it may crash). Therefore, Contego- TEE also allows operation of a F AIL-SAFE mode, i.e., if it nds any mismatch, it ignor es the requests from RT tasks and sends the predetermine d (and/or based on the output of CheckInv (·) ) commands to make the system safe/operational. As an example, if there is a sudden change in the propeller sp eed of a U A V , the F AIL- SAFE strategy sets a safe , predened speed, based on the current state of the U A V . Depending on the target system, both of the above strategies may be required to keep the physical system operational. W e note that invariant checking and response mechanisms are application dependent. Contego-TEE provides exibility for the system engineers to develop appropriate mechanisms depending on the application requirements. In T able 1 we summarize possible invariant conditions and response mechanisms that are applicable for various RT -Io T platforms – ho wever , this is by no stretch meant to an exhaustive list. 3.2.3 Schedulability A nalysis: In order to p erform invariant checking and execute the response mechanisms at runtime, we 4 T able 2: Summary of the Implementation Platform Artifact Conguration Platform Broadcom BCM2837 (Raspberry Pi 3) CP U 1.2 GHz 64-bit ARM Cortex- A53 Memory 1 Gigabyte Operating System Linux (N W), OP- TEE (SW) Kernel version Linux kernel 4.16.56, OP- TEE core 3.4 Peripheral interface I2C Boot parameters dtparam=i2c_arm=on , dtparam=spi=on need to ensure that our framework should not cause delays and the timing requirements of RT tasks are satised ( e.g., they complete execution b efore deadline). W e therefore develop design-time schedulability tests that ensure the taskset is schedulable (refer to the Appendix for details). For instance, the RT task τ i is schedulable in Contego- TEE if the W orst Case Response Time (W CRT) R T E E i is less than deadline, i.e., R T E E i = C T E E i + I T E E i ≤ D i , where C T E E i is the task WCET (including the time for world switching and invariant checking) and I T E E i is the interference 4 from other tasks. The taskset Γ is referred to as schedulable if all the tasks are schedulable, viz., R T E E i ≤ D i , ∀ τ i ∈ Γ . 4 EV ALU A TION In this se ction we rst present the implementation details of Contego- TEE (§4.1) and then show the viability of our approach using a case-study on a robotic vehicle (§4.2). T able 2 summarizes the system congurations and implementation details. 4.1 System Implementation W e implemented a proof-of-concept prototype of Contego- TEE on Raspberr y Pi 3 (RPi3) Mo del B [ 7 ] (equipped with 1.2 GHz 64-bit ARMv8 CP U and 1 GB RAM). W e selecte d RPi3 as our implementation platform since (a) it supp orts ARM TrustZone and (b) previous research has shown feasibility of deploying multiple Io T -sp ecic applications on RPi3 [ 16 , 19 , 29 – 31 ]. W e developed Contego- TEE using the Open-Portable Trusted Execution Environment (OP- TEE) [ 32 ] software stack that uses GlobalP latform TEE APIs [ 15 ] to provide T rustZone functionality . OP- TEE pr ovides a minimal secure kernel (called OP- TEE core) that can be run in parallel with the N W OS ( e.g., Linux). In particular , we used Ubuntu 18.04 lesystem with a 64-bit Linux kernel (v ersion 4.16.56) as the NW OS and our invariant checker is running on OP-TEE secure kernel (version 3.4). The enclave client was statically built with the Linux kernel. In or der to implement the enclave client, we extended the Linux TEE interface ( /linux/drivers/tee/ ) and enable d SMC from Linux kernel space 5 . W e implemente d the invariant checker as an OP-TEE kernel-level trusted application 6 ( e.g., in /optee_os/core/arch/arm/pta/ ). In our current implementation Contego- TEE supports actuators that ar e controlled via the 4 In RT scheduling theory , the term ‘interference ’ refers to the amount of time (from release to deadline) the task τ i is ready but can not be scheduled due to execution of other tasks. 5 Since GlobalPlatform APIs only support SMC from user space. 6 This is known as PT A (Pseudo Trusted Application) in OP- TEE terminology . I2C interface. Sp ecically , we modied the built-in structure i2cdev_fops ( e.g., in /linux/drivers/i2c/i2c-dev.c ) with our enclave client functions that is then switch the control to the invariant checker ( e.g., by using SMC). Our implementation code is available in a public repository [33]. 4.2 Case-Study: Robotic V ehicle W e implemented Contego- TEE in a COTS rover (named GoPiGo2, manufactured by Dexter Industries [ 34 ]) that can be used in multiple Io T -spe cic applications such as remote surveillance, agricultur e, manufacturing, etc. [ 35 ]. The rover is equipped with two optical encoders that are connected to the motors ( e.g., actuator in this setup): it can turn left by switching o the right encoder and vice- versa. The detailed specications of the rover are available on the vendor website [34]. 4.2.1 Results. W e rst demonstrate how Contego- TEE can be used to protect such systems from actuation attacks and then measure the performance overheads. Security A nalysis: For the following experiments, we conducted a line following mission where the robot ste ered from an initial location to a target lo cation by following a line. The controller task was running as a N W Linux application and execute d vendor-provided PID (Proportional–Integral–Derivative) closed- loop control [ 36 ] to track the planned path using the data received from sensors. The rover used the following commands: fwd () , l () , rht () , st _ sp ( δ ) for navigating the rover forward/left/right and set the speed to δ , respectively , where each command sent a 5-byte value to the actuator registers ( e.g., wheel enco ders/motors) using the I2C interface. For this mission we dened the following three invariant conditions 7 that were use d to monitor control signals ( e.g., cmd ): INV 1 :: s L F < − θ → cmd = st _ sp ( δ 1 ) ∧ rht () , INV 2 :: s L F > θ → cmd = st _ sp ( δ 1 ) ∧ l () and INV 3 :: s L F ∈ [− θ , θ ] → cmd = st _ sp ( δ 2 ) ∧ fwd () where s L F was the readings from the sensor , θ = 2500 was a vendor-provided threshold ( e.g., to follow the line) and δ 1 , δ 2 ∈ [ 0 , 255 ] were used to set the speed of the rover . W e show the case where the access ag is set ( e.g., a i k = 1 ) since Contego- TEE will trivially deny r equests if the corresponding ag is zero . In our experiments we used both the F AIL-SAFE and IGNORE (to enforce rate control) strategies. For each actuation signal, our invariant checker matches with the desired signal and choose the appropriate strategy as we present in the following. In Fig. 4a we illustrate our invariant checking mechanism with F AIL-SAFE strategy . The x-axis of the gur e shows the time ( e .g., count of the controller job) and the y-axis is the total distance travelled by the rov er ( e.g., readings from the encoders). In order to demonstrate malicious behavior , we followed a strategy similar to that considered in prior work [24, 25, 35, 37]. In particular , during program execution, we injected a logic bomb (during the shaded region in Fig. 4a) and sent erroneous commands to the controller . In this case, during the control spoong attack, the rover deviated from the mission ( e.g., PID control loop) and falsely sent commands to turn o one of the motors. As a result, when Contego- TEE was not active, the rover was not following the line and the encoder 7 W e manually inspected the vendor-provided control code and translated them into invariant conditions. 5 0 50 100 150 Time (Job Index) 0 100 200 300 400 500 Total Distance Traveled (cm) Vanilla Contego-TEE Control Spoofing (a) 0 50 100 150 Time (Job Index) 0 100 200 300 400 500 Total Distance Traveled (cm) Vanilla Contego-TEE DoS Attack (b) Figure 4: Illustration of Contego- TEE under (a) control spoong and (b) DoS attacks. Contego- TEE prevents the sending of malicious commands to the motors and ensures that the rover moves at a steady spe ed.                   (a)                   (b) Figure 5: Runtime of rover control tasks with and without Contego- TEE: (a) for 99-th percentile and (b) worst-case. Contego- TEE increases the execution time by upto 43.47 ms (worst-case) and 23.31 ms (99th-percentile). readings ( i.e., traversed distance) remained same (see the maroon line in the gure). W e next executed the same code with Contego- TEE enabled (green cur ve in the gure). In this case, when each control command was issued, our checker followed the invariant conditions ( e.g., INV i , 1 ≤ i ≤ 3 ) and sent desired commands to the motors (and hence the rov er was moving as expecte d). W e next show the eect of our rate control mechanism (Fig. 4b). In this experiment, when the DoS logic bomb was triggered ( shaded region in the gure) it sent multiple requests to increase the speed of the rover . When Contego- TEE was not enabled, this caused the rover to move faster and hence there was a rapid increase in the encoder readings ( e.g., maroon line, shaded region in the gure). In contrast, when Contego- TEE was active (green line), it disallowed multiple increase speed requests per perio d ( e.g., according to IGNORE strategy) and hence the rov er followed the line with a steady speed. Overhead Analysis: T o measure the runtime overheads we conducted experiments with the vendor-provided control tasks [ 34 ] as a benchmark (Fig. 5). In this setup our invariant checker was following a rate control policy and ignored more that one actuation request per period ( 200 ms). The x-axis of Fig. 5 shows the control tasks and y-axis represents execution time (a) when Contego-TEE is not enabled (dark bar ) and (b) with Contego- TEE enabled ( light bar). W e present the timing results for 99th p ercentile (Fig. 5a) and worst-case (Fig. 5b). The timing values were measur e d using the Linux clock_gettime() system call with CLOCK_MONOTONIC clock parameter and we present data from 10 , 000 trials. As we see from the gure, Contego- TEE increases the execution time – this is expecte d due to (world) context switching as well for invariant checking. From our experiments we found that Contego- TEE increases execution times by (i) 34.11 to 43.47 ms (w orst-case), (ii) 22.87 to 23.31 ms (99-th percentile) and (iii) 19.55 to 19.60 ms (average-case) for the various contr ol tasks and hence can be used with 15 Hz (or slow er) controllers (for this setup). This e xtra overhead results in incr eased security and we expect this could b e acceptable for various RT -Io T platforms. 5 RELA TED WORK Enhancing se curity in time-critical cyber-physical systems is an active research area (see the related sur vey [ 1 ]). Perhaps the closest line of work to ours is PROTC [ 31 ] where a monitor in the SW enforces se cure access control policy (given by the control center) for some p eripherals of the drone and ensures that only authorized applications can access certain peripherals. Unlike our scheme, PROTC is limited for specic applications ( e.g., aerial robotic vehicles) and requires a centralized control center to validate/enforce security policies. In early work we proposed mechanisms to secure legacy time-critical systems [ 38 – 40 ]. Researchers also proposed anomaly detection approaches for robotic vehicles [ 35 , 37 , 41 ]. Howev er these (prior) approaches do not provide any response me chanism and are vulnerable if the adversary can compromise the host OS. There exist various hardwar e/software-based mechanisms and architectural frameworks [ 19 , 23 – 25 , 42 , 43 ] to se cure RT -Io T systems. However those frameworks ar e not designed to protect against contr ol-specic attacks and may not be suitable for systems developed with COTS components. There also exist large number of research for generic Io T systems as well as use of TrustZone to secure traditional embedded/mobile applications (too many to enumerate her e, r efer to the related surveys [ 10 , 44 – 46 ]) – however the consideration of time-critical and control-centric asp ects of RT -Io T applications distinguish Contego-TEE fr om other research. 6 CONCLUSION In this paper we presented a new framework named Contego- TEE that enhances the security and safety of the RT -Io T systems. W e use a combination of trusted hardware, intrinsic real-time nature and domain-specic characteristics of such systems to detect control intrusions and prevent the physical plants from being misbehaved under attacks. W e believe our framework is tangential and can be incorporated into multiple RT -Io T and cyb er-physical domains. REFERENCES [1] C.- Y. Chen, M. Hasan, and S. Mohan, “Se curing real-time Internet-of-things, ” Sensors , vol. 18, no. 12, 2018. [2] J. W estling, “Future of the Internet of things in mission critical applications, ” 2016. [3] N. Falliere, L. O. Murchu, and E. Chien, “W32. stuxnet dossier, ” White paper, Symantec Corp., Security Response , vol. 5, p. 6, 2011. [4] S. S. Clark and K. Fu, “Recent results in computer security for medical devices, ” in MobiHealth , 2011, pp. 111–118. 6 [5] S. Checkoway , D. McCoy , B. Kantor , D. Anderson, H. Shacham, S. Savage, K. Koscher , A. Czeskis, F. Roesner , T. K ohno et al. , “Comprehensive experimental analyses of automotive attack surfaces, ” in USENIX Se c. Symp. , 2011. [6] “I 2 C manual, ” Philips Semiconductors, 2003. [Online]. Available: https: //tinyurl.com/i2c- manual [7] “Raspberry Pi, ” https://w ww .raspb errypi.org/products/raspberr y- pi- 3- model- b/. [8] “Robot arm control, ” https://github.com/tutRPi/6DOF- Rob ot- Arm. [9] M. Sabt, M. Achemlal, and A. Bouab dallah, “Trusted execution environment: What it is, and what it is not, ” in IEEE Trustcom/BigDataSE/ISP A , 2015, pp. 57–64. [10] S. Pinto and N. Santos, “Demystifying ARM TrustZone: A comprehensiv e survey , ” ACM CSUR , vol. 51, no. 6, p. 130, 2019. [11] V . Costan and S. Devadas, “Intel SGX Explaine d, ” IACR Crypt. ePrint Arch. , no. 086, pp. 1–118, 2016. [12] C. L. Liu and J. W . Layland, “Scheduling algorithms for multiprogramming in a hard-real-time environment, ” JACM , vol. 20, no . 1, pp. 46–61, 1973. [13] N. A udsley , A. Burns, M. Richardson, K. Tindell, and A. J. W ellings, “ Applying new scheduling the ory to static priority pre-emptive scheduling, ” SE Journal , vol. 8, no. 5, pp. 284–292, 1993. [14] E. Bini and G. C. Buttazzo, “Sche dulability analysis of periodic xed priority systems, ” IEEE Trans. on Comp. , vol. 53, no. 11, pp. 1462–1473, 2004. [15] “TEE client API specication v1.0, ” https://globalplatform.org/sp ecs- library/ tee- client- api- specication/. [16] T . Liu, A. Hojjati, A. Bates, and K. Nahrstedt, “ Alidrone: Enabling trustworthy proof-of-alibi for commercial drone compliance, ” in IEEE ICDCS , 2018, pp. 841– 852. [17] F. Loi, A. Sivanathan, H. H. Gharakheili, A. Radford, and V . Sivaraman, “Systematically evaluating security and privacy for consumer IoT devices, ” in ACM IoTS&P , 2017, pp . 1–6. [18] R. Pellizzoni, N. Paryab, M.-K. Y oon, S. Bak, S. Mohan, and R. B. Bobba, “ A generalized model for prev enting information leakage in hard r eal-time systems, ” in IEEE RT AS , 2015, pp. 271–282. [19] M.-K. Y oon, S. Mohan, J. Choi, M. Christo dorescu, and L. Sha, “Learning execution contexts from system call distribution for anomaly detection in smart embedde d system, ” in ACM/IEEE Io TDI , 2017, pp. 191–196. [20] L. Sha, “Using simplicity to control complexity , ” IEEE Software , vol. 18, no. 4, pp. 20–28, 2001. [21] X. Liu, Q. Wang, S. Gopalakrishnan, W . He, L. Sha, H. Ding, and K. Lee, “ORTEGA: An ecient and exible online fault tolerance architecture for real-time control systems, ” IEEE T . on Ind. Inf. , vol. 4, no. 4, pp. 213–224, 2008. [22] X. Wang, N. Hovakimyan, and L. Sha, “L1Simplex: Fault-tolerant control of cyber-physical systems, ” in 2013 ACM/IEEE ICCPS , 2013, pp. 41–50. [23] S. Mohan, S. Bak, E. Betti, H. Yun, L. Sha, and M. Caccamo, “S3A: Secure system simplex architecture for enhance d se curity and robustness of cyb er-physical systems, ” in ACM international conference on High condence networked systems . ACM, 2013, pp . 65–74. [24] F. Abdi, M. Hasan, S. Mohan, D. Agarwal, and M. Caccamo, “ReSecure: A restart- based security protocol for tightly actuated hard real-time systems, ” in IEEE CERTS , 2016, pp. 47–54. [25] M.-K. Y oon, S. Mohan, J. Choi, J.-E. Kim, and L. Sha, “SecureCore: A multicore- based intrusion detection architecture for real-time embedded systems, ” in IEEE RT AS , 2013, pp. 21–32. [26] “SPI block guide V04.01, ” Motorola Inc, 2004. [Online]. A vailable: https://tinyurl.com/spi- blo ck [27] S. Adepu and A. Mathur , “From design to invariants: Detecting attacks on cyber physical systems, ” in IEEE QRS-C , 2017, pp. 533–540. [28] R. Wilhelm, J. Engblom, A. Ermedahl, N. Holsti, S. Thesing, D. Whalley , G. Bernat, C. Ferdinand, R. He ckmann, T. Mitra et al. , “The worst-case execution-time problem—overview of metho ds and sur vey of to ols, ” ACM TECS , vol. 7, no. 3, p. 36, 2008. [29] L. Cheng, K. Tian, and D. D . Y ao, “Orpheus: Enforcing cyber-physical execution semantics to defend against data-oriented attacks, ” in ACM ACSA C , 2017, pp. 315–326. [30] R. Liu and M. Srivastava, “VirtSense: Virtualize Sensing through ARM TrustZone on Internet-of- Things, ” in ACM SysTEX , 2018, pp. 2–7. [31] R. Liu and M. Srivastava, “PROTC: PROT eCting drone ’s peripherals through ARM trustzone, ” in ACM DroNet , 2017, pp. 1–6. [32] “Open Portable Trusted Execution Environment, ” https://ww w .op- tee.org/. [33] “Implementation co de for Contego-TEE, ” https://github.com/mnwrhsn/rt_ actuator_security. [34] “GoPiGo, ” https://github.com/DexterInd/GoPiGo. [35] P. Guo, H. Kim, N. Virani, J. Xu, M. Zhu, and P . Liu, “RoboADS: Anomaly detection against sensor and actuator misbehaviors in mobile rob ots, ” in IEEE/IFIP DSN , 2018, pp. 574–585. [36] “Dexter Industries Sensors, ” https://github.com/DexterInd/DI_Sensors. [37] H. Choi, W .-C. Lee, Y. Aafer , F. Fei, Z. T u, X. Zhang, D. Xu, and X. Xinyan, “Detecting attacks against robotic vehicles: A control invariant approach, ” in ACM CCS , 2018, pp. 801–816. [38] M. Hasan, S. Mohan, R. Pellizzoni, and R. B. Bobba, “Contego: An adaptive framework for integrating se curity tasks in real-time systems, ” in Euromicro ECRTS , 2017, pp. 23:1–23:22. [39] M. Hasan, S. Mohan, R. B. Bobba, and R. Pellizzoni, “Exploring opportunistic execution for integrating security into legacy hard real-time systems, ” in IEEE RTSS , 2016, pp. 123–134. [40] M. Hasan, S. Mohan, R. Pellizzoni, and R. B. Bobba, “ A design-space exploration for allocating security tasks in multicore real-time systems, ” in DA TE , 2018, pp. 225–230. [41] F. Fei, Z. T u, R. Yu, T . Kim, X. Zhang, D . Xu, and X. Deng, “Cross-layer retrotting of U A V s against cyber-physical attacks, ” in IEEE ICRA , 2018, pp. 550–557. [42] M.-K. Y oon, S. Mohan, J. Choi, and L. Sha, “Memor y heat map: anomaly detection in real-time embedded systems using memory behavior , ” in A CM/EDA C/IEEE DA C , 2015, pp. 1–6. [43] F. Abdi, C.- Y. Chen, M. Hasan, S. Liu, S. Mohan, and M. Caccamo, “Guaranteed physical security with restart-based design for cyber-physical systems, ” in ACM/IEEE ICCPS , 2018, pp. 10–21. [44] Y . Y ang, L. Wu, G. Yin, L. Li, and H. Zhao, “A survey on security and privacy issues in Internet-of- Things, ” IEEE IoT J. , vol. 4, no. 5, pp. 1250–1258, 2017. [45] M. Ammar , G. Russello, and B. Crispo, “Internet of Things: A survey on the security of Io T frameworks, ” Elsevier J. of Inf. Se c. & App. , v ol. 38, pp. 8–27, 2018. [46] W . Li, H. Chen, and H. Chen, “Research on ARM TrustZone, ” ACM GetMobile , vol. 22, no. 3, pp. 17–22, 2019. [47] M. Joseph and P. Pandya, “Finding response times in a real-time system, ” The Comp. J. , vol. 29, no. 5, pp. 390–395, 1986. APPENDIX Response Time Analysis for RT T asks Our schedulability test is base d on the xed-priority response time analysis propose d in RT literature [ 13 ]. Let N i be the number of actuation request for τ i and C o i is the additional computation time due to world switch and invariant checking. Then the W CET of τ i can be represent as C T E E i = C i + N i C o i . Since our enclave client and invariant che cker can serve one actuation request at a time ( e.g., an atomic process), τ i may be delay ed due to pr ocessing requests of lower priority tasks. Let B T E E i = max τ j ∈ l p ( τ i ) N j C o j denote the ‘blocking’ factor from tasks that are with lower-priority that τ i (denoted as l p ( τ i ) ). W e note that the maximum computational demand for a given task τ j in any interval length 0 ≤ w ≤ T j can be no more than the maximum execution time required by one job of τ j multiplied by the maximum number of jobs of τ j that can execute in that inter val [ 13 , 14 ]. The maximum interference experience by τ i from other tasks for an inter val w can b e expressed as: B T E E i + Í τ h ∈ h p ( τ i ) l w T h m C T E E h where hp ( τ i ) denotes the set of tasks with a priority higher than τ i . Therefor e, we can calculate the response time of τ i (denoted as r i ) as follows: r i = C T E E i + B T E E i + Õ τ h ∈ h p ( τ i )  r i T h  C T E E h . (1) The W CRT then can b e obtained by solving this recurrence using an iterative xed-point search, e.g., R T E E i = r ( α ) i = r ( α − 1 ) i for some iteration α with initial condition r ( 0 ) i = 0 . The iteration is guaranteed to be converged if we assume that the total pr ocessor utilization ( i.e., Í τ i C T E E i T i ) is less than 1 [ 47 ]. The taskset is considered as ‘unschedulable’ if there exists an α such that r ( α ) i > D i . Such unschedulability result will hint the designers to update parameters ( e.g., periods, number of tasks, invariant checking policies) to incorporate Contego- TEE framework for the target system. 7