Motion Decoupling and Composition via Reduced Order Model Optimization for Dynamic Humanoid Walking with CLF-QP based Active Force Control

Motion Decoupling and Composition via Reduced Order Model Optimization f or Dynamic Humanoid W alking with CLF-QP based Activ e F orce Contr ol Xiaobin Xiong and Aaron D. Ames Abstract — In this paper , 3D humanoid walking is decou- pled into periodic and transitional motion, each of which is decoupled into planar walking in the sagittal and lateral plane. Reduced order models (ROMs), i.e. actuated Spring- loaded In verted P endulum (aSLIP) models and Hybrid-Linear In verted Pendulum (H-LIP) models, are utilized f or motion generation on the desired center of mass (COM) dynamics for each type of planar motion. The periodic motion is planned via point foot (underactuated) ROMs for dynamic motion with minimum ankle actuation, while the transitional motion is planned via f oot-actuated ROMs for fast and smooth transition. Composition of the planar COM dynamics yields the desired COM dynamics in 3D, which is embedded on the humanoid via control L yapunov function based Quadratic pr ograms (CLF- QPs). Additionally , the ground r eaction f orce profiles of the aSLIP walking are used as desired refer ences f or gr ound contact for ces in the CLF-QPs f or smooth domain transitions. The proposed framework is realized on a lower -limb exoskeleton in simulation wherein differ ent walking motions are achie ved. I . I N T RO D U C T I O N In verted pendulums are valuable template models for realizing walking on humanoid robots. Since the in verted pendulum models are lower dimensional, they are often termed r educed order models (R OMs) . One of the most commonly used R OMs is the Linear Inv erted Pendulum (LIP) [2] [3] that is applied in the Zero Moment Point (ZMP) approaches [4] [5] [6]. The constant height of the point mass yields linear dynamics allowing fast online optimization [7] for gait generation. Closed-form solutions [4] hav e ev en been suggested for gait generation and stabilization. Ho we ver , the ZMP w alking oftentimes is quasi-dynamic to some extent due to the constant center of mass (COM) height. The ZMP approach [2] can also be viewed as embedding the LIP dynamics into the full dimensional humanoid robot. Similar idea has also appeared in the literature [8] [9] [10] [11] for embedding the canonical Spring-loaded Inv erted Pendulum (SLIP) [12]. For instance, [9] realized planar periodic bipedal walking in simulation by embedding a SLIP walking dynamics on the robot. [11] embedded a 3D- SLIP for humanoid running. The SLIP w alking is oftentimes considered dynamic since it generates natural oscillation of the COM and human-like ground reaction forces [13]. The difference in the dynamics of the LIP and SLIP results in dif ferent approaches to walking gait generation. The linear dynamics of LIP motiv ates solvable receding horizon optimizations [2] [7] [5] for continuous planning and *This work is supported by Amazon Fellowship in AI. The authors are with the Department of Mechanical and Civil Engineering, California Institute of T echnology , Pasadena, CA 91125 xxiong@caltech.edu , ames@caltech.edu Per i o d i c  Motion Transitional Motion Sagittal Plane Lateral Plane 𝑢  𝐹  𝑢  𝐹  CLF‐QP Force Control Cassie Atlas Exoskeleton 𝑢 𝑢 𝑦 𝑦 𝑢 𝑥 𝐹 𝑧 𝐹 𝑧 P P eri eri odi odi c c Moti Moti o on n Transi Transi ti ti onal onal Mot Mot i ion on Sagi Sagi ttal ttal P P l l an ane e Lat Lateral eral P P l l an an e e Fig. 1. Illustration of the motion decoupling and composition for humanoid walking via reduced order models. stabilization, thus periodic beha viors [14] are less studied. On the other hand, the nonlinear and underactuated dynamics of the SLIP dri ves researchers to study periodic orbits and their stabilization [13] [9], instead of nonperiodic behaviors or continuous planning methods. In this paper , we propose a composition of LIP and SLIP models for walking to leverage the advantages of each R OM. W alking is firstly decoupled into periodic and nonperiodic/transitional motion. Both are further decoupled into w alking in the sagittal and lateral planes, each of which is characterized by a R OM. Each R OM is modified from the canonical R OMs to best match the type of motion (Fig. 1). Specifically , we use an actuated SLIP (aSLIP) model and a Hybrid LIP (H-LIP) model for sagittal and lateral walking, respectiv ely . Point foot models of them are used for synthesizing periodic walking, and the footed models (with ankle actuation) are applied for generating transitional motion. The 3D coupling between planar models comes from the ground normal forces. T o generate optimal walking behaviors, we formulate nonlinear optimization problems on the aSLIP models for the periodic and transitional motion. The periodic orbits of the point foot H-LIP can be directly identified [14]. The transitional motion of the footed H-LIP is synthesized via a quadratic program (QP). ZMP constraint and ankle actuation limit are included in the transitional optimizations. The motion from the R OMs is encoded by the COM, swing foot position and ground reaction forces. W e propose an embedding of the desired R OM dynamics on humanoids via control L yapunov function based Quadratic programs (CLF-QPs). The CLF-QPs formulation also enables force control on matching ground normal forces of the aSLIP on the humanoid, which naturally creates smooth transitions between different domains. The composition of R OMs naturally combines the benefit of using each R OM. The aSLIP model enables the walking to be dynamic, and the H-LIP model facilitates solvable QP for planning. The optimization of the aSLIP model is also fast to solve due to its low dimensionality . Moreov er , the optimization for each transition is only required for solving once, which potentially can perform online with the recent progresses [15] [16] on the online optimization. The proposed framework is implemented on an exoskeleton in simulation to generate different walking behaviors, which illustrates periodic motion generation on pointed foot R OMs, the transition optimization on footed R OMs, and finally the CLF-QPs based force control for the dynamics embedding on the robot. I I . R O B O T M O D E L W e use an exoskeleton (Exo) [17] as the example system for ev aluating our methods. Although the exoskeleton is not designed as a humanoid, it does have the identical anthropomorphic leg design to many current humanoids, such as the Atlas robot in Fig. 1. Here we briefly describe the robot model and the hybrid model of walking. The hybrid models of the R OMs are described similarly . A. Dynamics Model The Exo has a standard anthropomorphic leg design. Each leg has a hip joint with three degrees of freedom (DOF), a knee joint with one DOF and an ankle joint with two DOF . There are four load cells on each foot for contact force measurement. The dynamics of the system is described with the floating base model by the Euler-Lagrange equation: M ( q ) ¨ q + h ( q , ˙ q ) = B u + J v ( q ) T F v , (1) J v ( q ) ¨ q + ˙ J v ( q ) ˙ q = 0 , (2) where q ∈ S E (3) × R n =12 , M ( q ) is the mass matrix, h ( q , ˙ q ) is the Coriolis, centrifugal and gra vitational term, B and u ∈ R 12 are the actuation matrix and the motor torque vector , and F v ∈ R n v and J v are the holonomic force vector from ground contact and the corresponding Jacobian respectiv ely . W e use subscript v to denote different domains. F or example, when robot has two feet flatly contacting the ground, i.e. in Double Support Phase (DSP), n v = DSP = 12 . B. Hybrid Model of T wo Domain W alking In this paper , we are interested in two domain w alking, i.e. walking with Single Support Phase (SSP) and Double Support Phase (DSP) depending on the number of legs that contact the ground. W e assume that the feet alw ays ha ve flat contact with the ground. The transition from DSP to SSP happens when the one of the stance feet is about to lift of f the ground (ground reaction force becomes 0). The transition from SSP to DSP happens when the swing foot strikes the ground. The impact between the swing foot and the ground is modeled as plastic impact [18], where the velocity of the system undergoes a discrete jump, i.e., ˙ q + DSP = ∆( q ) ˙ q − SSP , (3) where ∆( q ) is the impact map. I I I . P E R I O D I C W A L K I N G G E N E R A T I O N V I A P O I N T F O OT R E D U C E D O R D E R M O D E L S In this section, we describe the periodic motion generation of the point foot reduced order models. Specifically , the actuated Spring-loaded In verted Pendulum (aSLIP) model is used for generating target COM dynamics in the sagit- tal plane for periodic walking; the Hybrid passiv e Linear In verted Pendulum (H-LIP) model is used for identifying the COM dynamics for the lateral periodic motion. Both models have been introduced in [19], and the H-LIP model is further studied in [14]. Here we briefly describe them for the application of this paper and complement certain details for the ease of understanding. A. P oint F oot aSLIP Model for Sagittal P eriodic W alking The point foot aSLIP model is a SLIP model with leg length actuation (Fig. 2 (S1)) [19]. The emphasis on the point foot is to dif ferentiate from the later one with foot actuation. SLIP models with actuation hav e existed in the literature [8] [20] [21]. The one we use is originally proposed for approximating the w alking dynamics of the robot Cassie [19]. The springs on the aSLIP come from the leg spring ap- proximation of the leg dynamics [22]. The main differences of the aSLIP , comparing with the others, are that: the springs hav e nonlinear stiffness and damping which are functions of leg length L ; the actuation is realized via changing leg length, i.e. ¨ L . The dynamics are in the Appendix. W e use the aSLIP for generating periodic walking motion of the COM. The walking is modeled by two domains, i.e. SSP and DSP (Fig. 2 (S1)). Specifications of the walking are encoded as equality and inequality constraints in the optimization, such as the durations of each domain, step length or speed and the range of the COM height. Physical constraints include nonneg ativ e contact forces, friction cones, and spring deflection limits. The optimization is formulated via the direct collocation [23] method to av oid integration of the dynamics. The trajectories are discretized ov er time in each domain with ev en nodal spacing. W e use trapezoidal integration for ap- proximating the dynamics by algebraic constraints. Domain transition constraints are directly enforced at the nodes of the start and end of each domain. W ith the cost being minimizing the virtual actuation ¨ L , the optimization is formulated as: z ∗ = argmin z N X i =1 ∆ t 2 ( ¨ L i 2 L + ¨ L i 2 R ) , (4) s.t. z min ≤ z ≤ z max , c min ≤ c ( z ) ≤ c max , (5) where z is the vector of all optimization variables, ∆ t is the time discretization, and c ( z ) includes all constraints. The subscripts L , R denote the left and right leg. The simple dynamics facilitates fast performance on solving the optimization, which is normally solved within 1 ∼ 2 seconds with random initial guesses using IPOPT [24]. The canonical SLIP model can also be used for finding different walking motions [9]. Howe ver , the system energy of canonical SLIP is conserved, which is not preferred for generating versatile motions. Thus we use the aSLIP model. The direct use of the stiffness and damping from a physical robot Cassie (Fig. 1) also av oids the trial and error on finding the appropriate parameters of the springs. B. P oint F oot H-LIP for P eriodic Lateral Motion Generation The aSLIP optimization generates periodic motion in the sagittal plane. The durations of each domain, i.e. T SSP and T DSP , are thus fixed. Here we apply the Hybrid passiv e Linear In verted Pendulum (H-LIP) for generating periodic lateral motion giv en the domain durations. The H-LIP is the point foot (passiv e) Linear In verted Pendulum (LIP). It also has double support phase (DSP) in walking (Fig. 2 (L1)). The v elocity in DSP is assumed to be constant. W e used it in [19] to approximate the lateral dynamics of the pelvis of Cassie during walking. Its dynamics are: ¨ y SSP = λ 2 y , (SSP) ¨ y DSP = 0 , (DSP) where λ = q g z 0 , and z 0 is the averaged height of the point mass of aSLIP walking. The transition from SSP to DSP , ∆ S → D , and the transition from DSP to SSP , ∆ D → S , are assumed to be smooth, thus the impact maps are defined as: ∆ S → D :  ˙ y + = ˙ y − y + = y − (6) ∆ D → S :  ˙ y + = ˙ y − y + = y − − w (7) where w is the step width from the stance foot position to the landing foot. The transitions are time-based since { T SSP , T DSP } are already determined by the aSLIP walking in the sagittal plane. The periodic orbit for the lateral motion can be identified in closed-form from the linear dynamics of the H-LIP . For the lateral motion, we limit the choice to the period-2 (P2) orbits, which are periodic orbits with two steps. Fig. 2 (L2) illustrates different P2 orbits in the phase portrait. Note that the lines with slope σ 2 is called the lines of characteristics for the P2 orbits [14], where σ 2 := λ tanh  T SSP 2 λ  . (8) Any point of the line yields a periodic orbit by following the flo w of the dynamics. More details can be found in [14]. Fig. 2 (L3) shows the examples of se veral P2 orbits with the trajectories of the point mass. Orbit Composition. Composition of the mass trajectory of the aSLIP in the sagittal plane and that of the H-LIP in the lateral plane yields the desired COM trajectory in 3D. The point foot nature of the two reduced order models locates the center of pressure at the center of the foot, which is expected to provide a maximum margin for foot stability for embedding the COM on the humanoid. Fig. 2 (C) shows an example of the composed trajectory in 3D. 𝑧  𝑙 SSP DSP (S1) (L1) 0.2 0 0.2 1 0 1 𝜎  𝑦 󰇗 󰇛m/s󰇜 𝑦 󰇛m 󰇜 (L3) 𝑟 𝑠 𝐿 𝛽 DSP SSP (C) (S2) 𝑡 󰇛𝑠󰇜 (L2) 𝑡 󰇛𝑠󰇜 𝑦 󰇛m 󰇜 𝑧 󰇛m󰇜 𝐹  󰇛N󰇜 𝑥 󰇗 󰇛m/s󰇜 𝑦 󰇛m 󰇜 𝑧 󰇛m󰇜 𝑥 󰇛 m󰇜 Fig. 2. (S1, L2) Illustration of the point foot aSLIP and H-LIP . (S2) T rajectories of an optimized periodic gait in terms of the vertical position z , normal reaction force F z of foot contact and forward velocity ˙ x of the point mass. (L2) The characterized periodic orbits of the H-LIP in the phase portrait. (L3) T rajectories of the point mass for different periodic orbits. (C) An example of the composed trajectory of the point mass in 3D. I V . T R A N S I T I O N AL M OT I O N G E N E R A T I O N V I A F O O T E D R E D U C E D O R D E R M O D E L S In this section, we describe footed reduced order models for fast and dynamic motion transition to enable v ersatile walking behaviors. The foot actuation is directly applied on the reduced order models with the torque limits and the constraint that the center of pressure (COP) lies within the feet. The transition beha vior is defined as the motion that transits the robot states between standing configurations to periodic motions or between periodic motions themselves. Optimization on the transition motion of the full dimensional robot dynamics is still a difficult problem to solv e due to complex specifications on the dynamics constraints, contact sequences, and actuation limits. The transition optimiza- tion on the footed reduced order models can simplify the optimization while providing the desired transition COM dynamics. The application of ankle actuation also provides fast and smooth transitions. W e assume that the transition motion is specifically real- ized within two domains, one DSP and one SSP . In other words, we pre-specify the contact sequence and number of domains for the transition motion. It is also possible to for- mulate the optimization into a contact implicit optimization using the Linear Complementarity Problem formulation [25] or using potential smooth techniques [26] for the hybrid dynamics, which is not within the scope of this paper . A. F ooted aSLIP for T ransition in Sagittal Plane The footed aSLIP model is the aSLIP with actuated foot (Fig. 3 (S1)). The foot actuation resembles the ankle pitch on the humanoid. The foot length equals to that of the humanoid. The dynamics are in the Appendix. The transition optimization formulation is similar to the previous aSLIP optimization. The differences are as follows. Cost Function. With the foot actuation, the cost for the transitional optimization includes both the leg length actuation and the ankle torques: J = N X i =1 ∆ t 2 ( ¨ L i 2 L + ¨ L i 2 R + u i 2 L y + u i 2 R y ) , (9) where u i L y , u i R y are the ankle pitch torques. ZMP and Actuation Constraints. T o ensure the opti- mized COM trajectories are realizable on the humanoid, the zero moment point (ZMP) and physical actuation constraints must be included, − L h F i L/R < u i L/R y < L t F i L/R ,    u i L/R y    < u y max , (10) where L h , L t are the distance between the projected ankle pitch axis to the heel and to the toe, F i L/R is the normal reaction force of foot contact, and u y max is the maximum ankle pitch torque of the humanoid. The interpretation of the ZMP constraint is simply that the COP must be in the foot so that the foot does not rotate on the ground. Initial/Final States. The initial and final states must be identical to the ones at the beginning and the end of the motion transition. The specification of the states comes from the humanoid. The real leg length r , leg angle β and their velocities can directly mapped from the COM states of the robot. The leg length state can be solved by force balancing on the point mass with its acceleration. B. F ooted LIP for T ransition via Quadratic Pro grams The durations of each domain in the transitional motion are determined by the footed aSLIP optimization. For generating the corresponding motion in the lateral plane, we use the footed H-LIP model, which is the H-LIP with foot actuation (Fig. 3 (L1)). The ankle actuation on the foot becomes the input to the system. Its continuous dynamics can be written compactly in each domain as: ¨ y SSP = λ 2 y + 1 mz 0 u L/R x , (SSP) ¨ y DSP = 1 mz 0 ( u L x + u R x ) , (DSP) where u L x and u R x are the ankle roll actuations. u L/R x = 0 if the foot is not in contact with the ground. z 0 is the averaged height of the mass of the footed aSLIP in transition motion. The linear dynamics motiv ates a quadratic program (QP) formulation for optimizing the transition from an initial state [ y 0 ; ˙ y 0 ] to a final state [ y f ; ˙ y f ] . W e discrete the trajectory over (S1) 𝑢  𝐹  (S2) 𝐹  𝑧  𝐹  𝐹  𝐹  0 𝑢  𝑢  𝑢  𝑡 󰇛𝑠󰇜 𝑡 󰇛𝑠󰇜 𝑢   󰇛Nm󰇜 𝑢   󰇛Nm󰇜 𝑦 󰇗 󰇛m/s󰇜 𝑦 󰇛m󰇜 SSP DSP (L1) DSP SSP 𝐹  𝑢   𝑧  𝐹  𝐹  𝑢   𝑢   𝑢   𝐹  𝐹  𝐹  𝑢   𝑢   (L2) 𝑧 󰇛m󰇜 𝐹  󰇛N󰇜 𝑥 󰇛 m/ s󰇜 Fig. 3. (S1, L1) Illustration of the footed aSLIP and footed H-LIP . (S2, L2) The results of optimizations of the footed reduced order models transiting from standing to periodic walking. The blue lines indicate those in DSP , and the red indicate those in SSP . The yellow and black indicate the lower and upper bounds respectiv ely on the ZMP constraint in the H-LIP optimization. time with the discretized linear dynamics being,  y k +1 ˙ y k +1  | {z } Y k +1 = A DSP/SSP  y k ˙ y k  | {z } Y k + " T 2 2 mz 0 T mz 0 # | {z } B u DSP/SSP x , (11) where, A SSP =  1 T λT 1  , A DSP =  1 T 0 1  , (12) u SSP = u L/R x , u DSP = u L x + u R x . (13) ZMP and Actuation Constraints. W ith an eye towards the embedding on the humanoid, we also enforce the ZMP and actuation constraints in the lateral transition as: − W 1 F k L/R ≤ u k L/R x ≤ W 2 F k L/R ,   u k L/R x   < u x max , (14) where W 1 , W 2 are the distances between the projected ankle roll axis to each edge of the foot, and u x max is the maximum ankle roll torque. Coupling with Footed aSLIP . The footed R OMs in each plane are coupled by the ground normal forces. The vertical reaction forces in Eq. (14) are from F L/R ( t ) of the footed aSLIP 1 . Quadratic Pr ogram. The final QP formulation with min- imizing u for efficienc y is as follows, Y { 1 ,...,N } = argmin { u L x ,u R x ,Y }∈ R N × N × 2 N X N k =1 u k 2 L x + u k 2 R x (15) s.t. Y k +1 = A SSP/DSP Y k + B u SSP/DSP x (H-LIP)   u k L/R x   < u x max (T orque Limit) − W 1 F k L/R ≤ u k L/R x ≤ W 2 F k L/R (ZMP) [ y N ; ˙ y N ] = [ y f ; ˙ y f ] (Final State) [ y 1 ; ˙ y 1 ] = [ y 0 ; ˙ y 0 ] . (Initial State) 1 The sum of the vertical reaction forces on the H-LIP , i.e. F L ( t ) + F R ( t ) , does not equal to mg , since the COM height varies from the footed aSLIP . Comparison with ZMP Approaches. Quadratic pro- grams [2] [27] have been widely used on the LIP dynamics for controlling humanoid walking. In our approach, planning on the ZMP trajectory is not required and the QP is only required to be solved once instead of online recursively . Remark. The optimizations on the footed reduced order models are connected by the ground reaction force. Therefore the footed aSLIP optimization and the footed H-LIP QP can be combined into a single nonlinear optimization. The steps are straightforward by combining the v ariables, constraints and cost functions. V . D Y N A M I C S E M B E D D I N G O N H U M A N O I D S W ith the constructed COM dynamics from reduced order models, one can embed the dynamics on the fully actuated humanoid robot by tracking the desired COM trajectory . The walking is thus realized by defining all the required outputs and zeroing the outputs via feedback control. In this section, we briefly define the outputs for walking of the Exo and then describe the feedback control with ground reaction force matching using the control L yapunov function based Quadratic programs (CLF-QPs). A. Output Definition 1) Outputs for DSP: In DSP , foot contacts introduce 12 holonomic constraints. There are 6 outputs required for the 18 DOF robot. Except for the COM position, we require the pelvis orientation to be fixed. Thus the outputs for the DSP is defined as, Y DSP ( q , t ) =  p COM ( q ) φ φ φ pelvis ( q )  −  p desired COM ( t ) 0  . (16) 2) Outputs for SSP: In SSP , only one foot contacts the ground. Thus we define 12 outputs with additional 6 being on the swing foot position and orientation, Y SSP ( q , t ) =     p COM ( q ) φ φ φ pelvis ( q ) p swingFoot ( q ) φ φ φ swingFoot ( q )     −     p desired COM ( t ) 0 p desired swingFoot ( t ) 0     . (17) The swing foot position p desired swingFoot ( t ) is constructed smoothly from the initial to the final position of the SSP . 𝐹 𝑧 𝑐 = 0 . 1 𝑐 = 0 . 25 Fig. 4. Illustration of the valid range of the relaxation on the desired normal force with different relaxation coefficient c . B. CLF-QP W e apply the control L yapunov function based Quadratic programs (CLF-QP) [28] [22] for zeroing the outputs. The control L yapunov function V is constructed over the feed- back linearized output dynamics by quadratic functions of the outputs. The e xponential con vergence of V is nicely enforced by the inequality condition on the deriv ative ˙ V , ˙ V ( u, q , ˙ q ) ≤ − γ V ( q , ˙ q ) , (18) with γ > 0 . The inequality is affine with respect to u , A CLF v ( q , ˙ q ) u ≤ b CLF v ( q , ˙ q ) . (19) Thus quadratic programs can be formulated for optimizing u subject to this L yapunov inequality and additional phyical constraint such as torque limits and contact constraints. Details about constructing the CLF-QPs are in [28] [22]. For the application of walking system with holonomic constraints, [28] [22] also include the contact forces as the optimization variables. Holonomic constraints and ground contact constraints are thus encoded via equality and inequal- ity constraints, respectiv ely . Here we remov e them from the optimization variables for QP formulation with minimum arguments for ef ficiency . It can be done since the contact forces are affine functions with respectiv e to u , F v = A v u + b v , (20) A v = − ( J v M − 1 J T v ) − 1 J v M − 1 B , (21) b v = ( J v M − 1 J T v ) − 1 ( J v M − 1 h − ˙ J v ˙ q ) , (22) where F v is the contact force v ector in domain v . It is deri ved from Eq. (1) (2). The constraints on the ground reaction forces (GRF) are thus enforced directly on u . Let C v denote the contact constraint matrix, i.e. C v F v ≤ 0 , which includes the constraints of the friction cones, nonnegati ve normal forces and ZMP [18]. Then the GRF constraint becomes, C v A v u ≤ − C v b v . (23) C. F or ce Contr ol Embedding W e would like the realized walking to exhibit the aSLIP walking behavior in terms of ground reaction forces. If the contact forces behav e identically to those of the aSLIP walk- ing, the transitions from DSP to SSP can happen naturally as the normal force goes to zero. The transition from SSP to DSP also behaves smoothly when the normal force increases matching the aSLIP . Thus the force control can be realized via enforcing S v F v = F z aSLIP , where S v is the selection matrix. W ith Eq. (20), the equality becomes, S v A v u = F z aSLIP − S v b v . (24) Since we control the vertical COM oscillation of the hu- manoid to be identical to that of the aSLIP simultaneously , the vertical GRF match needs to be relax ed. Otherwise, strictly matching the reaction force can destabilize the track- ing. Thus we relax the equality in Eq.(24) by the inequality (a) Join tt o r que CO P  p o s i t i o n Impact Transition Periodicwalki ng 0 1 2 -0.08 -0.06 -0.04 -0.02 0 0.02 0246 0.56 0.58 0.6 (b) (d) Desired Transition ActualTransition ActualPeriodic DesiredPeriodic Delt a inQP (c) Fig. 5. Simulation results of the walking from standing. (a) The tracking on the desired COM position. (b) The evolution of COM velocity . (c) Snopshots of the walking of the Exo. (d) Ground normal reaction forces are with in the relaxation under the CLF-QP control. | S v F v − F z aSLIP | ≤ cF z aSLIP , resulting in: (1 − c ) F z aSLIP − S v b v | {z } c lb ≤ S v A v u ≤ (1 + c ) F z aSLIP − S v b v | {z } c ub , (25) where c ∈ (0 , 1) is a coefficient of the relaxation. In practice, we use c = 0 . 1 . Fig. 4 shows the valid range of the F z with different c . It is important to note that this relaxation shrinks as the force decreases, which enforces the smooth transitions between DSP and SSP with the GRF decreasing to 0 or increasing from 0. D. Main Contr ol Law The main feedback control loop can thus be realized by the CLF-QP with force control embedded as an inequality constraint: u ∗ = argmin u ∈ R 12 ,δ ∈ R u T H u + 2 F u + pδ 2 , (26) s.t. A CLF v ( q , ˙ q ) u ≤ b CLF v ( q , ˙ q ) + δ, (CLF) C v A v u ≤ − C v b v , (GRF) u lb ≤ u ≤ u ub , (T orque Limit) c lb ≤ S v A v u ≤ c ub , (Force Control) where δ is a relaxation term for increasing the instantaneous feasibility of the QP , and p is a large positive penalty constant. Adding δ is relaxing the tracking performance. Since there is no guarantee for the feasibility of the QP with multiple constraints, it is appropriate to relax the performance while keeping the system safe. Remark. In the context of QP based controllers, task/operational space control (TSC) [11] could be utilized in an similar fashion. The CLF-QP encodes the control objectiv e by an inequality constraint; while the TSC encodes the objectiv e in the cost function, which minimizes the difference between the actual output accelerations and the desired v alues from a PD control. The ground reaction force matching can also be put into the TSC formulation. V I . R E S U L T S The proposed framew ork is primarily implemented in simulation. The dynamics are numerically integrated using MA TLAB’ s ode45 function with e vent functions for trigger - ing different contact domains. The nonlinear programs for the trajectory optimization of the aSLIP are solved by IPOPT [24]. The QP for the footed H-LIP and the CLF-QP for the control of the Exo are solved by qpO ASES [29] with activ e set method. W e mainly e valuate the method for two walking scenarios, i.e. periodic walking from standing and transition between two periodic walking behaviors. For both cases, periodic walking is first composed from the periodic orbits of the aSLIP and the H-LIP . Then the nonlinear program on the footed aSLIP and the QP for the footed H-LIP are solved for generating the transition in each plane. The ground normal forces of the aSLIP are used as references in Eq. (25). The composed trajectories of the COM are set to the desired outputs for the CLF-QP to generate desired torques in the control loop. Fig. 5 shows one of the simulation results. Additional simulation and video can be found in [1]. The COM of the Exo follo ws the desired COM trajectory well. The ground normal forces are within the range of the relaxation. Note that the impact of the foot-ground contact still exists in the walking since we do not necessarily require the foot to strike the ground with zero velocity . It does not destabilize the system due to the force control. On the hardware, we expect in future work to use the load cells for contact detection and use the force control to generate compliant behavior instantaneously after impact. V I I . C O N C L U S I O N A N D F U T U R E W O R K In this paper , we propose the decoupling of walking in terms of transitional and periodic walking, each of which is further decoupled into walking in the sagittal and lat- eral plane. Optimizations for planar reduced order models (R OMs) are utilized for motion generation and 3D com- position. Desired walking behavior is thus encoded by the COM dynamics and the ground reaction forces of the R OMs. Control L yapunov function based quadratic programs (CLF- QPs) with activ e force control are applied for the dynamics and force embedding on the fully actuated humanoid robots. Future work will be heavily focused toward the hardware implementation of the proposed approach on the Exoskele- ton. This will include online gait generation via the reduced order models and realization of the CLF-QP with contact force control. A P P E N D I X 1) Dynamics of the P oint F oot and F ooted aSLIP: The dynamics the footed aSLIP is, S :    ¨ r 1 = F 1 m − g cos ( β 1 ) + r ˙ β 2 1 ¨ β 1 = 1 r 1 ( − 2 ˙ β 1 ˙ r 1 + g sin ( β 1 ) + u 1 mr 1 ) ¨ s 1 = ¨ L 1 − ¨ r 1 , D :                      ¨ r 1 = F 1 + F 2 cos ( δ q ) m − g cos ( q 1 ) + r 1 ˙ q 2 1 + u 2 mr 2 sin ( δ q ) ¨ q 1 = − 2 ˙ q 1 ˙ r 1 + g sin ( q 1 ) − F 2 m sin ( δ q ) r 1 + cos ( δ q ) mr 1 r 2 u 2 + u 1 mr 2 1 ¨ r 2 = F 2 + F 1 cos ( δ q ) m − g cos ( q 2 ) + r 2 ˙ q 2 2 − u 1 mr 1 sin ( δ q ) ¨ q 2 = − 2 ˙ q 2 ˙ r 2 + g sin ( q 2 )+ F 1 m sin ( δ q ) r 2 + cos ( δ q ) mr 1 r 2 u 1 + u 2 mr 2 2 ¨ s 1 = ¨ L 1 − ¨ r 1 ¨ s 2 = ¨ L 2 − ¨ r 2 , where δ q = q 1 − q 2 , ¨ L 1 / 2 are assumed to be the virtual input [19], and u 1 / 2 are the ankle actuation. The spring forces F 1 / 2 come from the leg spring [22]. The dynamics of point foot aSLIP is that of the footed aSLIP with zero ankle actuation. 2) Impact Map Assumption: The impact map is, ∆ SSP → DSP :    ˙ q + 2 = 1 r 2 ( ˙ q 1 ˙ r 1 cos ( δ q ) + ˙ r 1 sin ( δ q )) ˙ r + 2 = ˙ r 1 cos ( δ q ) − ˙ q 1 r 1 sin ( δ q ) ˙ s + 2 = ˙ L − 2 − ˙ r + 2 , where we only specify the discontinuous states to save space. The transition map ∆ DSP → SSP is smooth. 3) Le g Spring Appr oximation for Cassie: The nonlinear prismatic spring on the aSLIP model comes from the approx- imation of the leg dynamics of Cassie [22]. Here we detail the coefficients for reference. K ( L ) = (23309 L 4 − 55230 L 2 + 48657 L − 9451) c, D ( L ) = (348 L 4 − 824 L 2 + 726 L − 141) c, where c = 2 . 28 is the mass ratio between the humanoids and Cassie, and the leg spring force is F = K ( L ) s + D ( L ) ˙ s , R E F E R E N C E S [1] https://youtu.be/L3XW - dRE3E0. [2] S. Kajita, F . Kanehiro, K. Kaneko, K. Fujiwara, K. Harada, K. Y okoi, and H. Hirukawa, “Biped walking pattern generation by using previe w control of zero-moment point, ” in Robotics and Automation, IEEE International Confer ence on , vol. 2, 2003, pp. 1620–1626. [3] J. Pratt, T . K oolen, T . De Boer, J. Rebula, S. Cotton, J. Carff, M. Johnson, and P . Neuhaus, “Capturability-based analysis and control of legged locomotion, part 2: Application to m2v2, a lower -body humanoid, ” The International J ournal of Robotics Research , vol. 31, no. 10, pp. 1117–1133, 2012. [4] R. T edrake, S. K uindersma, R. Deits, and K. Miura, “ A closed-form so- lution for real-time zmp gait generation and feedback stabilization, ” in 2015 IEEE-RAS 15th International Confer ence on Humanoid Robots (Humanoids) , pp. 936–940. [5] T . T akenaka, T . Matsumoto, T . Y oshiike, and S. Shirokura, “Real time motion generation and control for biped robot-2 nd report: Running gait pattern generation, ” in 2009 IEEE/RSJ International Confer ence on Intelligent Robots and Systems , pp. 1092–1099. [6] S. Feng, X. Xinjilefu, W . Huang, and C. G. Atkeson, “3d walking based on online optimization, ” in 2013 13th IEEE-RAS International Confer ence on Humanoid Robots (Humanoids) , pp. 21–27. [7] J. Kim, N. S. Pollard, and C. G. Atkeson, “Quadratic encoding of optimized humanoid walking, ” in Humanoid Robots (Humanoids), 2013 13th IEEE-RAS International Conference on , pp. 300–306. [8] Y . Liu, P . M. W ensing, D. E. Orin, and Y . F . Zheng, “Dynamic walking in a humanoid robot based on a 3d actuated dual-slip model, ” in 2015 IEEE International Confer ence on Robotics and Automation (ICRA) , pp. 5710–5717. [9] G. Garofalo, C. Ott, and A. Albu-Sch ¨ affer , “W alking control of fully actuated robots based on the bipedal slip model, ” 2012 IEEE International Confer ence on Robotics and Automation , pp. 1456–1463. [10] I. Mordatch, M. De Lasa, and A. Hertzmann, “Rob ust physics-based locomotion using low-dimensional planning, ” in ACM T ransactions on Graphics (TOG) , v ol. 29, no. 4. A CM, 2010, p. 71. [11] P . M. W ensing and D. E. Orin, “High-speed humanoid running through control with a 3d-slip model, ” in 2013 IEEE/RSJ International Confer ence on Intelligent Robots and Systems , pp. 5134–5140. [12] R. J. Full and D. E. Koditschek, “T emplates and anchors: neu- romechanical hypotheses of legged locomotion on land, ” Journal of experimental biology , v ol. 202, no. 23, pp. 3325–3332, 1999. [13] H. Geyer , A. Seyfarth, and R. Blickhan, “Compliant leg behaviour explains basic dynamics of walking and running, ” Proceedings of the Royal Society of London B: Biological Sciences , vol. 273, no. 1603, pp. 2861–2867, 2006. [14] X. Xiong and A. D. Ames, “Orbit characterization, stabilization and composition of 3d underactuated bipedal walking via hybrid passive linear in verted pendulum model, ” in 2019 IEEE/RSJ International Confer ence on Intelligent Robots and Systems (IROS) . [15] Y . Ding, A. Pandala, and H.-W . Park, “Real-time model predicitiv e control for versatile dynamic motions in quadrupedal robots, ” in IEEE International Confer ence on Robotics and Automation, ICRA 2019 . [16] J. Di Carlo, P . M. W ensing, B. Katz, G. Bledt, and S. Kim, “Dynamic locomotion in the mit cheetah 3 through con vex model-predictive control, ” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pp. 1–9. [17] W andercraft https://www .wandercraft.eu/en/exo/. [18] J. Grizzle, C. Chev allereau, R. W . Sinnet, and A. D. Ames, “Models, feedback control, and open problems of 3d bipedal robotic walking, ” Automatica , v ol. 50, no. 8, pp. 1955–1988, 2014. [19] X. Xiong and A. D. Ames, “Coupling reduced order models via feedback control for 3d underactuated bipedal robotic walking, ” in 2018 IEEE-RAS 18th International Confer ence on Humanoid Robots (Humanoids) , pp. 1–9. [20] I. Poulakakis and J. Grizzle, “The spring loaded in verted pendulum as the hybrid zero dynamics of an asymmetric hopper, ” IEEE T rans- actions on Automatic Contr ol , v ol. 54, no. 8, pp. 1779–1793, 2009. [21] G. Piov an and K. Byl, “Reachability-based control for the active slip model, ” The International Journal of Robotics Resear ch , vol. 34, no. 3, pp. 270–287, 2015. [22] X. Xiong and A. D. Ames, “Bipedal hopping: Reduced-order model embedding via optimization-based control, ” in 2018 IEEE/RSJ Inter - national Conference on Intelligent Robots and Systems (IROS) , pp. 3821–3828. [23] A. Hereid, E. A. Cousineau, C. M. Hubicki, and A. D. Ames, “3d dynamic w alking with underactuated humanoid robots: A direct col- location framew ork for optimizing hybrid zero dynamics, ” in Robotics and A utomation (ICRA), 2016 IEEE International Conference on , pp. 1447–1454. [24] A. W ¨ achter and L. T . Biegler , “On the implementation of an interior- point filter line-search algorithm for large-scale nonlinear program- ming, ” Mathematical pr ogramming , v ol. 106, no. 1, pp. 25–57, 2006. [25] M. Posa, C. Cantu, and R. T edrake, “ A direct method for trajectory op- timization of rigid bodies through contact, ” The International Journal of Robotics Researc h , vol. 33, no. 1, pp. 69–81, 2014. [26] T . W estenbroek, X. Xiong, A. D. Ames, and S. S. Sastry , “Optimal control of piecewise-smooth control systems via singular perturba- tions, ” arXiv preprint , 2019. [27] C. Brasseur , A. Sherikov , C. Collette, D. Dimitrov , and P .-B. Wieber , “ A robust linear mpc approach to online generation of 3d biped walking motion, ” in 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids) , pp. 595–601. [28] A. D. Ames and M. Powell, “T owards the unification of locomotion and manipulation through control lyapunov functions and quadratic programs, ” in Contr ol of Cyber-Physical Systems . Springer , 2013, pp. 219–240. [29] H. Ferreau, C. Kirches, A. Potschka, H. Bock, and M. Diehl, “qpO ASES: A parametric activ e-set algorithm for quadratic program- ming, ” Mathematical Pr ogramming Computation , vol. 6, no. 4, pp. 327–363, 2014.