Latent Diffeomorphic Co-Design of End-Effectors for Deformable and Fragile Object Manipulation

Manipulating deformable and fragile objects remains a fundamental challenge in robotics due to complex contact dynamics and strict requirements on object integrity. Existing approaches typically optimize either end-effector design or control strategies in isolation, limiting achievable performance. In this work, we present the first co-design framework that jointly optimizes end-effector morphology and manipulation control for deformable and fragile object manipulation. We introduce (1) a latent diffeomorphic shape parameterization enabling expressive yet tractable end-effector geometry optimization, (2) a stress-aware bi-level co-design pipeline coupling morphology and control optimization, and (3) a privileged-to-pointcloud policy distillation scheme for zero-shot real-world deployment. We evaluate our approach on challenging food manipulation tasks, including grasping and pushing jelly and scooping fillets. Simulation and real-world experiments demonstrate the effectiveness of the proposed method.

💡 Research Summary

This paper tackles the longstanding challenge of manipulating deformable and fragile objects (DFOM) by jointly optimizing end‑effector morphology and the associated manipulation control—a co‑design approach that has been largely unexplored for such tasks. The authors introduce three core technical contributions. First, they propose a Latent Diffeomorphic Morphology (LDM) representation that encodes end‑effector shapes as smooth, invertible deformations of a base mesh. A stationary velocity field (SVF) is parameterized with radial basis functions (RBFs), yielding a 72‑dimensional raw design vector. By fitting this representation to a curated dataset of 1,000 finger‑like designs and applying Principal Component Analysis, they obtain a compact latent space of about 10‑15 dimensions, enabling efficient exploration while guaranteeing topology preservation.

Second, they formulate the co‑design problem as a bi‑level optimization. The lower level finds a design‑conditioned control policy π*​(d) for a given shape d. This involves (a) pre‑contact configuration optimization using CMA‑ES on a differentiable objective that combines signed‑distance‑field based penetration avoidance, proximity to the target, and task‑specific scores; (b) a two‑stage evaluation where thousands of candidate configurations are screened with a rigid‑body surrogate before a small set is validated in a soft‑body simulator. After the pre‑contact pose is fixed, a sequence of motion primitives is executed via closed‑loop Cartesian controllers that adapt online to privileged simulator feedback such as internal object stress and centroid motion, allowing phase transitions (e.g., approach‑to‑grasp) to be triggered safely.

The upper level then searches the latent design space with CMA‑ES, maximizing an objective J(d,π*​(d)) that explicitly trades off task success against internal stress (object integrity). This bi‑level pipeline couples morphology and control, enabling the optimizer to exploit shape‑induced changes in contact geometry and force distribution.

Third, to bridge simulation and reality without relying on tactile sensors, the authors employ a teacher‑student policy distillation. The privileged teacher policy, which has access to simulator stress signals, generates rollouts on the optimized design. Point‑cloud observations from these rollouts are used to train a diffusion‑based student policy that operates solely on raw point clouds. Domain randomization ensures that the student policy can be deployed zero‑shot on a real robot, preserving the gentle manipulation behavior learned in simulation.

Experiments focus on food‑handling tasks such as grasping, pushing, and scooping jelly and fish fillets. In both simulation and real‑world trials, the co‑designed end‑effectors achieve significantly higher success rates and reduce object damage by up to 70 % compared to baseline rigid or soft grippers. The student policy matches the teacher’s performance despite lacking privileged information.

Overall, the paper delivers a unified framework that (1) provides an expressive yet low‑dimensional shape parameterization, (2) integrates non‑gradient‑based design search with model‑based adaptive control, and (3) enables sensor‑free sim‑to‑real transfer via policy distillation. The approach is broadly applicable to domains requiring delicate handling of deformable or fragile items, such as surgical robotics, caregiving, and advanced manufacturing. Future work may explore higher‑dimensional design spaces, multimodal sensing, and hybrid reinforcement‑learning co‑design to further enhance performance and generality.