A curated list of resources on simulating deformable objects (cloth, rope, soft bodies) for embodied intelligence — simulation environments, manipulation tasks, sim-to-real transfer, and asset generation.
Project website | English | 中文
- Simulation Environments & Platforms
- Deformable Object Manipulation Tasks
- Sim-to-Real Transfer
- Benchmarks & Evaluation
- 3D Asset Generation for Simulation
- Foundational Simulation Techniques
- Legend
- RFUniverse: A Multiphysics Simulation Platform for Embodied AI - Multiphysics environment for household robot learning, including soft-body interactions, fluids, airflow, and heat transfer. 📄 🔧
- SAPIEN: A SimulAted Part-based Interactive ENvironment - Interactive simulation environment based on PhysX, excelling at articulated and deformable object simulation. Widely used in Embodied AI research. 📄 🔧 ⭐
- Genesis World - Multi-physics simulation platform covering rigid, deformable, and fluid bodies. 29k+ GitHub stars. 🔧 ⭐
- SAPIEN ManiSkill - Unified framework for manipulation skill learning built on SAPIEN, with rich deformable object tasks. 🔧 📊 ⭐
- Isaac Sim / Isaac Lab - NVIDIA's robotics simulation platform with GPU-accelerated deformable body support for large-scale Embodied AI training. 🔧 ⭐
- MuJoCo - Advanced physics engine with deformable body simulation support (since v3.x), widely adopted in robotics research. 🔧 ⭐
- Taichi - High-performance parallel computing language, popular for differentiable soft body simulation in Embodied AI research. 🔧
- Real2Render2Real: Scaling Robot Data Without Dynamics Simulation or Robot Hardware - Scales robot training data via a Real→Render→Real pipeline without dynamics simulation or physical robot hardware. 📄 ⭐
- DexGarmentLab: Dexterous Garment Manipulation Environment with Generalizable Policy - Bimanual dexterous garment simulation with structural-correspondence demonstration generation and a hierarchical manipulation policy. 📄 📊
- DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physics - Learns dexterous soft-object skills from human demonstrations and refines planned actions with differentiable physics. 📄
- GraphGarment: Learning Garment Dynamics for Bimanual Cloth Manipulation Tasks - Learns garment dynamics in simulation with graph networks and uses a residual model for real-world bimanual garment hanging. 📄
- GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation - Unified simulation and benchmark for garment manipulation tasks — folding, hanging, dressing. 📄 📊 ⭐
- ClothesNet: An Information-Rich 3D Garment Model Repository with Simulated Clothes Environment - Large-scale 3D garment dataset with rich annotations for manipulation research. 📄 📊
- RAPID: Rapid Adaptation of Particle Dynamics for Generalized Deformable Object Mobile Manipulation - Rapid adaptation of particle dynamics for generalized deformable object mobile manipulation. 📄 ⭐
- Benchmarking the Sim-to-Real Gap in Cloth Manipulation - Systematic evaluation of the sim-to-real gap in cloth manipulation across different simulators. 📄 📊
- SoftMimicGen: A Data Generation System for Scalable Robot Learning in Deformable Object Manipulation - Extends MimicGen to deformable object manipulation, enabling scalable robot demonstration data generation. 📄 ⭐
- DeformGen: Dynamics-Based Topology Augmentation for Deformable Manipulation Policy Learning - Dynamics-based topology augmentation for deformable object manipulation policy learning, built on PhysTwin and Real2Render2Real. 📄
- SimWeaver: Zero-Shot RGB Sim-to-Real for Deformable Manipulation - Combines measurement-backed cloth simulation, topology-aware trajectory synthesis, and photometric augmentation for RGB policy transfer. 📄
- SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds - Real-to-sim-to-real data generation with scene digitization, calibrated deformable dynamics, and filtered trajectory synthesis. 📄
- Sim-to-Real Gentle Manipulation of Deformable and Fragile Objects with Stress-Guided Reinforcement Learning - Uses stress-penalized rewards and a rigid-to-deformable curriculum to transfer gentle manipulation policies to fragile real objects. 📄
- Learning to Manipulate Deformable Objects in the Real World via Sim2Real - End-to-end sim2real pipeline for deformable object manipulation with domain randomization. 📄
- DaXBench: Benchmarking Deformable Object Manipulation with Differentiable Physics - JAX-based differentiable benchmark covering cloth, rope, and fluids for planning, imitation learning, and reinforcement learning. 📄 📊
- Real Garment Benchmark (RGBench): A Comprehensive Benchmark for Robotic Garment Manipulation featuring a High-Fidelity Scalable Simulator - Combines garment meshes, a cloth simulator, and real-motion measurements to evaluate physical simulation fidelity. 📄 📊
- MoDeSuite: Robot Learning Task Suite for Benchmarking Mobile Manipulation with Deformable Objects - Mobile manipulation task suite for deformable objects, with reinforcement and imitation learning baselines and real-robot transfer experiments. 📄 📊
- SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects (Early Version) - Early-version visuo-tactile benchmark using FEM deformation states to distinguish task success from physically safe manipulation. 📄 📊
- Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions - Reconstructs soft-body digital twins from videos and evaluates robot policies using physics simulation and Gaussian splatting rendering. 📄
- PokeFlex: A Real-World Dataset of Volumetric Deformable Objects for Robotics - Real-world multimodal deformation data with meshes, RGB-D observations, and interaction forces for learning and validating deformable models. 📄 📊
- SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation - Benchmark suite for deformable object manipulation (cloth, rope, fluid) with standardized RL environments. 📄 📊 ⭐
- EMPM: Embodied MPM for Modeling and Simulation of Deformable Objects - Reconstructs deformable objects from RGB-D videos and identifies material parameters with differentiable MPM and online sensory feedback. 📄
- NeuSpring: Neural Spring Fields for Reconstruction and Simulation of Deformable Objects from Videos - Reconstructs deformable digital twins using piecewise spring topology and neural spring fields for spatially varying physical properties. 📄
- Image2Garment: Simulation-ready Garment Generation from a Single Image - Predicts garment geometry and fabric physics from a single image using material attributes and measured material-to-physics mappings. 📄
- PhysX-3D: Physical-Grounded 3D Asset Generation - End-to-end paradigm for physical-grounded 3D asset generation with PhysXNet dataset. 📄 ⭐
- PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Image - Generate simulation-ready 3D assets with physical properties from a single image. 📄
- PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects - First to include deformable objects in PhysX generation scope, unifying rigid/deformable/articulated. 📄 ⭐
- DiffGI: Differentiable Geometry Images - 3D garment generation via differentiable geometry images (no integrated physics yet). 📄
- PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos - Physics-informed reconstruction and simulation of deformable objects from video; a precursor to Real2Render2Real and DeformGen. 📄 ⭐
- Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video - Reconstructs simulation-ready, photorealistic standalone garment assets from multi-view video. 📄
- DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact - Differentiable cloth simulation with dry frictional contact for parameter identification, assisted dressing, and control. 📄 🔧
- AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation - Material-conditioned graph dynamics with online physical-property estimation for adapting manipulation to unfamiliar deformable materials. 📄
- Learning Mesh-Based Simulation with Graph Networks - MeshGraphNets learns mesh-based dynamics, including cloth, with adaptive discretization and mesh-space and world-space interactions. 📄
- XRTailor (OpenXRLab) - GPU-accelerated cloth simulation engine for large-scale data generation. 🔧
- Position Based Dynamics (PBD) - Classic introductory resource for Position Based Dynamics simulation. 🔧
- ThinShellLab: Thin-Shell Object Manipulations With Differentiable Physics Simulations - Fully differentiable simulation platform for thin-shell materials (paper, cloth) with varying bending stiffness. 📄 🔧
- Second-Order FEM for Deformable Surfaces - High-order finite element method for improving accuracy in cloth / thin-shell simulation. 📄
| Tag | Meaning |
|---|---|
| 📄 | Paper |
| 🔧 | Tool / Framework / Engine |
| 📊 | Benchmark / Dataset |
| ⭐ | Recommended / Important |
Contributions welcome! Please read the contribution guidelines first.
Quick way to add a paper:
- Edit
papers.yml— add your entry under the appropriate section - Run
python generate.py - Submit a Pull Request