GarmentTracking: Category-Level Garment Pose Tracking
Han Xue, Wenqiang Xu, Jieyi Zhang, Tutian Tang, Yutong Li, Wenxin Du, Ruolin Ye, Cewu Lu
Shanghai Jiao Tong University
CVPR 2023
Abstract: Garments are important to humans. A visual system that can estimate and track the complete garment pose can be useful for many downstream tasks and real-world applications. In this work, we present a complete package to address the category-level garment pose tracking task: (1) A recording system VR-Garment, with which users can manipulate virtual garment models in simulation through a VR interface. (2) A large-scale dataset VR-Folding, with complex garment pose configurations in manipulation like flattening and folding. (3) An end-to-end online tracking framework GarmentTracking, which predicts complete garment pose both in canonical space and task space given a point cloud sequence. Extensive experiments demonstrate that the proposed GarmentTracking achieves great performance even when the garment has large non-rigid deformation. It outperforms the baseline approach on both speed and accuracy. We hope our proposed solution can serve as a platform for future research.
Code: https://github.com/xiaoxiaoxh/GarmentTracking
Dataset: here
VR-Garment Recording System: https://github.com/xiaoxiaoxh/VR-Garment
Poster: here
Shirt Flattening
Input
Baseline
Ours
GT
Pants Flattening
Input
Baseline
Ours
GT
Top Flattening
Input
Baseline
Ours
GT
Skirt Flattening
Input
Baseline
Ours
GT
Shirt Folding
Input
Baseline
Ours
GT
Pants Folding
Input
Baseline
Ours
GT
Top Folding
Input
Baseline
Ours
GT
Skirt Folding
Input
Baseline
Ours
GT