GarmentTracking: Category-Level Garment Pose Tracking 

Han Xue, Wenqiang Xu, Jieyi Zhang, Tutian Tang, Yutong Li, Wenxin Du, Ruolin Ye, Cewu Lu

Shanghai Jiao Tong University

CVPR 2023

Abstract: Garments are important to humans. A visual system that can estimate and track the complete garment pose can be useful for many downstream tasks and real-world applications. In this work, we present a complete package to address the category-level garment pose tracking task: (1) A recording system VR-Garment, with which users can manipulate virtual garment models in simulation through a VR interface. (2) A large-scale dataset VR-Folding, with complex garment pose configurations in manipulation like flattening and folding. (3) An end-to-end online tracking framework GarmentTracking, which predicts complete garment pose both in canonical space and task space given a point cloud sequence. Extensive experiments demonstrate that the proposed GarmentTracking achieves great performance even when the garment has large non-rigid deformation. It outperforms the baseline approach on both speed and accuracy. We hope our proposed solution can serve as a platform for future research.

Shirt Flattening

Input

Baseline

Ours

GT

Pants Flattening

Input

Baseline

Ours

GT

Top Flattening

Input

Baseline

Ours

GT

Skirt Flattening

Input

Baseline

Ours

GT

Shirt Folding

Input

Baseline

Ours

GT

Pants Folding

Input

Baseline

Ours

GT

Top Folding

Input

Baseline

Ours

GT

Skirt Folding

Input

Baseline

Ours

GT