CVPR 2020 (Oral)

Robust 3D Self-portraits in Seconds

 

Zhe Li, Tao Yu, Chuanyu Pan, Zerong Zheng, Yebin Liu

Tsinghua University

 

Abstract

In this paper, we propose an efficient method for robust 3D self-portraits using a single RGBD camera. Benefiting from the proposed PIFusion and lightweight bundle adjustment algorithm, our method can generate detailed 3D self-portraits in seconds and shows the ability to handle extremely loose clothes. To achieve highly efficient and robust reconstruction, we contribute PIFusion, which combines learning-based 3D recovery with volumetric non-rigid fusion to generate accurate sparse partial scans of the performer. Moreover, a non-rigid volumetric deformation method is proposed to continuously refine the learned shape prior. Finally, a lightweight bundle adjustment algorithm is proposed to guarantee that all the partial scans can not only ``loop'' with each other, but also keep consistent with the selected live key observations. Results and experiments show that the proposed method achieves more robust and efficient 3D self-portraits compared with state-of-the-art methods.

 

 

Fig 1. Overview of our method.

 


Results

 

 

Fig 2. System setup and live demo.

 

 

 

Fig 3. Example 3D portraits acquired by our system.

 


Technical Paper

 


Oral Presentation

 


Supplementary Video

 


Citation

Zhe Li, Tao Yu, Chuanyu Pan, Zerong Zheng, Yebin Liu. "Robust 3D Self-portraits in Seconds". CVPR 2020

 

@InProceedings{Li2020portrait,
  author = {Li, Zhe and Yu, Tao and Pan, Chuanyu and Zheng, Zerong and Liu, Yebin},
  title = {Robust 3D Self-portraits in Seconds},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  month={June},
  year={2020},
}