3D/4D Reconstruction and Markerless Mocap
DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor
CVPR 2018 (oral) Project Page
We propose DoubleFusion, a new real-time system that combines volumetric dynamic reconstruction with datadriven template fitting to simultaneously reconstruct detailed geometry, non-rigid motion and the inner human body shape from a single depth camera.
Structure from Recurrent Motion: From Rigidity to Recurrency
CVPR 2018
This paper proposes a new method for Non-Rigid Structure-from-Motion (NRSfM), to exploit the property of shape recurrence (i.e., many deforming shapes tend to repeat themselvesintime).
BodyFusion: Real-time Capture of Human Motion and Surface Geometry Using a Single Depth Camera
ICCV 2017 Project Page
We propose BodyFusion, a novel real-time geometry fusion method that can track and reconstruct non-rigid surface motion of a human performance using a single consumer-grade depth camera.
SurfaceNet: an End-to-end 3D Neural Network for Multiview Stereopsis
ICCV 2017
SurfaceNet takes a set of images and their corresponding camera parameters as input and directly infers the 3D model.
Real-time Geometry, Albedo and Motion Reconstruction Using a Single RGBD Camera
TOG 2017 (present in SIGGRAPH 2017) Project Page
This paper proposes a real-time method that uses a single-view RGBD input to simultaneously reconstruct a casual scene with a detailed geometry model, surface albedo, per-frame non-rigid motion and per-frame low-frequency lighting, without requiring any template or motion priors.
FlyCap: Markerless Motion Capture Using Multiple Autonomous Flying Cameras
IEEE Trans. VCG 2017 Project Page
This paper presents a new generation markerless motion capture technique, the FlyCap system, to capture surface motions of moving characters using multiple autonomous flying cameras.
Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization
IEEE Trans. VCG 2017 Project Page Source Code
We present a new motion tracking method to robustly reconstruct non-rigid geometries and motions from single view depth inputs captured by a consumer depth sensor.
Outdoor Markerless Motion Capture with Sparse Handheld Video Cameras
IEEE Trans. VCG 2017 Project Page
We present a method for outdoor markerless motion capture with a few handheld cameras.
Video-based Outdoor Human Reconstruction
IEEE Trans. CSVT 2017
A system using a single RGB video camera for outdoor human body scanning is proposed.
SPA: Sparse Photorealistic Animation Using a Single RGB-D Camera
IEEE Trans. CSVT 2017
We propose a new method to synthesize plausible videos of human actors with new motions using a single cheap RGB-D camera.
Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization
ICCV 2015 Project Page Source Code
We present a new motion tracking method to robustly reconstruct non-rigid geometries and motions from single view depth inputs captured by a consumer depth sensor.
Free-viewpoint Video Relighting from Multi-view Sequence under General Illumination
Machine Vision and Application 2014
We proposed an approach to create plausible freeviewpoint relighting video using multi-view cameras array under general illumination.
Markerless Motion Capture of Multiple Characters Using Multi-view Image Segmentation
IEEE Trans.PAMI 2013 (Spotlight Paper) Project Page
We present a markerless motion capture approach that reconstructs the skeletal motion and detailed time-varying surface geometry of multiple people from multi-view video.
Video-based Manipulation Capture Through Composite Motion Control
SIGGRAPH 2013 Project Page
This paper introduces physics-based motion control for acquiring physically realistic hand grasping and manipulation data from multiple video streams.
Capturing Relightable Human Performances under General Uncontrolled Illumination
Eurographics 2013 Project Page
We present a novel approach to create relightable free-viewpoint human performances from multi-view video recorded under general uncontrolled and uncalibated illumination.
Performance Capture of Interacting Characters with Handheld Kinects
ECCV 2012 & IEEE Trans. SMC-B 2013 Project Page
We present an algorithm for marker-less performance capture of interacting humans using only three hand-held Kinect cameras.
Performance Capture of High-Speed Motion Using Staggered Multi-view Recording
Computer Graphics Forum (PG) 2012 Project Page
We present a markerless performance capture system that can acquire the motion and the texture of human actors performing fast movements using only commodity hardware.
Noisy Depth Maps Fusion for Multi-view Stereo via Matrix Completion
IEEE Journal of Selected Topics in Signal Processing 2012
This paper introduces a matrix completion framework to fuse noisy point clouds from multi-view images of the same object.
Shading-based Dynamic Shape Refinement from Multi-view Video under General Illumination
ICCV 2011
We present an approach to add true fine-scale spatiotemporal shape detail to dynamic scene geometry captured from multi-view video footage.
Video-based Characters - Creating New Human Performances from a Multi-view Video Database
SIGGRAPH 2011 Project Page
We present a data driven method to synthesize plausible video sequences of humans according to user-defined body motions and viewpoints.
Markerless Motion Capture of Interacting Characters Using Multi-view Image Segmentation
CVPR 2011 (Oral, accept rate: 3.5%) Project Page
We present a markerless motion capture approach that reconstructs the skeletal motion and detailed time-varying surface geometry of two closely interacting people from multi-view video.
Dynamic Shape Capture via Periodical-illumination Optical Flow Estimation and Multi-view Photometric Stereo
3DIMPVT 2011
We present an optical flow estimation works under periodically varying illuminations, and in cooperation with photometric stereo, enables high-quality 3D reconstruction of dynamic objects.
Fusing Multiview and Photometric Stereo for 3D Reconstruction under Uncalibrated Illumination
IEEE Trans. VCG 2011 Project Page
We propose a method to obtain a complete and accurate 3D model from multi-view images captured under a variety of unknown illuminations.
A Point-Cloud-Based Multi-View Stereo Algorithm For Free-viewpoint Video
IEEE Trans. VCG 2010 Project Page
We propose a point cloud based multi-view stereo algorithm for free-viewpoint video.
Continuous Depth Estimation for Multi-view Stereo
CVPR 2009 Project Page
We propose a method to obtain a complete and accurate 3D model from multi-view images captured under a variety of unknown illuminations.
A Multi-camera and Multi-Lighting Dome for 3D Reconstruction and Relighting
ICME09a ICME09b ICME09c Project Page
We construct a dome to record the geometry, texture and motion of human actors in a dedicated multiple-camera studio with controlled lighting and a chromakey backgrounds.