Reseach Agenda:

We target top-tier research at the intersection of computer vision, computer graphics and machine learning. Our research mission is to capture, perceive and understand the human-centric dynamic and static scenes in the complex real world. Our goal is to digitalize humans, objects and events, and eventually to enable realistic and immersive tele-presence in virtual reality and augmented reality.


2020

EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera

We propose the first approach for 3D capturing of high-speed human motions using a single event camera. We can capture fast motions at millisecond resolution with significantly higher data efficiency.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. Oral
[Paper]   [Project Page]   [Video]   [arXiv]   [bibtex]

OccuSeg: Occupancy-aware 3D Instance Segmentation

We propose an occupancy-aware 3D instance segmentation scheme, which achieves state-of-the-art performance on 3 real-world datasets, while maintaining high efficiency.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[Paper(Coming soon)]  [Project Page(Coming soon)]   [Video]   [arXiv]   [bibtex]

Multiscale-VR: Multiscale Gigapixel 3D Panoramic Videography for Virtual Reality

We propose a VR camera which can zoom-in to local regions at a great distance away, allowing multi-scale, gigapixel-level, and 3D panoramic videography for VR content generation.

International Conference on Computational Photography (ICCP), 2020. Oral
[Paper(Coming soon)]  [Project Page(Coming soon)]   [Video]  

Live Semantic 3D Perception for Immersive Augmented Reality

We present a real-time simultaneous 3D reconstruction and semantic segmentation system working on mobile devices, with a live immersive AR demo, where the users can interact with the environment.

IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR), 2020.
[Paper]   [bibtex]


2019

UnstructuredFusion: Realtime 4D Geometry and Texture Reconstruction using Commercial RGBD Cameras

We propose UnstructuredFusion, which allows realtime, high-quality, complete reconstruction of 4D textured models of human performance via only three commercial RGBD cameras.

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019.
[Paper]   [Video]   [Project Page(Coming soon)]   [bibtex]

FlyFusion: Realtime Dynamic Scene Reconstruction Using a Flying Depth Camera

We explore active dynamic scene reconstruction based on a single flying camera, wihch can adaptively select the capture view targeting on real-time dynamic scene reconstruction.

IEEE Transactions on Visualization and Computer Graphics (TVCG), 2019.
[Paper]   [Video]   [Project Page(Coming soon)]   [bibtex]

Real-Time Global Registration for Globally Consistent RGB-D SLAM

We achieve globally consistent pose estimation in real-time via CPU computing, and owns comparable accuracy as state-of-the-art that use GPU computing, enabling the practical usage of globally consistent RGB-D SLAM.

IEEE Transactions on Robotics (TRO), 2019.
[Paper]   [bibtex]


2018

FlyCap: Markerless motion capture using multiple autonomous flying cameras

We propose to use three autonomous flying cameras for motion capture, which simultaneously performs non-rigid reconstruction and localization of the camera in each frame and each view.

IEEE Transactions on Visualization and Computer Graphics (TVCG), 2018.
[Paper]   [Video]   [Project Page(Coming soon)]   [bibtex]

iHuman3D: Intelligent Human Body 3D Reconstruction using a Single Flying Camera

In this work, we present an adaptive human body 3D reconstruction system using a single fl ying camera, which removes the extra manual labor constraint.

ACM International Conference on Multimedia (ACMMM), 2018. Oral
[Paper]   [bibtex]

Beyond SIFT using binary features in loop closure detection

A binary feature based LCD approach is presented in this paper, which achieves the highest accuracy compared with state-of-the-art while running at 30Hz on a laptop.

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018. Oral
[Paper]   [bibtex]