Reseach Agenda:

We target top-tier research at the intersection of computer vision, computer graphics and machine learning. Our research mission is to capture, perceive and understand the human-centric dynamic and static scenes in the complex real world. Our goal is to digitalize humans, objects and events, and eventually to enable realistic and immersive tele-presence in virtual reality and augmented reality.

(# for the corresponding author, * for equal contributors)

2021

SportsCap: Monocular 3D Human Motion Capture and Fine-grained Understanding in Challenging Sports Videos

Xin Chen,  Anqi Pang,  Wei Yang,  Yuexin MaLan Xu#  and Jingyi Yu#.

International Journal of Computer Vision (IJCV), Accepted, 2021.

TightCap: 3D Human Shape Capture with Clothing Tightness

Xin Chen*,  Anqi Pang*,  Wei Yang,  Yuexin MaLan Xu#  and Jingyi Yu#.

ACM Transactions of Graphics (TOG), Accepted, 2021.

Neural Video Portrait Relighting in Real-time via Consistency Modeling

Longwen Zhang,  Qixuan Zhang,  Minye WuJingyi Yu  and Lan Xu.

International Conference on Computer Vision (ICCV), 2021.

GNeRF: GAN-based Neural Radiance Field without Posed Camera

Quan Meng,  Anpei Chen,  Haimin Luo,  Minye WuHao SuLan XuXumin He  and Jingyi Yu.

International Conference on Computer Vision (ICCV), 2021. Oral

iButter: Neural Interactive Bullet Time Generator for Human Free-viewpoint Rendering

Liao Wang,  Ziyu Wang,  Pei Lin,  Yuheng Jiang,  Xin Suo,  Minye WuLan Xu  and Jingyi Yu.

Proceedings of the 29th ACM international conference on Multimedia (ACMMM), 2021. Oral

Neural Free-Viewpoint Performance Rendering under Complex Human-object Interactions

Guoxing Sun,  Xin Chen,  Yizhang Chen,  Anqi Pang,  Pei Lin,  Yuheng Jiang,  Lan XuJingyi Yu  and Jingya Wang.

Proceedings of the 29th ACM international conference on Multimedia (ACMMM), 2021. Oral

Towards Controllable and Photorealistic Region-wise Image Manipulation

Ansheng You,  Chenglin Zhou,  Qixuan Zhang  and Lan Xu.

Proceedings of the 29th ACM international conference on Multimedia (ACMMM), 2021.

IREM: High-Resolution Magnetic Resonance (MR) Image Reconstruction via Implicit Neural Representation

Qing Wu,  Yuwei LiLan Xu,  Ruimin Feng,  Hongjiang Wei,  Qing Yang,  Boliang Yu,  Xiaozhao Liu,  Qing Yang,  Jingyi Yu  and Yuyao Zhang.

International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), 2021.

PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging

Yuwei LiMinye WuYuyao ZhangLan Xu  and Jingyi Yu.

International Joint Conferences on Artificial Intelligence Organization (IJCAI), 2021.

Few-shot Neural Human Performance Rendering from Sparse RGBD Videos

Anqi Pang,  Xin Chen,  Haimin Luo,  Minye WuJingyi Yu   and Lan Xu.

International Joint Conferences on Artificial Intelligence Organization (IJCAI), 2021.

Editable free-viewpoint video using a layered neural representation

Jiakai Zhang,  Xinhang Liu,  Xinyi Ye,  Fuqiang Zhao,  Yanshun Zhang,  Minye Wu,  Yingliang Zhang,  Lan Xu#  and Jingyi Yu#.

ACM Transactions on Graphics (Proc. of SIGGRAPH), 2021.

ChallenCap: Monocular 3D Capture of Challenging Human Performances using Multi-Modal References

Yannan He,  Anqi Pang,  Xin Chen,  Han Liang,  Yuexin MaMinye Wu  and Lan Xu.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. Oral

NeuralHumanFVV: Real-Time Neural Volumetric Human Performance Rendering using RGB Cameras

Xin Suo,  Yuheng Jiang,  Pei Lin,  Yingliang Zhang,  Minye WuKaiwen Guo,  and Lan Xu.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

MirrorNeRF: One-shot Neural Portrait Radiance Field from Multi-mirror Catadioptric Imaging

Ziyu Wang,  Liao Wang,  Fuqiang Zhao,  Minye WuLan Xu and Jingyi Yu.

International Conference on Computational Photography (ICCP), 2021.

Convolutional Neural Opacity Radiance Fields

Haimin Luo,  Anpei Chen,  Qixuan Zhang,  Bai Pang,  Minye WuLan Xu and Jingyi Yu.

International Conference on Computational Photography (ICCP), 2021.

Boosting Single Image Super-Resolution Learnt From Implicit Multi-Image Prior

Dingjian Jin,  Mengqi JiLan Xu,nbsp Gaochang Wu,  Liejun Wang,  and Fang Lu.

IEEE Transactions on Image Processing (TIP), Accepted, 2021.

2020

BuildingFusion: Semantic-aware Structural Building-scale 3D Reconstruction

Tian Zheng,  Guoqing Zhang,  Lei HanLan Xu  and Lu Fang.

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Accepted, 2020.

Neural3D: Light-weight Neural Portrait Scanning via Context-aware Correspondence Learning

Xin Suo,  Minye Wu,  Yanshun Zhang,  Yingliang Zhang,  Qiang Hu,  Lan Xu  and Jingyi Yu.

Proceedings of the 28th ACM international conference on Multimedia (ACMMM), 2020.

RobustFusion: Human Volumetric Capture with Data-driven Visual Cues using a RGBD Camera

Zhuo Su*,  Lan Xu*Zerong ZhengTao YuYebin Liu   and Lu Fang.

European Conference on Computer Vision and Pattern Recognition (ECCV), 2020. Spotlight

EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera

Lan XuWeipeng XuVladislav GolyanikMarc HabermannLu Fang   and Christian Theobalt.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. Oral

OccuSeg: Occupancy-aware 3D Instance Segmentation

Lei Han,  Tian Zheng,  Lan Xu   and Lu Fang.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

Multiscale-VR: Multiscale Gigapixel 3D Panoramic Videography for Virtual Reality

Jianing Zhang,  Tianyi Zhu,  Anke Zhang,  Zihan Wang,  Xiaoyun Yuan,  Sebastian Beetschen,  Lan Xu,   Xing Lin,  Lu Fang  and Qionghai Dai.

International Conference on Computational Photography (ICCP), 2020.

Live Semantic 3D Perception for Immersive Augmented Reality

Lei Han,  Tian Zheng,  Yinheng Zhu,  Lan Xu   and Lu Fang.

IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR), Accepted, 2020.

2019

UnstructuredFusion: Realtime 4D Geometry and Texture Reconstruction using Commercial RGBD Cameras

Lan Xu,  Zhuo Su,  Lei HanTao YuYebin Liu  and Lu Fang.

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Accepted, 2019.

FlyFusion: Realtime Dynamic Scene Reconstruction Using a Flying Depth Camera

Lan Xu,  Wei Cheng,  Kaiwen GuoLei HanYebin Liu  and Lu Fang.

IEEE Transactions on Visualization and Computer Graphics (TVCG), Accepted, 2019.

Real-Time Global Registration for Globally Consistent RGB-D SLAM

Lei HanLan XuDmytro BobkovEckehard Steinbach  and Lu Fang.

IEEE Transactions on Robotics (TRO), Accepted, 2019.

2018

FlyCap: Markerless Motion Capture Using Multiple Autonomous Flying Cameras

Lan XuYebin Liu  Wei Cheng,  Kaiwen Guo,  Guyue Zhou,  Qionghai Dai  and Lu Fang.

IEEE Transactions on Visualization and Computer Graphics (TVCG), Accepted, 2018.

iHuman3D: Intelligent Human Body 3D Reconstruction using a Single Flying Camera

Wei Cheng*,  Lan Xu*Lei Han,  Yuanfang Guo  and Lu Fang.

Proceedings of the 26th ACM international conference on Multimedia (ACMMM), 2018. Oral

Beyond SIFT using binary features in loop closure detection

Lei Han,  Guyue Zhou,  Lan Xu,  and Lu Fang.

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017. Oral