Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance

Qingcheng Zhao, Pengyu Long, Qixuan Zhang, Dafei Qin, Han Liang, Longwen Zhang, Yingliang Zhang, Jingyi Yu, Lan Xu

ACM SIGGRAPH, 2024

[Webpage | Paper | Arxiv | Video | Code]

OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllers

Han Liang, Jiacheng Bao, Ruichi Zhang, Sihan Ren, Yuecheng Xu, Sibei Yang, Xin Chen, Jingyi Yu, Lan Xu

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

[Webpage | Paper | Arxiv | Video | Code]

InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions

Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, Lan Xu

International Journal of Computer Vision (IJCV), 2024

[Webpage | Paper | Arxiv | Video | Code]

HybridCap: Inertia-aid monocular capture of challenging human motions

Han Liang, Yannan He, Chengfeng Zhao, Mutian Li, Jingya Wang, Jingyi Yu, Lan Xu

AAAI Conference on Artificial Intelligence (AAAI), 2023 [Oral]

[Webpage | Paper | Arxiv | Video]

LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors

Yiming Ren, Chengfeng Zhao, Yannan He, Peishan Cong, Han Liang, Jingyi Yu, Lan Xu, Yuexin Ma

IEEE Transactions on Visualization and Computer Graphics (TVCG), 2023

[Webpage | Paper | Arxiv | Video]

ChallenCap: Monocular 3d capture of challenging human performances using multi-modal references

Yannan He, Anqi Pang, Xin Chen, Han Liang, Minye Wu, Yuexin Ma, Lan Xu

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021 [Oral]

[Webpage | Paper | Arxiv | Video]

RhyCap: Sparse-view real-time full-body motion capture system

We propose a lightweight real-time markerless mocap system. With even only three consumer-grade web cameras, the system achieves close industry-level accuracy. The system has been integrated into the Bilibili live streaming pipeline.

[Demo]

RhyLive: Monocular full-body motion capture for real-time streaming

Achieving fine-grained capture of the upper body, face, and hands using a single camera. The system has been integrated into the Bilibili live streaming pipeline.

[Demo]