NPGA: Neural Parametric Gaussian Avatars (SIGGRAPH Asia24) @MatthiasNiessner
NPGA: Neural Parametric Gaussian Avatars (SIGGRAPH Asia24)  @MatthiasNiessner
Uploaded September 2024 | Updated November 2026, 4 days ago
Project: simongiebenhain.github.io/NPGA
Paper: arxiv.org/abs/2405.19331

The creation of high-fidelity, digital versions of human heads is an important stepping stone in the process of further integrating virtual components into our everyday lives. Constructing such avatars is a challenging research problem, due to a high demand for photo-realism and real-time rendering performance. In this work, we propose Neural Parametric Gaussian Avatars (NPGA), a data-driven approach to create high-fidelity, controllable avatars from multi-view video recordings. We build our method around 3D Gaussian splatting for its highly efficient rendering and to inherit the topological flexibility of point clouds. In contrast to previous work, we condition our avatars' dynamics on the rich expression space of neural parametric head models (NPHM), instead of mesh-based 3DMMs. To this end, we distill the backward deformation field of our underlying NPHM into forward deformations which are compatible with rasterization-based rendering. All remaining fine-scale, expression-dependent details are learned from the multi-view videos. To increase the representational capacity of our avatars, we augment the canonical Gaussian point cloud using per-primitive latent features which govern its dynamic behavior. To regularize this increased dynamic expressivity, we propose Laplacian terms on the latent features and predicted dynamics. We evaluate our method on the public NeRSemble dataset, demonstrating that NPGA significantly outperforms the previous state-of-the-art avatars on the self-reenactment task by ~2.6PSNR. Furthermore, we demonstrate accurate animation capabilities from real-world monocular videos.
NPGA: Neural Parametric Gaussian Avatars (SIGGRAPH Asia24)ViewAL: Active Learning with Viewpoint Entropy for Semantic Segmentation (CVPR 2020)3DGS-LM: Faster Gaussian-Splatting Optimization with Levenberg-MarquardtClipFace: Text-guided Editing of Textured 3D Morphable Models (SIGGRAPH23)ICCV 2023 Paper Compilation - TUM Visual Computing Lab & CollaboratorsEnd2End Multi-View Feature Matching with Differentiable Pose Optimization (ICCV2023)SHeaP: Self-Supervised Head Geometry Predictor Learned via 2D GaussiansSIGGRAPH 2023 Paper Compilation - TUM Visual Computing Lab & CollaboratorsDeleted videoModeling 3D Shapes by Reinforcement Learning (ECCV 2020)TUM AI Lecture Series - Explainability and Compositionality for Visual Recognition (Zeynep Akata)Face Anything: 4D Face Reconstruction from Any Image Sequence
Matthias Niessner |

NPGA: Neural Parametric Gaussian Avatars (SIGGRAPH Asia'24)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER