MonoNPHM: Dynamic Head Reconstruction from Monocular Videos @MatthiasNiessner
MonoNPHM: Dynamic Head Reconstruction from Monocular Videos  @MatthiasNiessner
Uploaded December 2023 | Updated November 2026, 1 hour ago
Project: simongiebenhain.github.io/MonoNPHM

We present Monocular Neural Parametric Head Models (MonoNPHM) for dynamic 3D head reconstructions from monocular RGB videos. To this end, we propose a latent appearance space that parameterizes a texture field on top of a neural parametric model. We constrain predicted color values to be correlated with the underlying geometry such that gradients from RGB effectively influence latent geometry codes during inverse rendering. To increase the representational capacity of our expression space, we augment our backward deformation field with hyper-dimensions, thus improving color and geometry representation in topologically challenging expressions. Using MonoNPHM as a learned prior, we approach the task of 3D head reconstruction using signed distance field based volumetric rendering. By numerically inverting our backward deformation field, we incorporated a landmark loss using facial anchor points that are closely tied to our canonical geometry representation. To evaluate the task of dynamic face reconstruction from monocular RGB videos we record 20 challenging Kinect sequences under casual conditions. MonoNPHM outperforms all baselines with a significant margin, and makes an important step towards easily accessible neural parametric face models through RGB tracking.
MonoNPHM: Dynamic Head Reconstruction from Monocular VideosInverse Path Tracing for Joint Material and Lighting Estimation (CVPR 2019 Oral)TUM AI Lecture Series - Neural Fields Beyond Novel View Synthesis (Andrea Tagliasacchi)DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head AvatarsDeleted videoNeural Non-Rigid Tracking (NeurIPS 2020)TUM AI Lecture Series - FLUX: Flow Matching for Content Creation at Scale (Robin Rombach)TUM AI Lecture Series - Dream Machine: Emergent Capabilities from Video Models (Jiaming Song)Introduction to Deep Learning - 12. Advanced Deep Learning Topics (Summer 2020)Deleted videoIntroduction to Deep Learning (I2DL 2023) - 1. IntroductionDiffRF: Rendering-guided 3D Radiance Field Diffusion (CVPR2023)
Matthias Niessner |

MonoNPHM: Dynamic Head Reconstruction from Monocular Videos

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER