DPHMs: Diffusion Parametric Head Models for Depth-based Tracking @MatthiasNiessner
DPHMs: Diffusion Parametric Head Models for Depth-based Tracking  @MatthiasNiessner
Uploaded December 2023 | Updated November 2026, 10 hours ago
Project: tangjiapeng.github.io/projects/DPHMs

We introduce Diffusion Parametric Head Models (DPHMs), a generative model that enables robust volumetric head reconstruction and tracking from monocular depth sequences. While recent volumetric head models, such as NPHMs, can now excel in representing high-fidelity head geometries, tracking and reconstruction heads from real-world single-view depth sequences remains very challenging, as the fitting to partial and noisy observations is underconstrained. To tackle these challenges, we propose a latent diffusion-based prior to regularize volumetric head reconstruction and tracking. This prior-based regularizer effectively constrains the identity and expression codes to lie on the underlying latent manifold which represents plausible head shapes. To evaluate the effectiveness of the diffusion-based prior, we collect a dataset of monocular Kinect sequences consisting of various complex facial expression motions and rapid transitions. We compare our method to state-of-the-art tracking methods, and demonstrate improved head identity reconstruction as well as robust expression tracking.
DPHMs: Diffusion Parametric Head Models for Depth-based TrackingHyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion (ICCV2023)FaceForensics++: Learning to Detect Manipulated Facial Images (ICCV 2019)Introduction to Deep Learning (I2DL 2023) - 6. Training Neural NetworksBecomingLit: Relightable Gaussian Avatars with Hybrid Neural ShadingIntroduction to Deep Learning - 11. Recurrent Neural Networks (Summer 2020)TUM AI Lecture Series - I2DL Guest Lecture 2023 - 2D Priors for 3D Generation  (Ben Poole)TUM AI Lecture Series - A Question of Representation in 3D Computer Vision (Bharath Hariharan)3D Scanning & Motion Capture: 4. OptimizationDeferred Neural Rendering: Image Synthesis using Neural Textures (SIGGRAPH 2019)GaussianGPT: Towards Autoregressive 3D Gaussian Scene GenerationDeleted video
Matthias Niessner |

DPHMs: Diffusion Parametric Head Models for Depth-based Tracking

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER