DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars @MatthiasNiessner
DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars  @MatthiasNiessner
Uploaded November 2023 | Updated November 2026, 4 days ago
Project: tobias-kirschstein.github.io/diffusion-avatars

DiffusionAvatars synthesizes a high-fidelity 3D head avatar of a person, offering intuitive control over both pose and expression. We propose a diffusion-based neural renderer that leverages generic 2D priors to produce compelling images of faces. For coarse guidance of the expression and head pose, we render a neural parametric head model (NPHM) from the target viewpoint, which acts as a proxy geometry of the person. Additionally, to enhance the modeling of intricate facial expressions, we condition DiffusionAvatars directly on the expression codes obtained from NPHM via cross-attention. Finally, to synthesize consistent surface details across different viewpoints and expressions, we rig learnable spatial features to the head’s surface via TriPlane lookup in NPHM’s canonical space. We train DiffusionAvatars on RGB videos and corresponding tracked NPHM meshes of a person and test the obtained avatars in both self-reenactment and animation scenarios. Our experiments demonstrate that DiffusionAvatars generates temporally consistent and visually appealing videos for novel poses and expressions of a person, outperforming existing approaches.
DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head AvatarsDeleted videoNeural Non-Rigid Tracking (NeurIPS 2020)TUM AI Lecture Series - FLUX: Flow Matching for Content Creation at Scale (Robin Rombach)TUM AI Lecture Series - Dream Machine: Emergent Capabilities from Video Models (Jiaming Song)Introduction to Deep Learning - 12. Advanced Deep Learning Topics (Summer 2020)Deleted videoIntroduction to Deep Learning (I2DL 2023) - 1. IntroductionDiffRF: Rendering-guided 3D Radiance Field Diffusion (CVPR2023)World Reconstruction from Inconsistent ViewsTUM AI Lecture Series - Deep Surface Meshes (Pascal Fua)SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scans (CVPR)
Matthias Niessner |

DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER