Video Diffusion Avatars: Taming Video Diffusion Models into Controllable 3D Avatars @MatthiasNiessner
Video Diffusion Avatars: Taming Video Diffusion Models into Controllable 3D Avatars  @MatthiasNiessner
Uploaded October 2026 | Updated November 2026, 3 hours ago
Project: simongiebenhain.github.io/video-diffusion-avatars

We leverage the world prior of existing video diffusion models (VDMs) to create 3D avatars from a single image, by carefully fine-tuning the VDM on public multi-view datasets and view-disentangled expression codes. VDAs let the user generate videos with free control over facial expressions, rendering camera and neck pose. The videos exhibit physically plausible dynamics and light interactions, such as reflections, shadows and refractions.
Video Diffusion Avatars: Taming Video Diffusion Models into Controllable 3D AvatarsIntrinsic Image Diffusion for Single-view Material EstimationDeleted videoDeleted videoMonoNPHM: Dynamic Head Reconstruction from Monocular VideosInverse Path Tracing for Joint Material and Lighting Estimation (CVPR 2019 Oral)TUM AI Lecture Series - Neural Fields Beyond Novel View Synthesis (Andrea Tagliasacchi)DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head AvatarsDeleted videoNeural Non-Rigid Tracking (NeurIPS 2020)TUM AI Lecture Series - FLUX: Flow Matching for Content Creation at Scale (Robin Rombach)TUM AI Lecture Series - Dream Machine: Emergent Capabilities from Video Models (Jiaming Song)
Matthias Niessner |

Video Diffusion Avatars: Taming Video Diffusion Models into Controllable 3D Avatars

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER