GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion @MatthiasNiessner
GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion  @MatthiasNiessner
Uploaded December 2024 | Updated November 2026, 5 days ago
Project: tangjiapeng.github.io/projects/GAF

We propose a novel approach for reconstructing animatable 3D Gaussian avatars from monocular videos captured by commodity devices like smartphones. Photorealistic 3D head avatar reconstruction from such recordings is challenging due to limited observations, which leaves unobserved regions under-constrained and can lead to artifacts in novel views. To address this problem, we introduce a multi-view head diffusion model, leveraging its priors to fill in missing regions and ensure view consistency in Gaussian splatting renderings. To enable precise viewpoint control, we use normal maps rendered from FLAME-based head reconstruction, which provides pixel-aligned inductive biases. We also condition the diffusion model on VAE features extracted from the input image to preserve details of facial identity and appearance. For Gaussian avatar reconstruction, we distill multi-view diffusion priors by using iteratively denoised images as pseudo-ground truths, effectively mitigating over-saturation issues. To further improve photorealism, we apply latent upsampling to refine the denoised latent before decoding it into an image. We evaluate our method on the NeRSemble dataset, showing that GAF outperforms the previous state-of-the-art methods in novel view synthesis and novel expression animation. Furthermore, we demonstrate higher-fidelity avatar reconstructions from monocular videos captured on commodity devices.
GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view DiffusionDeleted videoECCV 2020 Paper Compilation - TUM Visual Computing Lab & CollaboratorsDeleted video3D Scanning & Motion Capture: 8. Parametric Face ModelsDeleted videoDeleted video2nd ScanNet Indoor Scene Understanding Workshop (CVPR20)ViewDiff: 3D-Consistent Image Generation with Text-To-Image ModelsMesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation (ECCV24)CVPR 2020 Paper Compilation - TUM Visual Computing Lab & CollaboratorsScanRefer: 3D Object Localization in RGB-D Scans using Natural Language (ECCV 2020)
Matthias Niessner |

GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER