Face Anything: 4D Face Reconstruction from Any Image Sequence @MatthiasNiessner
Face Anything: 4D Face Reconstruction from Any Image Sequence  @MatthiasNiessner
Uploaded April 2026 | Updated November 2026, 2 days ago
kocasariumut.github.io/FaceAnything

Accurate reconstruction and tracking of dynamic human faces from image sequences is challenging because non-rigid deformations, ex pression changes, and viewpoint variations occur simultaneously, creating significant ambiguity in geometry and correspondence estimation. We present a unified method for high-fidelity 4D facial reconstruction based on canonical facial point prediction, a representation that assigns each pixel a normalized facial coordinate in a shared canonical space. This formulation transforms dense tracking and dynamic reconstruction into
a canonical reconstruction problem, enabling temporally consistent geometry and reliable correspondences within a single feed-forward model. By jointly predicting depth and canonical coordinates, our method enables accurate depth estimation, temporally stable reconstruction, dense
3D geometry, and robust facial point tracking within a single architecture. We implement this formulation using a transformer-based model that jointly predicts depth and canonical facial coordinates, trained using multi-view geometry data that non-rigidly warps into the canonical
space. Extensive experiments on image and video benchmarks demonstrate state-of-the-art performance across reconstruction and tracking tasks, achieving approximately 3× lower correspondence error and faster inference than prior dynamic reconstruction methods, while improving depth accuracy by 16%. These results highlight canonical facial point prediction as an effective foundation for unified feed-forward 4D facial reconstruction.
Face Anything: 4D Face Reconstruction from Any Image SequenceDPHMs: Diffusion Parametric Head Models for Depth-based TrackingHyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion (ICCV2023)FaceForensics++: Learning to Detect Manipulated Facial Images (ICCV 2019)Introduction to Deep Learning (I2DL 2023) - 6. Training Neural NetworksBecomingLit: Relightable Gaussian Avatars with Hybrid Neural ShadingIntroduction to Deep Learning - 11. Recurrent Neural Networks (Summer 2020)TUM AI Lecture Series - I2DL Guest Lecture 2023 - 2D Priors for 3D Generation  (Ben Poole)TUM AI Lecture Series - A Question of Representation in 3D Computer Vision (Bharath Hariharan)3D Scanning & Motion Capture: 4. OptimizationDeferred Neural Rendering: Image Synthesis using Neural Textures (SIGGRAPH 2019)GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generation
Matthias Niessner |

Face Anything: 4D Face Reconstruction from Any Image Sequence

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER