NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads (SIGGRAPH23) @MatthiasNiessner
NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads (SIGGRAPH23)  @MatthiasNiessner
Uploaded May 2023 | Updated November 2026, 1 day ago
Project: tobias-kirschstein.github.io/nersemble
Paper: arxiv.org/abs/2305.03027

We focus on reconstructing high-fidelity radiance fields of human heads, capturing their animations over time, and synthesizing re-renderings from novel viewpoints at arbitrary time steps. To this end, we propose a new multi-view capture setup composed of 16 calibrated machine vision cameras that record time-synchronized images at 7.1 MP resolution and 73 frames per second. With our setup, we collect a new dataset of over 4700 high-resolution, high-framerate sequences of more than 220 human heads, from which we introduce a new human head reconstruction benchmark. The recorded sequences cover a wide range of facial dynamics, including head motions, natural expressions, emotions, and spoken language. In order to reconstruct high-fidelity human heads, we propose Dynamic Neural Radiance Fields using Hash Ensembles (NeRSemble). We represent scene dynamics by combining a deformation field and an ensemble of 3D multi-resolution hash encodings. The deformation field allows for precise modeling of simple scene movements, while the ensemble of hash encodings helps to represent complex dynamics. As a result, we obtain radiance field representations of human heads that capture motion over time and facilitate re-rendering of arbitrary novel viewpoints. In a series of experiments, we explore the design choices of our method and demonstrate that our approach outperforms state-of-the-art dynamic radiance field approaches by a significant margin.
NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads (SIGGRAPH23)3D Scanning & Motion Capture: 5. Rigid ReconstructionIntrinsiX: High-Quality PBR Generation using Image PriorsIntroduction to Deep Learning (I2DL 2023) - 11. RNNs and TransformersLightIt: Illumination Modeling and Control for Diffusion Models (CVPR24)3D Scanning & Motion Capture: 0. IntroductionDeleted video3D Scanning & Motion Capture: 9. Face Tracking & ReconstructionIntroduction to Deep Learning (I2DL 2023) - 3. Intro Neural NetworksDeleted videoPBR-SR: Mesh PBR Texture Super Resolution from 2D Image PriorsLiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans
Matthias Niessner |

NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads (SIGGRAPH'23)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER