Intrinsic Image Diffusion for Single-view Material Estimation @MatthiasNiessner
Intrinsic Image Diffusion for Single-view Material Estimation  @MatthiasNiessner
Uploaded December 2023 | Updated November 2026, 3 hours ago
Project: peter-kocsis.github.io/IntrinsicImageDiffusion
Paper: arxiv.org/abs/2312.12274

We present Intrinsic Image Diffusion, a generative model for appearance decomposition of indoor scenes. Given a single input view, we sample multiple possible material explanations represented as albedo, roughness, and metallic maps. Appearance decomposition poses a considerable challenge in computer vision due to the inherent ambiguity between lighting and material properties and the lack of real datasets. To address this issue, we advocate for a probabilistic formulation, where instead of attempting to directly predict the true material properties, we employ a conditional generative model to sample from the solution space. Furthermore, we show that utilizing the strong learned prior of recent diffusion models trained on large-scale real-world images can be adapted to material estimation and highly improves the generalization to real images. Our method produces significantly sharper, more consistent, and more detailed materials, outperforming state-of-the-art methods by 1.5dB on PSNR and by 45% better FID score on albedo prediction. We demonstrate the effectiveness of our approach through experiments on both synthetic and real-world datasets.
Intrinsic Image Diffusion for Single-view Material EstimationDeleted videoDeleted videoMonoNPHM: Dynamic Head Reconstruction from Monocular VideosInverse Path Tracing for Joint Material and Lighting Estimation (CVPR 2019 Oral)TUM AI Lecture Series - Neural Fields Beyond Novel View Synthesis (Andrea Tagliasacchi)DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head AvatarsDeleted videoNeural Non-Rigid Tracking (NeurIPS 2020)TUM AI Lecture Series - FLUX: Flow Matching for Content Creation at Scale (Robin Rombach)TUM AI Lecture Series - Dream Machine: Emergent Capabilities from Video Models (Jiaming Song)Introduction to Deep Learning - 12. Advanced Deep Learning Topics (Summer 2020)
Matthias Niessner |

Intrinsic Image Diffusion for Single-view Material Estimation

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER