DiffRF: Rendering-guided 3D Radiance Field Diffusion (CVPR2023) @MatthiasNiessner
DiffRF: Rendering-guided 3D Radiance Field Diffusion (CVPR2023)  @MatthiasNiessner
Uploaded April 2023 | Updated November 2026, 3 hours ago
Project: sirwyver.github.io/DiffRF

We introduce DiffRF, a novel approach for 3D radiance field synthesis based on denoising diffusion probabilistic models. While existing diffusion-based methods operate on images, latent codes, or point cloud data, we are the first to directly generate volumetric radiance fields. To this end, we propose a 3D denoising model which directly operates on an explicit voxel grid representation. However, as radiance fields generated from a set of posed images can be ambiguous and contain artifacts, obtaining ground truth radiance field samples is non-trivial. We address this challenge by pairing the denoising formulation with a rendering loss, enabling our model to learn a deviated prior that favours good image quality instead of trying to replicate fitting errors like floating artifacts. In contrast to 2D-diffusion models, our model learns multi-view consistent priors, enabling free-view synthesis and accurate shape generation. Compared to 3D GANs, our diffusion-based approach naturally enables conditional generation such as masked completion or single-view 3D synthesis at inference time.
DiffRF: Rendering-guided 3D Radiance Field Diffusion (CVPR2023)World Reconstruction from Inconsistent ViewsTUM AI Lecture Series - Deep Surface Meshes (Pascal Fua)SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scans (CVPR)Deleted videoNeural Voice Puppetry: Audio-driven Facial Reenactment (ECCV 2020)IGNOR: Image-guided Neural Object Rendering (ICLR 2020)NPGA: Neural Parametric Gaussian Avatars (SIGGRAPH Asia24)ViewAL: Active Learning with Viewpoint Entropy for Semantic Segmentation (CVPR 2020)3DGS-LM: Faster Gaussian-Splatting Optimization with Levenberg-MarquardtClipFace: Text-guided Editing of Textured 3D Morphable Models (SIGGRAPH23)ICCV 2023 Paper Compilation - TUM Visual Computing Lab & Collaborators
Matthias Niessner |

DiffRF: Rendering-guided 3D Radiance Field Diffusion (CVPR'2023)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER