Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation (ECCV24) @MatthiasNiessner
Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation (ECCV24)  @MatthiasNiessner
Uploaded September 2024 | Updated November 2026, 1 hour ago
Project: terencecyj.github.io/projects/Mesh2NeRF
Paper: arxiv.org/pdf/2403.19319

We present Mesh2NeRF, an approach to derive ground-truth radiance fields from textured meshes for 3D generation tasks. Many 3D generative approaches represent 3D scenes as radiance fields for training. Their ground-truth radiance fields are usually fitted from multi-view renderings from a large-scale synthetic 3D dataset, which often results in artifacts due to occlusions or under-fitting issues. In Mesh2NeRF, we propose an analytic solution to directly obtain ground-truth radiance fields from 3D meshes, characterizing the density field with an occupancy function featuring a defined surface thickness, and determining view-dependent color through a reflection function considering both the mesh and environment lighting. Mesh2NeRF extracts accurate radiance fields which provides direct supervision for training generative NeRFs and single scene representation. We validate the effectiveness of Mesh2NeRF across various tasks, achieving a noteworthy 3.12dB improvement in PSNR for view synthesis in single scene representation on the ABO dataset, a 0.69 PSNR enhancement in the single-view conditional generation of ShapeNet Cars, and notably improved mesh extraction from NeRF in the unconditional generation of Objaverse Mugs.
Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation (ECCV24)CVPR 2020 Paper Compilation - TUM Visual Computing Lab & CollaboratorsScanRefer: 3D Object Localization in RGB-D Scans using Natural Language (ECCV 2020)FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human FacesGenRecon: Bridging Generative Priors for Multi-View 3D Scene ReconstructionPrEditor3D: Fast and Precise 3D Shape EditingIntroduction to Deep Learning - 5.Scaling Optimization and Stochastic Gradient Descent (Summer 2020)Matthias Niessner Live StreamL3DG: Latent 3D Gaussian Diffusion (SIGGRAPH Asia24)MeshGPT: Generating Triangle Meshes with Decoder-Only TransformersIMU2Face: Real-time Gesture-driven Facial ReenactmentDeleted video
Matthias Niessner |

Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation (ECCV'24)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER