PolyDiff: Generating 3D Polygonal Meshes with Diffusion Models @MatthiasNiessner
PolyDiff: Generating 3D Polygonal Meshes with Diffusion Models  @MatthiasNiessner
Uploaded December 2023 | Updated November 2026, 3 days ago
Paper: arxiv.org/abs/2312.11417

We introduce PolyDiff, the first diffusion-based model capable of directly generating realistic and diverse 3D polygonal meshes. In contrast to methods that use alternate 3D shape representations (e.g., implicit representations), our approach is a discrete denoising diffusion probabilistic model that operates natively on the mesh data structure. This enables the learning of both the geometric properties of vertices and the topological characteristics of faces. Specifically, we treat meshes as quantized triangle soups, progressively corrupted with categorical noise in the forward diffusion phase. A transformer-based denoiser is trained to revert this noise, restoring the original mesh structure. At inference, new meshes can be generated by applying this denoising network successively, starting with a completely noisy triangle soup. Consequently, our model is capable of producing high-quality 3D polygonal meshes, ready for integration into downstream 3D workflows. Our extensive experimental analysis shows that PolyDiff achieves significant improvements (avg. FID and JSD improvement of 18.2 and 5.8 respectively) over current state-of-the-art methods.
PolyDiff: Generating 3D Polygonal Meshes with Diffusion ModelsDeleted videoSceneCAD: Predicting Object Alignmentsand Layouts in RGB-D Scans (ECCV 2020)Deleted videoDeleted videoDeleted videoIntroduction to Deep Learning - 4. Optimization (Summer 2020)Deleted video3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans (CVPR 2019 Oral)Introduction to Deep Learning (I2DL 2023) - 10. ArchitecturesRC-MVSNet: Unsupervised Multi-View Stereo with Neural Rendering (ECCV22)Deleted video
Matthias Niessner |

PolyDiff: Generating 3D Polygonal Meshes with Diffusion Models

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER