Modeling 3D Shapes by Reinforcement Learning (ECCV 2020) @MatthiasNiessner
Modeling 3D Shapes by Reinforcement Learning (ECCV 2020)  @MatthiasNiessner
Uploaded March 2020 | Updated November 2026, 1 day ago
Project: arxiv.org/pdf/2003.12397.pdf

We explore how to enable machines to model 3D shapes like human modelers using reinforcement learning (RL). In 3D modeling software like Maya, a modeler usually creates a mesh model in two steps: (1) approximating the shape using a set of primitives; (2) editing the meshes of the primitives to create detailed geometry. Inspired by such artist-based modeling, we propose a two-step neural framework based on RL to learn 3D modeling policies. By taking actions and collecting rewards in an interactive environment, the agents first learn to parse a target shape into primitives and then to edit the geometry. To effectively train the modeling agents, we introduce a novel training algorithm that combines heuristic policy, imitation learning and reinforcement learning. Our experiments show that the agents can learn good policies to produce regular and structure-aware mesh models, which demonstrates the feasibility and effectiveness of the proposed RL framework.
Modeling 3D Shapes by Reinforcement Learning (ECCV 2020)TUM AI Lecture Series - Explainability and Compositionality for Visual Recognition (Zeynep Akata)Face Anything: 4D Face Reconstruction from Any Image SequenceDPHMs: Diffusion Parametric Head Models for Depth-based TrackingHyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion (ICCV2023)FaceForensics++: Learning to Detect Manipulated Facial Images (ICCV 2019)Introduction to Deep Learning (I2DL 2023) - 6. Training Neural NetworksBecomingLit: Relightable Gaussian Avatars with Hybrid Neural ShadingIntroduction to Deep Learning - 11. Recurrent Neural Networks (Summer 2020)TUM AI Lecture Series - I2DL Guest Lecture 2023 - 2D Priors for 3D Generation  (Ben Poole)TUM AI Lecture Series - A Question of Representation in 3D Computer Vision (Bharath Hariharan)3D Scanning & Motion Capture: 4. Optimization
Matthias Niessner |

Modeling 3D Shapes by Reinforcement Learning (ECCV 2020)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER