DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data (CVPR 2020) @MatthiasNiessner
DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data (CVPR 2020)  @MatthiasNiessner
Uploaded December 2019 | Updated November 2026, 12 hours ago
CVPR 2020 Paper Video
Paper: arxiv.org/pdf/1912.04302.pdf
Dataset/Code: github.com/AljazBozic/DeepDeform
Benchmark: kaldir.vc.in.tum.de/deepdeform_benchmark


Applying data-driven approaches to non-rigid 3D reconstruction has been difficult, which we believe can be attributed to the lack of a large-scale training corpus. One recent approach proposes self-supervision based on non-rigid reconstruction. Unfortunately, this method fails for important cases such as highly non-rigid deformations. We first address this problem of lack of data by introducing a novel semi-supervised strategy to obtain dense interframe correspondences from a sparse set of annotations. This way, we obtain a large dataset of 400 scenes, over 390,000 RGB-D frames, and 2,537 densely aligned frame pairs; in addition, we provide a test set along with several metrics for evaluation. Based on this corpus, we introduce a data-driven non-rigid feature matching approach, which we integrate into an optimization-based reconstruction pipeline. Here, we propose a new neural network that operates on RGB-D frames, while maintaining robustness under large non-rigid deformations and producing accurate predictions. Our approach significantly outperforms both existing non-rigid reconstruction methods that do not use learned data terms, as well as learning-based approaches that only use self-supervision.
DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data (CVPR 2020)Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head AvatarsCVPR 2024 Paper Compilation - TUM Visual Computing Lab & CollaboratorsSynthesia: Native Dubbing DemoScan2CAD: Learning CAD Model Alignment in RGB-D Scans (CVPR 2019 Oral)Introduction to Deep Learning - 1. Introduction (Summer 2020)TUM AI Lecture Series - Computer Vision Startup Trends & Commercializing Research (Evan Nisselson)Deleted videoPanoptic Lifting for 3D Scene Understanding with Neural Fields (CVPR2023)GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view DiffusionDeleted videoECCV 2020 Paper Compilation - TUM Visual Computing Lab & Collaborators
Matthias Niessner |

DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data (CVPR 2020)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER