[CVPR 2023] Unsupervised Continual Semantic Adaptation through Neural Rendering @aslteam
[CVPR 2023] Unsupervised Continual Semantic Adaptation through Neural Rendering  @aslteam
Uploaded June 2023 | Updated November 2026, 38 minutes ago
This is the accompanying video of our CVPR 2023 paper "Unsupervised Continual Semantic Adaptation through Neural Rendering".

Paper: arxiv.org/abs/2211.13969
Project page: ethz-asl.github.io/ucsa_neural_rendering
Code: github.com/ethz-asl/ucsa_neural_rendering

Abstract:
An increasing amount of applications rely on data-driven models that are deployed for perception tasks across a sequence of scenes. Due to the mismatch between training and deployment data, adapting the model on the new scenes is often crucial to obtain good performance. In this work, we study continual multi-scene adaptation for the task of semantic segmentation, assuming that no ground-truth labels are available during deployment and that performance on the previous scenes should be maintained. We propose training a Semantic-NeRF network for each scene by fusing the predictions of a segmentation model and then using the view-consistent rendered semantic labels as pseudo-labels to adapt the model. Through joint training with the segmentation model, the Semantic-NeRF model effectively enables 2D-3D knowledge transfer. Furthermore, due to its compact size, it can be stored in a long-term memory and subsequently used to render data from arbitrary viewpoints to reduce forgetting. We evaluate our approach on ScanNet, where we outperform both a voxel-based baseline and a state-of-the-art unsupervised domain adaptation method.

Reference:
Zhizheng Liu*, Francesco Milano*, Jonas Frey, Roland Siegwart, Hermann Blum and Cesar Cadena, "Unsupervised Continual Semantic Adaptation through Neural Rendering", IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023.

Affiliations:
Autonomous Systems Laboratory, ETH Zurich, Switzerland.
Robotic Systems Laboratory, ETH Zurich, Switzerland.
Max Planck ETH Center for Learning Systems, Switzerland.
[CVPR 2023] Unsupervised Continual Semantic Adaptation through Neural RenderingUnified States Estimation for a BallbotTSDF-based Change Detection for Long-Term Dense Reconstruction  and Dynamic Object DiscoverysenseSoar Solar Airplane Airborne!Hough2Map - Iterative Event-based Hough Transform for High-Speed Railway MappingV-Charge concept videoCooperating Robot for handling of dishes (1992)Voxblox: Incremental 3D Euclidean Signed Distance Fields for On-Board MAV PlanningROVIO: Robust Visual Inertial OdometryReshaping our model of the world over timeHuman - Robot Swarm Interaction for Entertainment: From animation display to gesture based controlFrom Perception to Decision: Deep End-to-end Motion Planning
aslteam |

[CVPR 2023] Unsupervised Continual Semantic Adaptation through Neural Rendering

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER