Colin Raffel Explicit and Implicit Entropy Minimization in Proxy-Label-Based Semi-Supervised L... Learning with Limited and Imperfect Data 2026-09-23 | Colin Raffel Explicit and Implicit Entropy Minimization in Proxy-Label-Based Semi-Supervised L...
L2ID @ ECCV 2022 Introduction Learning with Limited and Imperfect Data 2022-10-22 | Workshop introduction by the organizers. Please note that live Q&A sessions have been cancelled. Post all questions directly to the YouTube videos in the comments section.
Boyi Li: SITTA: Single Image Texture Translation for Data Augmentation Learning with Limited and Imperfect Data 2022-10-19 | Recent advances in data augmentation enable one to translate images by learning the mapping between a source domain and a target domain. Existing methods tend to learn the distributions by training a model on a variety of datasets, with results evaluated largely in a subjective manner. Relatively few works in this area, however, study the potential use of image synthesis methods for recognition tasks. In this paper, we propose and explore the problem of image translation for data augmentation. We first propose a lightweight yet efficient model for translating texture to augment images based on a single input of source texture, allowing for fast training and testing, referred to as Single Image Texture Translation for data Augmentation (SITTA). Then we explore the use of augmented data in long-tailed and few-shot image classification tasks. We find the proposed augmentation method and workflow is capable of translating the texture of input data into a target domain, leading to consistently improved image recognition performance. Finally, we examine how SITTA and related image translation methods can provide a basis for a data-efficient, "augmentation engineering'' approach to model training.
Boshen Zhang: Learning from Noisy Labels with Coarse-to-Fine Sample Credibility Modeling Learning with Limited and Imperfect Data 2022-10-19 | Training deep neural network~(DNN) with noisy labels is practically challenging since inaccurate labels severely degrade the generalization ability of DNN. Previous efforts tend to handle part or full data in a unified denoising flow via identifying noisy data with a coarse small-loss criterion to mitigate the interference from noisy labels, ignoring the fact that the difficulties of noisy samples are different, thus a rigid and unified data selection pipeline cannot tackle this problem well . In this paper, we first propose a coarse-to-fine robust learning method called CREMA, to handle noisy data in a divide-and-conquer manner. In coarse-level, clean and noisy sets are firstly separated in terms of credibility in a statistical sense. Since it is practically impossible to categorize all noisy samples correctly, we further process them in a fine-grained manner via modeling the credibility of each sample. Specifically, for the clean set, we deliberately design a memory-based modulation scheme to dynamically adjust the contribution of each sample in terms of its historical credibility sequence during training, thus alleviating the effect from noisy samples incorrectly grouped into the clean set. Meanwhile, for samples categorized into the noisy set, a selective label update strategy is proposed to correct noisy labels while mitigating the problem of correction error. Extensive experiments are conducted on benchmarks of different modality, including image classification (CIFAR, Clothing1M etc) and text recognition (IMDB), with either synthetic or natural semantic noises, demonstrating the superiority and generality of CREMA.
Holger Caesar: Autonomous vehicles from imperfect and limited labels Learning with Limited and Imperfect Data 2022-10-19 | Abstract: The past decade has seen enormous progress in autonomous vehicle performance due to new sensors, large scale datasets and ever deeper models. Yet this progress is fueled by human annotators manually labeling every object in painstaking detail. Newly released datasets now focus on more specific subproblems rather than fully labelling ever larger amounts of data. In this talk I will talk about how we developed an Offline Perception system to autolabel a 250x times larger dataset called nuPlan. This dataset serves as the world's first real-world ML planning benchmark. By combining real-world data with a closed-loop simulation framework, we get the best of both world's - realism and reactivity. I will discuss the role of imperfect (perception) data in planning and prediction and highlight the importance of up-to-date maps. I conclude that it is essential to detect these imperfections, quantify their impact and develop robust models that are able to learn from this data.Bio: Dr. Holger Caesar is an Assistant Professor at the Intelligent Vehicles group of TU Delft in the Netherlands. Holger's research interests are in the area of Autonomous Vehicle perception and prediction, with a particular focus on scalability of learning and annotation approaches. Previously Holger was a Principal Research Scientist at an autonomous vehicle company called Motional (formerly nuTonomy). There he started 3 teams with 20+ members that focused on Data Annotation, Autolabeling and Data Mining. Holger also developed the influential autonomous driving datasets nuScenes and nuPlan and contributed to the commonly used PointPillars baseline for 3d object detection from lidar data. He received his PhD in Computer Vision from the University of Edinburgh in Scotland under Prof. Dr. Vittorio Ferrari and studied in Germany and Switzerland (KIT Karlsruhe, EPF Lausanne, ETH Zurich).
Sukmin Yun: OpenCoS: Contrastive Semi-supervised Learning for Handling Open-set Unlabeled Data Learning with Limited and Imperfect Data 2022-10-19 | Semi-supervised learning (SSL) has been a powerful strategy to incorporate few labels in learning better representations. In this paper, we focus on a practical scenario that one aims to apply SSL when unlabeled data may contain out-of-class samples - those that cannot have one-hot encoded labels from a closed-set of classes in label data, i.e., the unlabeled data is an open-set. Specifically, we introduce OpenCoS, a simple framework for handling this realistic semi-supervised learning scenario based upon a recent framework of self-supervised visual representation learning. We first observe that the out-of-class samples in the open-set unlabeled dataset can be identified effectively via self-supervised contrastive learning. Then, OpenCoS utilizes this information to overcome the failure modes in the existing state-of-the-art semi-supervised methods, by utilizing one-hot pseudo-labels and soft-labels for the identified in- and out-of-class unlabeled data, respectively. Our extensive experimental results show the effectiveness of OpenCoS under the presence of out-of-class samples, fixing up the state-of-the-art semi-supervised methods to be suitable for diverse scenarios involving open-set unlabeled data.
Yu Cheng: Towards data efficient vision-language (VL) models Learning with Limited and Imperfect Data 2022-10-19 | Abstract: Language transformers have shown remarkable performance on natural language understanding tasks. However, these gigantic VL models are hard to deploy for real-world applications due to their impractically huge model size and the requirement for downstream fine-tuning data. In this talk, I will first present FewVLM, a few-shot prompt-based learner on vision-language tasks. FewVLM is trained with both prefix language modeling and masked language modeling and utilizes simple prompts to improve zero/few-shot performance on VQA and image captioning. Then I will introduce Grounded-FewVLM, a new version that learns object grounding and localization in pre-training and can adapt to diverse grounding tasks. The models have been evaluated on various zero-/few-shot VL tasks and the results show that they consistently surpass the state-of-the-art few-shot methods. Bio: Yu Cheng is a Principal Researcher at Microsoft Research. Before joining Microsoft, he was a Research Staff Member at IBM Research & MIT-IBM Watson AI Lab. He got a Ph.D. degree from Northwestern University in 2015 and a bachelor’s degree from Tsinghua University in 2010. His research covers deep learning in general, with specific interests in model compression and efficiency, deep generative models, and adversarial robustness. Currently, he focuses on productionizing these techniques to solve challenging problems in CV, NLP, and Multimodal. Yu is serving (or, has served) as an area chair for CVPR, NeurIPS, AAAI, IJCAI, ACMMM, WACV, and ECCV.
SangYun Lee: Learning Multiple Probabilistic Degradation Generators for Unsupervised Superresolution Learning with Limited and Imperfect Data 2022-10-19 | Unsupervised real world super resolution (USR) aims to restore high-resolution (HR) images given low-resolution (LR) inputs, and its difficulty stems from the absence of paired dataset. One of the most common approaches is synthesizing noisy LR images using GANs (i.e., degradation generators) and utilizing a synthetic dataset to train the model in a supervised manner. Although the goal of training the degradation generator is to approximate the distribution of LR images given a HR image, previous works have heavily relied on the unrealistic assumption that the conditional distribution is a delta function and learned the deterministic mapping from the HR image to a LR image. In this paper, we show that we can improve the performance of USR models by relaxing the assumption and propose to train the probabilistic degradation generator. Our probabilistic degradation generator can be viewed as a deep hierarchical latent variable model and is more suitable for modeling the complex conditional distribution. We also reveal the notable connection with the noise injection of StyleGAN. Furthermore, we train multiple degradation generators to improve the mode coverage and apply collaborative learning for ease of training. We outperform several baselines on benchmark datasets in terms of PSNR and SSIM and demonstrate the robustness of our method on unseen data distribution.
Rabab Abdelfattah: PLMCL: Partial-Label Momentum Curriculum Learning for Multi-label Classification Learning with Limited and Imperfect Data 2022-10-19 | Multi-label image classification aims to predict all possible labels in an image. It is usually formulated as a partial-label learning problem, given the fact that it could be expensive in practice to annotate all labels in every training image. Existing works on partial-label learning focus on the case where each training image is annotated with only a subset of its labels. A special case is to annotate only one positive label in each training image. To further relieve the annotation burden and enhance the performance of the classifier, this paper proposes a new partial-label setting in which only a subset of the training images are labeled, each with only one positive label, while the rest of the training images remain unlabeled. To handle this new setting, we propose an end-to-end deep network, PLMCL (Partial Label Momentum Curriculum Learning), that can learn to produce confident pseudo labels for both partially-labeled and unlabeled training images. The novel momentum-based law updates soft pseudo labels on each training image with the consideration of the updating velocity of pseudo labels, which help avoid trapping to low-confidence local minimum, especially at the early stage of training in lack of both observed labels and confidence on pseudo labels. In addition, we present a confidence-aware scheduler to adaptively perform easy-to-hard learning for different labels. Extensive experiments demonstrate that our proposed PLMCL outperforms many state-of-the-art multi-label classification methods under various partial-label settings on three different datasets.
Nir Zabari: Open-Vocabulary Semantic Segmentation using Test-Time Distillation Learning with Limited and Imperfect Data 2022-10-19 | Semantic segmentation is a key computer vision task that has been actively researched for decades. In recent years, supervised methods have reached unprecedented accuracy; however, obtaining pixel-level annotation is very time-consuming and expensive. In this paper, we propose a novel open-vocabulary approach to creating semantic segmentation masks, without the need for training segmentation networks or seeing any segmentation masks. At test time, our method takes as input the image-level labels of the categories present in the image. We utilize a vision-language embedding model to create a rough segmentation map for each class via model interpretability methods and refine the maps using a test-time augmentation technique. The output of this stage provides pixel-level pseudo-labels, which are utilized by single-image segmentation techniques to obtain high-quality output segmentations. Our method is shown quantitatively and qualitatively to outperform methods that use a similar amount of supervision, and to be competitive with weakly-supervised semantic-segmentation techniques.
Niv Cohen: This is my unicorn, Fluffy: Personalizing frozen vision-language representations Learning with Limited and Imperfect Data 2022-10-19 | Large Vision & Language models pretrained on web-scale data provide representations that are invaluable for numerous V&L problems. However, it is unclear how they can be used for reasoning about user-specific visual concepts in unstructured language. This problem arises in multiple domains, from personalized image retrieval to personalized interaction with smart devices. We introduce a new weakly supervised learning setup called Personalized Vision & Language (PerVL) with two new benchmark datasets for retrieving and segmenting user-specific ("personalized") concepts. In PerVL, one should learn personalized concepts (1) independently of the downstream task (2) allowing a pretrained model to reason about them with free language, and (3) does not require personalized negative examples. We propose an architecture for solving PerVL that operates by extending the input vocabulary of a pretrained model with new word embeddings for the new personalized concepts. The model can then reason about them by simply using them in a sentence. We demonstrate that our approach learns personalized visual concepts from a few examples and can effectively apply them in image retrieval and semantic segmentation using rich textual queries.
Hamza Khan: Timestamp-Supervised Action Segmentation with Graph Convolutional Networks Learning with Limited and Imperfect Data 2022-10-19 | We introduce a novel approach for temporal activity segmentation with timestamp supervision. Our main contribution is a graph convolutional network, which is learned in an end-to-end manner to exploit both frame features and connections between neighboring frames to generate dense framewise labels from sparse timestamp labels. The generated dense framewise labels can then be used to train the segmentation model. In addition, we propose a framework for alternating learning of both the segmentation model and the graph convolutional model, which first initializes and then iteratively refines the learned models. Detailed experiments on four public datasets, including 50 Salads, GTEA, Breakfast, and Desktop Assembly, show that our method is superior to the multi-layer perceptron baseline, while performing on par with or better than the state of the art in temporal activity segmentation with timestamp supervision.
Jiageng Zhu: SW-VAE: Weakly Supervised Learn Disentangled Representation Via Latent Factor Swapping Learning with Limited and Imperfect Data 2022-10-19 | Representation disentanglement is one important goal of representation learning that benefits bunches of downstream tasks. To achieve this goal, many unsupervised learning representation disentanglement approaches have been developed. We propose a novel weakly-supervised training approach to train representation disentanglement called SW-VAE. We describe the approach of generating pairs of input observations by using the generative factors of datasets. The special pair introduce supervision signals in learning representation disentanglement. We also describe strategies to increase the difficulty of learning during training to make sure the model is well trained. By experimenting on different datasets, our model shows significant improvement over state-of-the-art methods on representation disentanglement tasks.
Niv Cohen: Out-of-Distribution Detection Without Class Labels Learning with Limited and Imperfect Data 2022-10-19 | Out-of-distribution detection seeks to identify novelties, samples that deviate from the norm. The task has been found to be quite challenging, particularly in the case where the normal data distribution consist of multiple semantic classes (e.g. multiple object categories). To overcome this challenge, current approaches require manual labeling of the normal images provided during training. In this work, we tackle multi-class novelty detection without class labels. Our simple but effective solution consists of two stages: we first discover "pseudo-class" labels using unsupervised clustering. Then using these pseudo-class labels, we are able to use standard supervised out-of-distribution detection methods. We verify the performance of our method by favorable comparison to the state-of-the-art, and provide extensive analysis and ablations.
Andong Tan: Unsupervised Adaptive Object Detection with Class Label Shift Weighted Local Features Learning with Limited and Imperfect Data 2022-10-19 | Due to the high transferability of features extracted from early layers (called local features), aligning marginal distributions of local features has achieved compelling results in unsupervised domain adaptive object detection. However, such marginal feature alignment suffers from the class label shift between source and target domains. Existing class label shift correction methods focus on image classification, and cannot be directly applied to object detection due to objects' co-occurrence. Meanwhile, one property of local features is that they have small receptive fields and can be easily mapped back to specific areas of input images. Therefore, to handle object co-occurrence scenarios, we propose to leverage this property to decompose the source feature maps and compute the source domain class distribution at the pixel level. The decomposition is based on each feature pixel's receptive field overlap with ground- truth bounding boxes. In the target domain, where no labels are available, we estimate this distribution using predicted bounding boxes and thus get the estimated class label shift between domains. This estimated shift is further used to re-weight source local features during the feature alignment. To the best of our knowledge, this is the first work trying to explicitly correct class label shift in unsupervised domain adaptive object detection. Experimental results demonstrate that this approach can systematically improve several recent domain adaptive object detectors, such as SW and HTCN on benchmark datasets with different degrees of class label shift.
Bharath Hariharan: When life gives you lemons: Making lemonade from limited labels Learning with Limited and Imperfect Data 2022-10-19 | Abstract: Many research directions have been proposed for dealing with the limited availability of labeled data in many domains, including zero-shot learning, few-shot learning, semi-supervised learning and self-supervised learning. However, I argue that in spite of the volume of research in these paradigms, existing approaches discard vital domain knowledge that can prove useful in learning. I will show two case studies where thinking about where the data comes from in the problem domain leads to substantial improvements in accuracy. The first case study will look at the domain of self-driving, and will show how leveraging domain knowledge can allow systems to automatically discover objects and train detectors with no labels at all. The second study will look at zero-shot learning, where digging deeper into the provenance of class descriptions yields surprising and useful insight. Bio: Bharath Hariharan is an assistant professor of Computer Science at Cornell University, where he works on all things computer vision, but focusing on problems where data challenges prevail. He is a recipient of the NSF CAREER award as well as the PAMI Young Researcher award.
Vadim Sushko: One-Shot Synthesis of Images and Segmentation Masks Learning with Limited and Imperfect Data 2022-10-19 | Existing GAN approaches for joint synthesis of images and segmentation masks are typically trained using large amounts of image data, which limits their utilization in restricted image domains. In this work, we take a step to reduce this limitation, introducing the task of one-shot image-mask synthesis: generate diverse images and their segmentation masks given only a single labelled example. Inspired by recent architectural developments of single-image GANs, we introduce our OSMIS model, which enables the synthesis of segmentation masks that are precisely aligned to generated images in the one-shot regime. Besides achieving a high fidelity of annotations, OSMIS outperforms existing single-image GAN models in image synthesis quality and diversity.
Leonid Karlinsky: Different facets of limited supervision Learning with Limited and Imperfect Data 2022-10-19 | Abstract: Limited Supervision can assume many interesting and practical forms beyond (the very popular) classical few-shot learning. In this talk I would touch upon four of our recent works covering a range of alternative limited supervision tasks. We will considered learning with weak supervision (incomplete or noisy labeling, such as image level class labels for training a few-shot detector or image level captions for training a zero-shot grounding model); coarse-to-fine few-shot learning – where pre-training annotations are coarse (e.g. broad vehicle types such as car, truck, bus, etc) while the target novel classes for few-shot learning are fine-grained (e.g. specific models of cars); self-supervised cross-domain learning – where we want to semantically align learned representations between different domains without any labels in any of the domains; and self-supervised classification – discovering novel classes without any supervision.Bio: Leonid Karlinsky is a Principal Research Scientist (STSM) in the MIT-IBM lab. Prior to that Leonid led the AI Vision research group in the Multimedia department @ IBM Research AI. Leonid joined IBM Research in July 2015. Before joining IBM, he served as a research scientist in Applied Materials, Elbit, and FDNA. He is actively publishing, reviewing, and performing occasional chair duties at ECCV, ICCV, CVPR, ICLR, AAAI, WACV, and NeurIPS, and served as an IMVC steering committee member for the past 6 years. During his time at IBM, Leonid has co-authored over 30 research papers in the areas of augmented reality, medical applications, self-supervised, cross-domain, multi-modal, and few-shot learning. He received his PhD degree at the Weizmann Institute of Science, supervised by Prof. Shimon Ullman.
Abhay Rawat: Semi-Supervised Domain Adaptation by Similarity based Pseudo-label Injection Learning with Limited and Imperfect Data 2022-10-19 | Semi-supervised Domain Adaptation (SSDA) assumes a partially labeled target domain, a more realistic scenario in many computer vision tasks. One of the primary challenges in SSDA is the skewed ratio between the number of labeled source and target samples, causing the model to be biased towards the source domain. Recent works in SSDA show that aligning only the labeled target samples with the source samples potentially leads to incomplete domain alignment of the target domain to the source domain. In our approach, to align the two domains, we leverage contrastive losses to learn a semantically meaningful and a domain agnostic feature space using the supervised samples from both domains. To mitigate challenges caused by the skewed label ratio, we pseudo-label the unlabeled target samples by comparing their feature representation to those of the labeled samples from both the source and target domains. Furthermore, to increase the support of the target do- main, these potentially noisy pseudo-labels are gradually injected into the labeled target dataset over the course of training. Specifically, we use a temperature scaled cosine similarity measure to assign a soft pseudo- label to the unlabeled target samples. Additionally, we compute an exponential moving average of the soft pseudo-labels for each unlabeled sample. These pseudo-labels are progressively injected (or removed) into the (from) the labeled target dataset based on a confidence threshold to supplement the alignment of the source and target distributions. Finally, we use a supervised contrastive loss on the labeled and pseudo- labeled datasets to align the source and target distributions. Using our proposed approach, we showcase state-of-the-art performance on SSDA benchmarks - Office-Home, DomainNet and Office-31.
Ruiwen Li: TransCAM: Transformer Attention-based CAM Refinement for Weakly Supervised Segmentation Learning with Limited and Imperfect Data 2022-10-19 | Weakly supervised semantic segmentation (WSSS) with only image-level supervision is a challenging task. Most existing methods exploit Class Activation Maps (CAM) to generate pixel-level pseudo labels for supervised training. However, due to the local receptive field of Convolution Neural Networks (CNN), CAM applied to CNNs often suffers from partial activation -- highlighting the most discriminative part instead of the entire object area. In order to capture both local features and global representations, the Conformer has been proposed to combine a visual transformer branch with a CNN branch. In this paper, we propose TransCAM, a Conformer-based solution to WSSS that explicitly leverages the attention weights from the transformer branch of the Conformer to refine the CAM generated from the CNN branch. TransCAM is motivated by our observation that attention weights from shallow transformer blocks are able to capture low-level spatial feature similarities while attention weights from deep transformer blocks capture high-level semantic context. Despite its simplicity, TransCAM achieves a competitive performance of 69.3% and 69.6% on the respective PASCAL VOC 2012 validation and test sets, showing the effectiveness of transformer attention-based refinement of CAM for WSSS.
Ishan Misra: General purpose visual recognition across modalities with limited supervision Learning with Limited and Imperfect Data 2022-10-19 | Abstract: Modern computer vision models are good at specialized tasks. Given the right architecture, right supervision, supervised learning can yield great specialist models. However, specialist models also have severe limitations — they can only do what they are trained for and require copious amounts of pristine supervision for it. In this talk, I’ll focus on two limitations: specialist models cannot work on tasks beyond what they saw training labels for, or on new types of visual data. I’ll present our recent efforts that design better architectures, training paradigms and loss functions to address these issues.Our first work, called Omnivore, presents a single model that can operate on images, videos, and single-view 3D data. Omnivore leads to shared representations across visual modalities, without using paired input data. Omnivore can also be trained in a self-supervised manner. I’ll conclude the talk with Detic, a simple way to train large-vocabulary detectors using image-level labs which leads to a 20,000+ class detector. Bio: Ishan Misra is a Research Scientist at FAIR, Meta AI where he works on computer vision. His interests are primarily in learning visual representations with limited supervision - using self-supervised, and weakly supervised learning. For his work in self-supervised learning, Ishan was features in MIT Tech Review’s list of 35 innovators under 35 compiled globally across all areas of technology. You can hear about his work at length on Lex Fridman’s podcast.
Sharon Li: How to Handle Data Shifts? Challenges, Research Progress and Path Forward Learning with Limited and Imperfect Data 2022-10-19 | Abstract: The real world is open and full of unknowns, presenting significant challenges for machine learning systems that must reliably handle diverse, and sometimes anomalous inputs. Out-of-distribution (OOD) uncertainty arises when a machine learning model sees a test-time input that differs from its training data, and thus should not be predicted by the model. As machine learning is used for more safety-critical domains, the ability to handle out-of-distribution data is central in building open-world learning systems. In this talk, I will talk about challenges, research progress, and future opportunities in detecting OOD samples for safe and reliable predictions in an open world.Bio: Sharon Yixuan Li is an Assistant Professor in the Department of Computer Sciences at the University of Wisconsin Madison. Her broad research interests are in deep learning and machine learning. Her research focuses on learning and inference under distributional shifts and open-world machine learning. Previously she was a postdoc research fellow in the Computer Science department at Stanford AI Lab. She completed her Ph.D. from Cornell University in 2017, where she was advised by John E. Hopcroft. She led the organization of the ICML workshop on Uncertainty and Robustness in Deep Learning in 2019 and 2020. She is the recipient of several awards, including the Facebook Research Award, Amazon Research Award, and was named Forbes 30Under30 in Science.
Xiuye Gu: Open-Vocabulary Detection and Segmentation Learning with Limited and Imperfect Data 2022-10-19 | Abstract: Existing visual recognition models often only work on the closed-set categories available in the training sets. In our recent work, we aim at going beyond this limitation. We design an open-vocabulary object detection method, ViLD and an open-vocabulary image segmentation model, OpenSeg, where the models detect objects or segment images with categories described by arbitrary texts. The two models address open-vocabulary recognition from two different perspectives: ViLD distills the knowledge from a pretrained open-vocabulary classification model (teacher) into a two-stage detector (student); OpenSeg learns the open-vocabulary capacity from weakly-supervised learning on image caption datasets, where the model learns visual-semantic alignments by aligning the words in a caption to predicted masks. Both models learn the localization ability from class-agnostic training on base categories using very different network architectures. ViLD achieves 26.3 APr and 27.6 AP on LVIS and COCO's novel categories respectively. It also directly transfers to other detection datasets without finetuning. Trained on COCO and Localized Narrative, OpenSeg directly transfers to Ade20k (847 and 150 categories), Pascal Context (459 and 59 categories) with superior performance. Bio: Xiuye is a research engineer at Google Research. Her research interests are in computer vision, with a current focus on open-vocabulary recognition. She was an AI resident at Google Research working with Tsung-Yi Lin and Yin Cui.Before that, she received her M.S. in Computer Science from Stanford University in 2020. She was a visiting scholar working with Prof. Yong Jae Lee. She received her B.E. in CS from Zhejiang University in 2017, where she worked with Prof. Deng Cai.
Yinfei Yang: Learning Visual and Vision-Language Model With Noisy Image Text Pairs Learning with Limited and Imperfect Data 2022-10-19 | Abstract: Pre-training has become a key tool in state-of-the-art computer vision and language machine learning models. The benefits of very large-scale supervised pre-training were first demonstrated by By Bert in the language community, and BiT and ViT models in the vision community. However, popular vision-language datasets like Conceptual Captions, MSCOCO usually involve a non-trivial data collection and cleaning process, which limits the size of datasets and hence limits the large scale training of image-text models. In recent work, researchers leverage the noisy dataset of over billions of image alt-text pairs mined from the Web as pre-training. The resulting models have shown incredible performance on various visual and vision language tasks, including image-text retrieval, captioning, visual question answering et.c. In addition, researchers also show that the visual representations learned from noise text supervision achieves the state-of-the-art level results on various vision tasks including image classification, semantic segmentation, object detection et.c. Bio: Yinfei Yang is a research manager at Apple AI/ML working on general visual intelligence. Previously he was a staff research scientist at Google research working on various NLP and Computer Vision problems. Before Google, He worked at Redfin and Amazon as research engineers for machine learning and computer vision problems. Prior to that, He was a graduate student in computer science at UPenn. He received his master's in computer science. His research focuses on image and text representation learning for retrieval and transferring tasks. He is generally interested in problems in Computer Vision, Natural Language Processing, or combined.
A Causal View of Compositional Zero Shot Recognition Learning with Limited and Imperfect Data 2021-06-21 | ...
Cluster-driven Graph Federated Learning over Multiple Domains Learning with Limited and Imperfect Data 2021-06-21 | ...
Rethinking Ensemble Distillation for Semantic Segmentation Based Unsupervised Domain Adaptation Learning with Limited and Imperfect Data 2021-06-21 | ...
ProFeat: Unsupervised Image Clustering via Progressive Feature Refinement Learning with Limited and Imperfect Data 2021-06-21 | ...
One Shot GAN learning to generate samples from Single Images and Videos Learning with Limited and Imperfect Data 2021-06-21 | ...
Unlocking the Full Potential of Small Datawith Diverse Supervision Learning with Limited and Imperfect Data 2021-06-21 | ...
A Simple Framework for Cross-Domain Few-Shot Recognition with Unlabeled Data Learning with Limited and Imperfect Data 2021-06-20 | ...
Weak Multi-View Supervision for Surface Mapping Estimation Learning with Limited and Imperfect Data 2021-06-20 | ...
ReMP Rectified Metric Propagation for Few Shot Learning Learning with Limited and Imperfect Data 2021-06-20 | ...
BalaGAN: Cross-Modal Image Translation Between Imbalanced Domains Learning with Limited and Imperfect Data 2021-06-20 | ...
Dina Katabi - Making Contrastive Learning Robust to Shortcuts and Generalize it to New Modalities Learning with Limited and Imperfect Data 2021-06-19 | ...
Sanja Fidler - Image GANs for Reducing Pixel-Wise Supervision Learning with Limited and Imperfect Data 2021-06-19 | ...
Efficient Pre trained Features and Recurrent Pseudo Labeling in Unsupervised Domain Adaptation Learning with Limited and Imperfect Data 2021-06-19 | ...
One shot action recognition in challenging therapy scenarios Learning with Limited and Imperfect Data 2021-06-19 | ...
Humphrey Shi - Escaping the Big Data Paradigm with Compact Transformers Learning with Limited and Imperfect Data 2021-06-19 | ...
Anurag Arnab - Video Understanding with Imperfect Data Learning with Limited and Imperfect Data 2021-06-19 | ...
Trevor Darrell - Recent Progress on Unsupervised Detection and Adaptation Learning with Limited and Imperfect Data 2021-06-19 | ...
Unsupervised Discriminative Embedding for Sub Action Learning in Complex Activities Learning with Limited and Imperfect Data 2021-06-18 | ...
Learning Unbiased Representations via Mutual Information Backpropagation Learning with Limited and Imperfect Data 2021-06-18 | ...
Boosting Co teaching with Compression Regularization for Label Noise Poster Learning with Limited and Imperfect Data 2021-06-18 | ...
Adversarial Open Domain Adaption for Sketch to Photo Synthesis Learning with Limited and Imperfect Data 2021-06-18 | ...
Batch Normalization Embeddings for Deep Domain Generalization Learning with Limited and Imperfect Data 2021-06-18 | ...
Improving Semi Supervised Domain Adaptation Using Effective Target Selection and Semantics Learning with Limited and Imperfect Data 2021-06-18 | ...
