Uploaded June 2025 | Updated November 2026, 1 hour ago
Giannis Daras, Massachusetts Institute of Technology: Generative models have found a plethora of applications in Machine Learning, and various other scientific and applied fields, where they are used for sampling complex, high-dimensional distributions and are leveraged in downstream analyses involving such distributions. It is well-understood that the quality of these models depends on the size and quality of the data they are trained on, however creating large-scale, high-quality datasets is often expensive and sometimes impossible or even undesirable. Indeed, in many scientific domains we have no access to high-quality data due to physical or instrumentation constraints. In other applications, curating high-quality datasets is resource-intensive, e.g. finding the three-dimensional structure of proteins is a slow and expensive process, and producing high-quality MRI scans requires keeping subjects for a longer time in MRI machines. Finally, some of the most prominent deep generative models have been shown to memorize their training data, which they can be manipulated to reveal. Is it possible to train deep generative models to generate high-quality samples given a training set of samples that have been corrupted due to some underlying physical process or to protect sensitive information? And is it possible to train models given a training set of samples with heterogeneous quality? We develop such methods for denoising diffusion models, one of the most prominent classes of deep generative models. We illustrate the performance of our method in various domains, including obtaining state-of-the-art protein folding models.
Giannis Daras, Massachusetts Institute of Technology: Generative models have found a plethora of applications in Machine Learning, and various other scientific and applied fields, where they are used for sampling complex, high-dimensional distributions and are leveraged in downstream analyses involving such distributions. It is well-understood that the quality of these models depends on the size and quality of the data they are trained on, however creating large-scale, high-quality datasets is often expensive and sometimes impossible or even undesirable. Indeed, in many scientific domains we have no access to high-quality data due to physical or instrumentation constraints. In other applications, curating high-quality datasets is resource-intensive, e.g. finding the three-dimensional structure of proteins is a slow and expensive process, and producing high-quality MRI scans requires keeping subjects for a longer time in MRI machines. Finally, some of the most prominent deep generative models have been shown to memorize their training data, which they can be manipulated to reveal. Is it possible to train deep generative models to generate high-quality samples given a training set of samples that have been corrupted due to some underlying physical process or to protect sensitive information? And is it possible to train models given a training set of samples with heterogeneous quality? We develop such methods for denoising diffusion models, one of the most prominent classes of deep generative models. We illustrate the performance of our method in various domains, including obtaining state-of-the-art protein folding models.










