Uploaded November 2020 | Updated September 2026, 2 weeks ago
Join The Sound Of AI Slack community:
youtube.com/redirect?event=video_description&q=https%3A%2F%2Fjoin.slack.com%2Ft%2Fthesoundofai%2Fshared_invite%2Fzt-f71npumr-anli6W4QCuZ8UCj2gLoBkw&redir_token=QUFFLUhqbExxdFJ1S29mMVBKSFd0REZuUWFjUVpzQTFFUXxBQ3Jtc0ttWXZfTXpkRUlOdVNwU0poT1pmMkdkUzJFMkF4N19HSDVzbm5qUE43eV85bmwyNkFIdmlfbjVRTElhaUJIWmhfMDhJb1hQd1RaU1FmczNBWVA0ZjR6SXhiQ1JFV3owcURxc19KUmdlQU5KTzlmM0dxZw%3D%3D&v=KxRmbtJWUzI
In this video, you can get an understanding of the sound generation task, and learn to classify different types of sound generation systems.
I discuss the approaches and Deep Learning models used to generate sound. I also outline the challenges encountered with different methods, and discuss the features used to train generative sound systems.
Slides:
github.com/musikalkemist/generating-sound-with-neural-networks/blob/main/02%20Approaches%20and%20challenges/Sound%20Generation%20with%20Deep%20Learning%20Approaches%20and%20Challenges.pdf
Interested in hiring me as a consultant/freelancer?
valeriovelardo.com
Follow Valerio on Facebook:
facebook.com/TheSoundOfAI
Connect with Valerio on Linkedin:
linkedin.com/in/valeriovelardo
Follow Valerio on Twitter:
twitter.com/musikalkemist
===============================
Content:
0:00 Intro
0:33 Defining the sound generation task
1:17 Classification of sound generation systems
2:14 Types of generated sounds
3:41 Sound representations
4:07 Generation from raw audio
7:40 Challenges of raw audio generation
10:21 Generation from spectrograms
16:12 Advantages of generation from spectrograms
18:07 Challenges of generation from spectrograms
20:26 Can we generate sound with MFCCs?
21:26 DL architectures for sound generation
22:13 Inputs for generation
24:03 Details about the sound generative system we'll build
24:44 What's next?
===============================
Mentioned papers:
Wavenet: A Generative Model for Raw Audio:
arxiv.org/pdf/1609.03499.pdf
Jukebox: A Generative Model for Music
arxiv.org/pdf/2005.00341
DrumGAN: Synthesis of Drum Sounds with Timbral Feature Conditioning Using Generative Adversarial Networks
arxiv.org/pdf/2008.12073
Melnet: A generative model for audio in the frequency domain
arxiv.org/pdf/1906.01083.pdf
Join The Sound Of AI Slack community:
youtube.com/redirect?event=video_description&q=https%3A%2F%2Fjoin.slack.com%2Ft%2Fthesoundofai%2Fshared_invite%2Fzt-f71npumr-anli6W4QCuZ8UCj2gLoBkw&redir_token=QUFFLUhqbExxdFJ1S29mMVBKSFd0REZuUWFjUVpzQTFFUXxBQ3Jtc0ttWXZfTXpkRUlOdVNwU0poT1pmMkdkUzJFMkF4N19HSDVzbm5qUE43eV85bmwyNkFIdmlfbjVRTElhaUJIWmhfMDhJb1hQd1RaU1FmczNBWVA0ZjR6SXhiQ1JFV3owcURxc19KUmdlQU5KTzlmM0dxZw%3D%3D&v=KxRmbtJWUzI
In this video, you can get an understanding of the sound generation task, and learn to classify different types of sound generation systems.
I discuss the approaches and Deep Learning models used to generate sound. I also outline the challenges encountered with different methods, and discuss the features used to train generative sound systems.
Slides:
github.com/musikalkemist/generating-sound-with-neural-networks/blob/main/02%20Approaches%20and%20challenges/Sound%20Generation%20with%20Deep%20Learning%20Approaches%20and%20Challenges.pdf
Interested in hiring me as a consultant/freelancer?
valeriovelardo.com
Follow Valerio on Facebook:
facebook.com/TheSoundOfAI
Connect with Valerio on Linkedin:
linkedin.com/in/valeriovelardo
Follow Valerio on Twitter:
twitter.com/musikalkemist
===============================
Content:
0:00 Intro
0:33 Defining the sound generation task
1:17 Classification of sound generation systems
2:14 Types of generated sounds
3:41 Sound representations
4:07 Generation from raw audio
7:40 Challenges of raw audio generation
10:21 Generation from spectrograms
16:12 Advantages of generation from spectrograms
18:07 Challenges of generation from spectrograms
20:26 Can we generate sound with MFCCs?
21:26 DL architectures for sound generation
22:13 Inputs for generation
24:03 Details about the sound generative system we'll build
24:44 What's next?
===============================
Mentioned papers:
Wavenet: A Generative Model for Raw Audio:
arxiv.org/pdf/1609.03499.pdf
Jukebox: A Generative Model for Music
arxiv.org/pdf/2005.00341
DrumGAN: Synthesis of Drum Sounds with Timbral Feature Conditioning Using Generative Adversarial Networks
arxiv.org/pdf/2008.12073
Melnet: A generative model for audio in the frequency domain
arxiv.org/pdf/1906.01083.pdf
