Uploaded November 2024 | Updated September 2026, 2 weeks ago

---

Instead of manipulating synthesizers directly (through something like a keyboard), what if we could leverage the richness of sound itself as an input?

This is what Jordie Shier (Queen Mary University of London) explores with his AI-powered 808 synthesizer, which uses neural networks to grab information from drum hits and map them onto a drum synthesizer in real time.

It's an excellent learning experience for those wandering the boundaries between music and AI, as he describes his entire pipeline and thought process.

0:00 Intro
1:41 Real-time control of audio synthesis
2:50 Demo of harmonic synthesis
3:36 DDSP general overview
5:29 What about drums?
7:32 Timbre analogies
9:04 Acoustic drum mapping
10:17 Designing and implementing into a plugin
15:28 Feature loss difference
17:52 Designing the system for real-time
21:07 UI design
21:24 Engineering notes
22:50 DDSP challenges
24:15 DDSP resources
25:10 End of presentation
25:27 Audience questions
35:20 Outro

====

📚 Learn Audio Programming
theaudioprogrammer.com/books

🚀 Join our Community
theaudioprogrammer.com/community

🤝🏻 Hire Audio Programmers & Music Tech Professionals
theaudioprogrammer.com/recruitment

🎛️ Create Your Own Audio Plug-in with Our Experienced Team
theaudioprogrammer.com/development

Instagram: instagram.com/theaudioprogrammer
Facebook: facebook.com/theaudioprogrammer
Twitter: twitter.com/audioprogrammer