Uploaded June 2024 | Updated September 2026, 1 week ago
Become a Patreon: patreon.com/theaiepiphany
π¨βπ©βπ§βπ¦ Join our Discord community: discord.gg/peBrCpheKE
Albert & Karan joined us to talk state space models. They are the authors of the seminal S4 paper, Albert is the main author of thee Mamba paper - and together they're pioneering the SSM research and competing against transformer supremacy! :)
Recently they posted super cool demos demonstrating just how much more real-time these models are compared to the baselines.
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
check out Cartesia AI: cartesia.ai
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
βοΈ Timetable:
00:00:00 - 00:01:56 Intro
00:01:56 - 00:03:37 Hyperstack GPUs! (sponsored)
00:03:37 - 00:09:45 Cartesia AI, real-time intelligence
00:09:45 - 00:12:05 text-to-speech demo
00:12:05 - 00:36:35 state space models (SSMs)
00:36:35 - 00:38:30 bitter lessons transformers vs SSMs
00:38:30 - 00:41:30 free lunch? what are the actual tradeoffs?
00:41:30 - 00:42:55 Computing SSMs convolutionally
00:42:55 - 00:50:00 What is the state (RNNs, Attention, Conv, Neural ODEs)
00:50:00 - 00:55:30 Mamba 2, discrete vs continuous spectrum
00:55:30 - 01:03:39 Q&A: scaling laws
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
π° SPONSOR
The AI Epiphany - patreon.com/theaiepiphany
One-time donation - paypal.com/paypalme/theaiepiphany
Huge thank you to these AI Epiphany patreons:
Eli Mahler
Petar VeliΔkoviΔ
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
πΌ LinkedIn - linkedin.com/in/aleksagordic
π¦ Twitter - twitter.com/gordic_aleksa
π¨βπ©βπ§βπ¦ Discord - discord.gg/peBrCpheKE
πΊ YouTube - youtube.com/c/TheAIEpiphany
π Medium - gordicaleksa.medium.com
π» GitHub - github.com/gordicaleksa
π’ AI Newsletter - aiepiphany.substack.com
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
#statespacemodels #ssms #transformer #albertgu #karangoel #cartesiaai
Become a Patreon: patreon.com/theaiepiphany
π¨βπ©βπ§βπ¦ Join our Discord community: discord.gg/peBrCpheKE
Albert & Karan joined us to talk state space models. They are the authors of the seminal S4 paper, Albert is the main author of thee Mamba paper - and together they're pioneering the SSM research and competing against transformer supremacy! :)
Recently they posted super cool demos demonstrating just how much more real-time these models are compared to the baselines.
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
check out Cartesia AI: cartesia.ai
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
βοΈ Timetable:
00:00:00 - 00:01:56 Intro
00:01:56 - 00:03:37 Hyperstack GPUs! (sponsored)
00:03:37 - 00:09:45 Cartesia AI, real-time intelligence
00:09:45 - 00:12:05 text-to-speech demo
00:12:05 - 00:36:35 state space models (SSMs)
00:36:35 - 00:38:30 bitter lessons transformers vs SSMs
00:38:30 - 00:41:30 free lunch? what are the actual tradeoffs?
00:41:30 - 00:42:55 Computing SSMs convolutionally
00:42:55 - 00:50:00 What is the state (RNNs, Attention, Conv, Neural ODEs)
00:50:00 - 00:55:30 Mamba 2, discrete vs continuous spectrum
00:55:30 - 01:03:39 Q&A: scaling laws
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
π° SPONSOR
The AI Epiphany - patreon.com/theaiepiphany
One-time donation - paypal.com/paypalme/theaiepiphany
Huge thank you to these AI Epiphany patreons:
Eli Mahler
Petar VeliΔkoviΔ
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
πΌ LinkedIn - linkedin.com/in/aleksagordic
π¦ Twitter - twitter.com/gordic_aleksa
π¨βπ©βπ§βπ¦ Discord - discord.gg/peBrCpheKE
πΊ YouTube - youtube.com/c/TheAIEpiphany
π Medium - gordicaleksa.medium.com
π» GitHub - github.com/gordicaleksa
π’ AI Newsletter - aiepiphany.substack.com
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
#statespacemodels #ssms #transformer #albertgu #karangoel #cartesiaai
