Uploaded September 2024 | Updated September 2026, 1 week ago
To try everything Brilliant has to offer—free—for a full 30 days, visit brilliant.org/bycloud/. You’ll also get 20% off an annual premium subscription.
my newsletter:
mail.bycloud.ai
What is model distillation? Well today I will explain the concept and how it is applied to make SoTA cheaper for people to access. Model with names like Turbo, fast, lightning, are usually processed with distillation. It can also be used in some other interesting aspects, eg. step distillation or hide your model structures.
Special thanks to Cross Product & Neggles for helping with this video
This video is supported by the kind Patrons & YouTube Members:
🙏Andrew Lescelius, alex j, Chris LeDoux, Alex Maurice, Miguilim, Deagan, FiFaŁ, Robert Zawiasa, Owen Ingraham, Daddy Wen, Tony Jimenez, Panther Modern, Jake Disco, Demilson Quintao, Penumbraa, Shuhong Chen, Hongbo Men, happi nyuu nyaa, Carol Lo, Mose Sakashita, Miguel, Bandera, Gennaro Schiano, gunwoo, Ravid Freedman, Mert Seftali, Mrityunjay, Richárd Nagyfi, Timo Steiner, Henrik G Sundt, projectAnthony, Brigham Hall, Kyle Hudson, Kalila, Jef Come, Jvari Williams, Tien Tien, BIll Mangrum, owned, Janne Kytölä, SO, Richárd Nagyfi, Hector, Drexon, Claxvii 177th, Inferencer, Michael Brenner, Akkusativ, Oleg Wock, FantomBloth, Thipok Tham, Clayton Ford, Theo, Handenon, Diego Silva, mayssam
[Discord] discord.gg/NhJZGtH
[Twitter] twitter.com/bycloudai
[Patreon] patreon.com/bycloud
[Music] Massobeats - daydream
[Profile & Banner Art] twitter.com/pygm7
[Video Editor] @Askejm
To try everything Brilliant has to offer—free—for a full 30 days, visit brilliant.org/bycloud/. You’ll also get 20% off an annual premium subscription.
my newsletter:
mail.bycloud.ai
What is model distillation? Well today I will explain the concept and how it is applied to make SoTA cheaper for people to access. Model with names like Turbo, fast, lightning, are usually processed with distillation. It can also be used in some other interesting aspects, eg. step distillation or hide your model structures.
Special thanks to Cross Product & Neggles for helping with this video
This video is supported by the kind Patrons & YouTube Members:
🙏Andrew Lescelius, alex j, Chris LeDoux, Alex Maurice, Miguilim, Deagan, FiFaŁ, Robert Zawiasa, Owen Ingraham, Daddy Wen, Tony Jimenez, Panther Modern, Jake Disco, Demilson Quintao, Penumbraa, Shuhong Chen, Hongbo Men, happi nyuu nyaa, Carol Lo, Mose Sakashita, Miguel, Bandera, Gennaro Schiano, gunwoo, Ravid Freedman, Mert Seftali, Mrityunjay, Richárd Nagyfi, Timo Steiner, Henrik G Sundt, projectAnthony, Brigham Hall, Kyle Hudson, Kalila, Jef Come, Jvari Williams, Tien Tien, BIll Mangrum, owned, Janne Kytölä, SO, Richárd Nagyfi, Hector, Drexon, Claxvii 177th, Inferencer, Michael Brenner, Akkusativ, Oleg Wock, FantomBloth, Thipok Tham, Clayton Ford, Theo, Handenon, Diego Silva, mayssam
[Discord] discord.gg/NhJZGtH
[Twitter] twitter.com/bycloudai
[Patreon] patreon.com/bycloud
[Music] Massobeats - daydream
[Profile & Banner Art] twitter.com/pygm7
[Video Editor] @Askejm
![DeepSeeks Insane Architecture Breakthrough [Engram Explained]](https://i.ytimg.com/vi/xUlX6jvwVfM/mqdefault.jpg "DeepSeeks Insane Architecture Breakthrough [Engram Explained]
Stop scraping manually and check out SerpApi now!
https://serpapi.link/bycloud-march-2026
In this video, we will breakdown how DeepSeeks new engram works intuitively, with visualizations!
Check out my latest project: Intuitive AI Academy
We just wrote a new piece on MoE and Distillation!
https://intuitiveai.academy/
limited time code EARLY for 40% off yearly plan!
My Newsletter
https://mail.bycloud.ai/
my project: find, discover & explain AI research semantically
https://findmypapers.ai/
My Patreon
https://www.patreon.com/c/bycloud
DeepSeek Engram
[Paper] https://arxiv.org/abs/2601.07372v1
mentioned papers
[DeepSeek-V3.2] https://arxiv.org/abs/2512.02556
[DeepSeek Sparse Attention] https://github.com/deepseek-ai/DeepSeek-V3.2-Exp/blob/main/DeepSeek_V3_2.pdf
Try out my new fav place to learn how to code https://scrimba.com/?via=bycloudAI
This video is supported by the kind Patrons & YouTube Members:
🙏Spam Maj, Alex, Chris LeDoux, DX Research Group, Poof N Inu, Deagan, Robert Zawiasa, Ryszard Warzocha, Tobe2d, Louis Muk, Akkusativ, Kevin Tai, Mark Buckler, NO U, Tony Jimenez, Ângelo Fonseca, jiye, Anushka, Asad Dhamani, Binnie Yiu, Calvin Yan, Clayton Ford, Diego Silva, Etrotta, Gonzalo Fidalgo, Handenon, Hector, Jake Disco very, Michael Brenner, Nilly K, OlegWock, Daddy Wen, Shuhong Chen, Sid_Cipher, Stefan Lorenz, Sup, tantan assawade, Thipok Tham, Thomas Di Martino, Thomas Lin, Richárd Nagyfi, Paperboy, mika, Leo, Berhane-Meskel, Kadhai Pesalam, mayssam, Bill Mangrum, nyaa, Toru Mon, Lame Plane, Matej Macak, Len Mo, saylikhapekar
[Discord] https://discord.gg/NhJZGtH
[Twitter] https://twitter.com/bycloudai
[Patreon] https://www.patreon.com/bycloud
[Business Inquiries] bycloud@smoothmedia.co
[Music] @IraStoria
[Profile & Banner Art] https://twitter.com/pygm7
[Video Editor] Abhay
[Ko-fi] https://ko-fi.com/bycloudai")
![We had Image Gen copying LLM... and now the REVERSE?? [DiffusionLM]](https://i.ytimg.com/vi/yXHSPzHfe1s/mqdefault.jpg "We had Image Gen copying LLM... and now the REVERSE?? [DiffusionLM]
To try everything Brilliant has to offer—free—for a full 30 days, visit https://brilliant.org/bycloud/ . You’ll also get 20% off an annual premium subscription.
In 2023, image generation adopted LLMs transformer architecture to improve generation quality. Now, in 2024, the Diffusion Language Model, also known as DiffusionLM, is taking a step towards using diffusion for language modeling. Look how the turn tables...
my newsletter
https://mail.bycloud.ai
Check out my Patreon for the latest diffusionLM papers mapped out
https://www.patreon.com/bycloud
Structured Denoising Diffusion Models in Discrete State-Spaces
[Paper] https://arxiv.org/abs/2107.03006
Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution
[Paper] https://arxiv.org/abs/2310.16834
Scaling up Masked Diffusion Models on Text
[Paper] https://arxiv.org/abs/2410.18514
Discrete Diffusion for Complex Reasoning and Planning
[Paper] https://arxiv.org/abs/2410.14157
Simplified and Generalized Masked Diffusion for Discrete Data
[Paper] https://arxiv.org/abs/2406.04329
Speculative Diffusion Decoding
[Paper] https://arxiv.org/abs/2408.05636
Energy-Based Diffusion Language Models for Text Generation
[Paper] https://arxiv.org/abs/2410.21357
This video is supported by the kind Patrons & YouTube Members:
🙏Andrew Lescelius, Ben Shaener, Chris LeDoux, Miguilim, Deagan, FiFaŁ, Robert Zawiasa, Marcelo Ferreira, Owen Ingraham, Daddy Wen, Tony Jimenez, Panther Modern, Jake Disco, Demilson Quintao, Penumbraa, Shuhong Chen, Hongbo Men, happi nyuu nyaa, Carol Lo, Mose Sakashita, Miguel, Bandera, Gennaro Schiano, gunwoo, Ravid Freedman, Mert Seftali, Mrityunjay, Richárd Nagyfi, Timo Steiner, Henrik G Sundt, projectAnthony, Brigham Hall, Kyle Hudson, Kalila, Jef Come, Jvari Williams, Tien Tien, BIll Mangrum, owned, Janne Kytölä, SO, Richárd Nagyfi, Hector, Drexon, Claxvii 177th, Inferencer, Michael Brenner, Akkusativ, Oleg Wock, FantomBloth, Thipok Tham, Clayton Ford, Theo, Handenon, Diego Silva, mayssam, Kadhai Pesalam, Tim Schulz, jiye, Anushka, Henrik Sundt, Julian Aßmann, Raffay Rana
[Discord] https://discord.gg/NhJZGtH
[Twitter] https://twitter.com/bycloudai
[Patreon] https://www.patreon.com/bycloud
[Profile & Banner Art] https://twitter.com/pygm7
[Video Editor] @Askejm
[Bitcoin (BTC)] 3JFMJQVGXNA2HJE5V9qCwLiqy6wHY9Vhdx
[Ethereum (ETH)] 0x3d784F55E0bE5f35c1566B2E014598C0f354f190
[Litecoin (LTC)] MGHnqALjyU2W6NuJSSW9fTWV4dcHfwHZd7
[Bitcoin Cash (BCH)] 1LkyGfzHxnSfqMF8tN7ZGDwUTyBB6vcii9
[Solana (SOL)] 6XyMCEdVhtxJQRjMKgUJaySL8cGoBPzzA2NPDMPfVkKN
[Ko-fi] https://ko-fi.com/bycloudai")
