Uploaded November 2024 | Updated September 2026, 2 weeks ago
Is it possible to train a #stablediffusion model for just $2,000? Vikash Sehwag, a research scientist from #SonyAI, presents his work on democratizing large-scale diffusion models in this new @BuzzRobot talk.
Vikash introduces a novel patch masking technique paired with a lightweight patch-mixer, which significantly lowers computational costs while maintaining high performance. He also explores how #datasets choices and input masking impact diffusion transformers and #convolutionalneuralnetworks highlighting the critical role of architectural design in enabling efficient, scalable training.
Timestamps:
0:00 Introduction
2:10 Democratizing diffusion training at scale
5:20 Stable Diffusion - 1.5, Stable Diffusion - 2.1, MicroDiT
6:55 Factors contributing to cost reduction: model architecture, training setup and training datasets
7:45 Training a latent/stable diffusion model
9:12 Reducing FLOPs per image
10:16 CNNs vs Transformers
11:18 Challenge with input masking
13:48 Architecture of patch-mixer
16:20 Training setup: LLM training
21:02 Two stage training pipeline
24:06 Composition of training data
25:45 Advantage of real and synthetic data combined
27:47 Quantitative comparison of perfomance
28:54 Key takeaways
32:26 Q&A
#stablediffusion #stablediffusiontutorial #stablediffusionprompts #artificialintelligence #largelanguagemodels #llms #machinelearning #deeplearning #neuralnetworks #technology #tech #techtalk #techtalks #aitalks #aitalk #science #python #pythonprogramming #ai #programming #coding #softwareengineering #softwaredevelopment #softwareengineer #softwaredeveloper
Social Links:
Newsletter: buzzrobot.substack.com
X: https://x.com/sopharicks
Slack: join.slack.com/t/buzzrobot/shared_invite/zt-2s067rv7n-guPIMGe62rbp9ncxdnOUfQ
Is it possible to train a #stablediffusion model for just $2,000? Vikash Sehwag, a research scientist from #SonyAI, presents his work on democratizing large-scale diffusion models in this new @BuzzRobot talk.
Vikash introduces a novel patch masking technique paired with a lightweight patch-mixer, which significantly lowers computational costs while maintaining high performance. He also explores how #datasets choices and input masking impact diffusion transformers and #convolutionalneuralnetworks highlighting the critical role of architectural design in enabling efficient, scalable training.
Timestamps:
0:00 Introduction
2:10 Democratizing diffusion training at scale
5:20 Stable Diffusion - 1.5, Stable Diffusion - 2.1, MicroDiT
6:55 Factors contributing to cost reduction: model architecture, training setup and training datasets
7:45 Training a latent/stable diffusion model
9:12 Reducing FLOPs per image
10:16 CNNs vs Transformers
11:18 Challenge with input masking
13:48 Architecture of patch-mixer
16:20 Training setup: LLM training
21:02 Two stage training pipeline
24:06 Composition of training data
25:45 Advantage of real and synthetic data combined
27:47 Quantitative comparison of perfomance
28:54 Key takeaways
32:26 Q&A
#stablediffusion #stablediffusiontutorial #stablediffusionprompts #artificialintelligence #largelanguagemodels #llms #machinelearning #deeplearning #neuralnetworks #technology #tech #techtalk #techtalks #aitalks #aitalk #science #python #pythonprogramming #ai #programming #coding #softwareengineering #softwaredevelopment #softwareengineer #softwaredeveloper
Social Links:
Newsletter: buzzrobot.substack.com
X: https://x.com/sopharicks
Slack: join.slack.com/t/buzzrobot/shared_invite/zt-2s067rv7n-guPIMGe62rbp9ncxdnOUfQ
