Uploaded March 2024 | Updated September 2026, 2 weeks ago
*** Part 2 is now available at youtu.be/qbAvOgGmFuE : Model Breadcrumbs, Model Stock, DELLA
Model merging is an increasingly popular technique that makes it possible to add or remove capabilities to transformer models, without the need for any additional training.
In this video, we first introduce what model merging is. Then, we discuss different merging algorithms implemented in the mergekit library (github.com/arcee-ai): model soups, SLERP, Task Arithmetic, TIES, DARE, and Franken-merging.
Slides: fr.slideshare.net/slideshow/julien-simon-deep-dive-model-merging/270921708
⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos. Follow me on Medium at julsimon.medium.com or Substack at https://julsimon.substack.com. ⭐️⭐️⭐️
00:00 Introduction
01:16 What is model merging?
07:10 Model soups
14:00 Spherical Linear Interpolation (SLERP)
20:35 Task Arithmetic
27:15 Trim, Extract Sign and Merge (TIES)
36:20 Drop and Rescale (DARE)
43:40 Franken-merging
*** Part 2 is now available at youtu.be/qbAvOgGmFuE : Model Breadcrumbs, Model Stock, DELLA
Model merging is an increasingly popular technique that makes it possible to add or remove capabilities to transformer models, without the need for any additional training.
In this video, we first introduce what model merging is. Then, we discuss different merging algorithms implemented in the mergekit library (github.com/arcee-ai): model soups, SLERP, Task Arithmetic, TIES, DARE, and Franken-merging.
Slides: fr.slideshare.net/slideshow/julien-simon-deep-dive-model-merging/270921708
⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos. Follow me on Medium at julsimon.medium.com or Substack at https://julsimon.substack.com. ⭐️⭐️⭐️
00:00 Introduction
01:16 What is model merging?
07:10 Model soups
14:00 Spherical Linear Interpolation (SLERP)
20:35 Task Arithmetic
27:15 Trim, Extract Sign and Merge (TIES)
36:20 Drop and Rescale (DARE)
43:40 Franken-merging
